Until the totem is in the circle and activated, you should be able to click and drag it; after that you can not move it or pick it up again.