Sounds like you're dragging the item back as if your mouse button didn't let go