I think the OP's issue is that it's easy to inadvertently scroll while you're trying to do something else. Increasing the scroll delay a bit would help.