Yes, this is very annoying. If you aren't careful, then your character will move instead of attacking or attack an unintended target.