I totally agree with OP.

Sometimes I am trying to make the team jump across and god forbid you click back on a character. They all re-arrange and block your path again.

Stop it!