Disable the Dynamic Camera to stop that (also quite annoying!).
Gyson - thanks for the helpful suggestions, hadn't thought of the last two. That said these are more "workarounds" than real solutions, I think the devs should just stop the fidgeting when characters are hovered or find some other satisfactory solution.