I suggest removing the ability to attack party members via their portraits. It happens to many times that I accidentally have character X attack character Y when I wish to control character Y.
The easy solution would be that pressing on a portrait always selects that character, while targeting the character model "targets" the character for whatever reason (spells, attack etc.).
Alternatively, make it a toggle under options.