The only way I have found that offers me any modicum of control is using two hands.
One controls the mouse and the other deals with grouping/ungrouping, selecting individual members (F1 F2 etc), centering on the character/group and camera control.
I've gotten pretty decent at it but the system is so bad I still suffer occasional glitches (characters mistakenly trying to pickpocket party members while sneaking etc). I never climb ladders as a group nor jump as a group.
It's all very tedious but it is the only way to feel in control.