The only way I have found that offers me any modicum of control is using two hands.

One controls the mouse and the other deals with grouping/ungrouping, selecting individual members (F1 F2 etc), centering on the character/group and camera control.

I've gotten pretty decent at it but the system is so bad I still suffer occasional glitches (characters mistakenly trying to pickpocket party members while sneaking etc). I never climb ladders as a group nor jump as a group.

It's all very tedious but it is the only way to feel in control.