Yeah, the UI could (should) be better to manage a larger group.
The real issue here is that "managing a larger group" doesn't really seem to be in Larian's plans.
In general, especially IF (and that's a BIG IF) Larian goes back to its questionable decision "You'll have to commit to three companions and say goodbye to everyone else else after the first act", the camp/central hub should be the place where a player goes to quickly sort inventory for everyone, pick a different formation for a different questline/ combat encounter, etc.
