I'd so much rather have a party select screen that tells me I can only have up to X number of party members. I switch people in and out. Done. No unnecessary convos or "Well, if that's what you think is best. I mean, I question your judgment...". No thanks. Just quick select and done.
Well, much as Lae’zel’s “Dismiss your weakest warrior” line makes me giggle, I do largely agree with this. I’d at least prefer it if we could ask the fifth person to join, they’d agree, and then the game forced us to pick one of the party members (including the new one) to send to/stay at camp, rather than us having to manually remove someone first.
For me, that’s the minimum. I do think there are plenty of other potential improvements to the way we can manage our party at camp, but that’s off topic here and covered elsewhere so I’ll not try to derail us.