I think I read somewhere there might be a "crisis point" locking your party like in Divinity: Original Sin 2 which I could understand why they might want to limit the party, but that just made me always play someone with an origin (eg Lohse or Sebille), rather than a player-created character, unless I was maybe doing a multiplayer campaign with some of my friends. I would enjoy a larger party size, especially since I'd like to eventually play a bard, ranger, or maybe even a druid eventually, but when you have to look at frontline/healer/dps composition, 4 doesn't give a lot of flexibility right now, until maybe more classes are added, like paladin or something. Ultimately I suppose it depends on gameplay and story and how Larian feels about it, but I'd personally try to work harder to accomplish something if it means I get to keep more of my companions.