I think a party of 4 is based mostly off what they're comfortable with. Given the Divinity games which also had a party of 4. It would be easier for them to work with in terms of UI and combat balance.
Personally, I find it somewhat annoying to have to keep dismissing companions in camp just to access buffs or crafting of other characters... But hey, at least it isn't Divinity 2 where any non-party members are permenantly killed and thus never accessible (I literally used a mod for party size just to bypass this particular event, even if my first playthrough was just me and my waifu (And her demon))
In terms of having higher party limit, it would take more work in order to balance it (Be it a limit of 6 or uncapped), both for regular play and also when attempting to allow for Dark Urge type solo play (Bearing in mind that Divinity specifically also had the Lone Wolf skill to help enable solo/duo play so even a party of 4 is taking more effort to balance without such power enhancements)
The amount of work and overall gameplay feel likely played a large part in their decision to limit parties to 4 members, much like their decision to cap levels off at 12 instead of 20.
It's interesting to note that other similar games such as Pillars of Eternity and Pathfinder have a party limit of 5. So maybe Larian could have pushed the limit up to 5? But honestly, at the end of the day, I'd rather they stick to their comfort zone and provide a solid game, than to potentially sacrifice quality to chase down a particular party size.
So yeah, would it be nice if we could have our entire camp come with us on our adventure all the way to level 20? Sure. But I'd rather the game release this century too!
