You can't base party size on encounter balance. Why? Because party size is already too varied. From the very beginning of the game, the first encounter, you could have the MC, Lae'zel and Us OR just you and Lae'zel OR you, three other players, Lae'zel and Us, or you, three other players and Lae'zel. It depends on who you have playing. Is it single-player mode or multi?
This is it exactly! Party size is already something that can vary within the game, by Larian's own design! So arguing against the option of a party size of six based on encounter design is obviously an invalid argument.
Wrong, it actually a proof of how party size affects difficulty in encounters. Killing commander Zhalk with just Lae'zel is extremely difficult, with Lae'zel, US and Shadowheart it's a piece of cake.