A party of 4 takes away a lot of the choice or roleplaying because if forces you to have the tanked fighter, spell caster, thief, cleric
It definitely doesn't limit you to anything - none of those are necessary in BG3, it ain't BG2.
In generally I would welcome an extra spot or two in the party, however, the battles already feel overpopulated at times, and I wouldn't want extra 1/3 of enemies added.
