As I understand it, 4 players is considered the "normal" size for a D&D 5e party. I was interested to read from some older material on how to run a game, from around 2000 or so, that they talked about 5-7 being a normal party size for whatever version of D&D was around at the time (I think it was written for AD&D 2e). And I'd heard that Baldur's Gate 2 has a party limit of 6.

To be honest, whatever the limit is, people would be asking for it to be lifted. I figure having a standard limit in the game for what the game's designed and balanced for, and allowing people to lift that either with mods or even an option in the menus, is best. Make sure people know they're nor playing as intended, but hey - if you like it, have fun!