I'd rather not have a larger party because combat takes long enough as it is. Fewer party members and fewer opponents means less time doing combat = more playthroughs = being able to bring along a different set of companions on the 2nd run rather than running with the same guys because I had room for everyone the first time.
Not a single claim in this post is based on reality, especially looking at other games that did it (like the recent Kingmaker mod/addition to make the game turn-based).
For one, no one is asking for "more enemies". They could just be the same amount and slightly stronger. Second, more party members make "controlling and cleaning up the area" quicker. Third, the "speed" of combat is a relative and fairly low priority concern in a game where each encounter is an unique, non-repeatable event. Fourth, "being able to bring every time a different set of companions" doesn't work that well when you want to cover certain almost-fundamental roles and your slots are very limited to begin with.