As far as I'm concerned, the more party members the better (within reason). True, it can slow the pace of the game having to control more than one character, but the payoff is that combat becomes more tactically interesting.

But more importantly for me, joinable party members improve the story potential immensely IMO. As an example, the plot for the Neverwinter Nights official campaign was a huge letdown compared to that of Baldurs Gate 2, largely due to the lack of interesting party members (NWN's henchmen didn't count because they had no impact on the story, and their interaction with the player's character was minimal). A good story needs a good recurring cast of characters, and this is most easily accomplished in a party-based game.