I actually like the 4 party member limit. Bigger parties will slowdown combat (which is already an issue), increase the time spent managing all party members (this is a wide issue I have where I try to create my own character and 'connect' with it but I spend just as much effort managing all other characters) and will make every other party member less valuable. Just a personal preference, but I feel 4 is just fine.
I also like the fact that I might not have all bases covered with my party. If I don't have a rouge I can't do rouge things. Great. It adds to the story, force me to make certain choices. The limit number of characters actually adds to the verity of plays I will have over time.
Of course I can always choose to only take 4 characters (even if the games allows for 6) but I would bet the game would be balanced and design for a full party. It's not the perfect solution, but I can live with it if it's important to other people.