The main thing I dislike about limited party size is how it makes party composition incredibly boring. You are excluding a lot of accessory flavor because you want at least to cover a minimum of variety in your limited amount of slots.
The irony of the entire situation is that if they addressed both the "party size" concern and my other grief about how the party controls (dedicated thread here: http://forums.larian.com/ubbthreads.php?ubb=showflat&Number=679414#Post679414 ) they'd be automatically defusing 90% of the complaints about this not feeling like a proper Baldur's Gate. They just don't seem to care much about that.
Well, we are probably wasting our breath here, since Larian, with all its upsides, is almost NOTORIOUS for being stubborn and sticking to their poor design decision no matter how much feedback they get against it (the obscene randomized loot system of their past games or the massively criticized armor system in DOS 2 come to mind).