It always feels to me like the party is significantly weakened if you don't have a cleric, fighter and rogue. But since I'm playing a wizard, it leaves little scope for taking Gale or Wyll on my adventures. If I do, locked chests and doors are usually what gets left behind. And having somebody with decent nature abilities is almost out of the question.
I know it'd take a fair bit of rebalancing, but I'd love to be able to take a party of 6.