I still think 5 players would be fine. D&D game, mostly, have a warrior/tank, a rogue to pick locks/ more dps, a wizard, and a healer. That's the basic meta party. If my main character is a mage, then who do I leave behind when heading into the goblin fort with Wyll (cause of his quest)? In my first playthrough I left Astarion behind... just meant I had to go through it all again with him for the locks and stuff. Just seems kind of arbitrarily annoying. Just one extra spot would be enough, and shouldn't require THAT much of a difficulty tweak to make it work.