So long as the difficulty is exactly the same, which I'm afraid won't be possible. And that it's a hard toggle, and no way to recruit past 4 without it on.
Because if you can, you have to. That's just how it works sometimes.
But why is this Larian's problem? Isn't this your problem? You are choosing to make the game easier for you by taking six, in exactly the same way as those who are choosing to make the game harder for them by taking only one. You cannot legitimately complain about the game becoming easier by saying you are somehow "compelled" to take six. No one from Larian showed up at your home, put a gun to your head, and made you do it.
