We found the sweetspot ourselves as 3 seasoned gamer friends.
Play with 2 friends using a 3 player party. Playing 2xlone wolf is powerful, playing with 4 people on classic is not too hard either. However 3 player party disallows lone wolf and causes a -1 max player handicap which is basically 25% less life, phys armor, magic armor and damage.
We find the game very challenging as 3 players and having tons of fun, sometimes having to reload a boss fight many times or having to invent some really out-of-the-box tactics.