I'm going to write something extremely controversial, but once the player figures out how different builds and combat scenarios work it's really difficult to make the game harder. And honestly, there is a line when Larian's responsibility stops when it comes to game difficulty.
Most of the suggestions on this thread are things that can be "added" to the game by personalizing everyone's own game style. (restricting rests, not using certain gear and, not min-maxing points etc). If Larian were to add a nightmare difficulty option, it would still be too easy for some.
