Larian was never that good at balancing. The first game had the same issue, that the late game got less and less challenging, probably because some skills are so game breaking strong, that you can't really balance the game. (Not to mention all the possibilities to cheese the fights)
You put 90% dodge aura on enemie to make combat more challenging, players teleports aura guy far away and the whole challenge is gone.
The first part of Act 1 is only so difficult, because AI has high stats and skills advantage.
