If you go pure magic or pure physical the game will be easy.
3 DPS (2hander/Rogue or Ranger) and 1 supportive Geo/Hydro is probably the best four character party there is. Its unkillable, and obliterates everything easily.
You can get a similar effect from going four mages, although iv found mages are not nearly as good at DPS until late Act 2.
This armor system makes it so having a hybrid party simply isnt as good as a pure party. So if you find it easy you should probably take a hybrid to just make it harder.