I was in the same boat the OP until I read about not mixing Magic and Physical damage. The game went from frustrating where I was cheesing every fight to kinda easy. I was also surprised at how nasty some of the posts here are so I thought I'd share too.
I do miss having a balanced party for loot drops, play style and feel of a traditional RPG group. I have magic users with high finesse and strength so I can use gear drops, and running around with 4 wand / shield spell casters is plain goofy.
I love the game, but I am surprised that systems as big as armor were not balanced better.