I'd like to start a playthrough with two lone wolves, one who focuses on Fire/Water & the other Air/Earth. The problem I'm mostly seeing with this is that I know there are some bosses and mobs that can only really be hurt by non-magical means.

I really don't want to build a "battle mage" style character as part of the reasoning for doing this is to avoid spreading my points/talents/abilities too thin. Is there any way to build around this issue that doesn't require putting a ton of points into physical weapon stats/skills?