This discussion has been done to death.
You're also incorrect about it pushing you to all one damage type. a 2 magic, 2 physical party works just fine. You do NOT need to damage BOTH an enemy's armor types, you go for the lowest one with your characters who specialize in that type of damage. The only time that matters is when you've killed so many enemies that you have no weak targets left, at which point the fight should be pretty much done.
The only combinations this limits are 3/1 builds (either kind) for obvious reasons.
