Very basic calculation:

hp = vitality
ma = magical armor
pa = physical armor
ma = magical damage
pd = physical damage

Suppose you have a balanced team 1:
2 hp 1 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter
2 hp 0 pa 1 ma 1 md mage
2 hp 0 pa 1 ma 1 md mage

Against a purely physical team 2:
2 hp 1 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter

Assuming simultaneous turns:
In the first turn, team 1 can deal 2 magical damage and 2 physical damage. In the best case, through focus fire it can take down 1 fighter and reduce another fighter's hp to 1.
Team 2 can deal 4 physical damage, it can take down 2 mages. So you end up with:

Team 1:
2 hp 1 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter

Team 2:
1 hp 0 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter
2 hp 1 pa 0 ma 1 pd fighter

Team 2 is clearly at a advantage.

Of course it is a very simplified model and does not take other factors into account, but it should help explain the problem a bit.