That's true! I can see a situation where you're facing a clump of enemies with high physical armor and low magic armor. Your mage whittles down the magic armor, so you have your melee guy switch to a staff, then charge in and deal some damage instead of bashing his head against their physical armor.
The only problem I see with that is the skill, if I'm right, is entirely based upon weapon damage. So, for example, if you've invested in strength, the intelligence based staff won't be as effective.
