I do agree that right now skills lean heavily towards INT, which maybe should be balanced more. It should also perhaps be more clear what things scale off of, as many Warfare (or all?) spells scale with your weapon alignment (str, fin, or int).
Right now, with the given armor system, it's actually advantageous to have a party focus on 1 primary damage type (magic or physical) to quick get rid of armor and apply your 100% chance CC (since resistance appears to have no affect on CC chance). Building 3 physical with 1 magic (for buffs, heals, environment, and some damage) works a lot better than a balanced team of damage. I'm sure the opposite is also true (magic heavy).
Do non-damaging spells scale with an attribute?
What you are saying about armor is interesting. And it even more hurts multi-classing. Maybe this could be balanced if almost every enemy had at least minor armors of both types AND reducing both types of armor to 0 made enemies (and players) more vulnerable or otherwise debuffed.