I guess you forgot something important: P-skills mostly scale from weapons too, therefore finding a good weapon can give you a bigger boost than mages will ever get.
Also physical was always more of damage focused, while magic was more about applying effect, which will get totally denied as long armor is left.
