realistically you will never need more than a 3 in most spell schools.

also, due to resists, you will generally want a mage type to be an 'elementalist' and at least have basic options to deal with various enemy resistances.

personally i would go with a 3, maybe a 4 if really needed, in all 4 elements then at least 2 loremaster boosted to 5 with gear so you can see enemy resists. you can do this for around 40 of your skill points. with lone wolf and no books you end up with about 68 points, so very feasible.

if you really want those end-game spells, and most suck, just deal with 2AP penalty on cast. you can use them at most once per fight and with glass cannon you will be ok.