What if higher level skills take more memory slots (even more so than they currently do), but having more points in that school reduces memory cost?
For example:
Chain Lightning takes 5 memory slots at Aerothurge level 1. Each additional point in Aerothurge reduces the memory cost of Chain Lightning by 1 to a minimum of 1 or 2.
So, someone who is a jack of all trades can only memorize a few powerful skills because they don't have a lot of investment in any particular school and each skill costs a lot. Whereas someone with 5 points in Aerothurge can learn a ton of Aerothurge spells with relative ease, but might have a harder time remembering skills from other schools.