just make source abilities cost no memory, u have already a pretty rare resource needed to use them, no reason to waste memory there.
Also make that every 2 points in one shool add 1 spell slot for that kind of spells. if u do so u can have an average spell pool with a for example full pyro mage. and at the same time if u wanna do a jack of all trades type u still have to sacrifice intelligence to amplify ur memory.
