There isn't a way to turn it off, but you CAN unlock your skillbar and remove things form it, then relock it.
As for the unrelated '125 skill slots', I'm pretty sure you gain memory slots every few levels even without memory investment, and memory investment is a 1/1 thing for stat per slot, so between items, equipment, powers gained by using OTHER spells (The ram horn thingy and flying, for example) you're likely to use up a good few by mid game.