I do think there should be more of a benefit for investing heavily into certain skills; as of now the slight percentage increase feels too inconsequential per point, even if they may eventually add up to a significant bonus.

However, specialization seems to come from your memory restrictions rather than your point investment. You can learn skills from every school, but unless you have an abundance of memory slots, you're probably only going to be able to focus on one or two of them in your "deck".