I think that if you can use all skills and spells you have in your spellbook all the time, that the game will be difficult to balance. One aspect of RPGs like this for me was to prepare my group for certain encounters. If i can change around everything in mid combat this aspect is voided to a degree.
I think 8-16 Spell/skill slots + 4 other slots for items, usables should be enough.
We will get a third character the companion which will also bring skills to battle. I think this would be enough.
