Yeah, its my main issue - I will still want to use most of my skills like rain, but instead of just casting it, i will have to open skill screen, relearn it, cast it and relearn it again...

Same with tough encounters. Just press F5 every 10 sec, and when you get into fight, look what spells you will need, reload, assign them, win the fight and assign back your previous skills.

Tedious, not fun, and balancing could still suffer, as you will always have all the skills you need in that fight ready after reload.