I understood what you were saying the first time. I don't think strictly locking down the skill bar is the best either. If your mage has 6 fire spells on his belt, and 6 ice skills which he has in his spellbook (but not on the bar), he's not going to be pleased if he wanders into an encounter with fire elementals and can't switch out anything.
I don't want a tedious interface either, but I'd take it in a heartbeat over too-easy combat. I am okay with swapping out skills, and having multiple skillbars but it can't for free in combat.
I debate how tedious it would be to change out the inventory. You shouldn't need to put scrolls and potions on your skillbar anyway: this is not an action RPG, there's no difference using those things from the inventory and skillbar.