Seems to be a common issue, though I've yet to see anything officially acknowledging it. I've posted on Steam forums, as well as here ( http://larian.com/forums/ubbthreads.php?ubb=showflat&Number=634445&#Post634445 ).
In my case, playing multiplayer - host does not have the issue if they take control of my character/summons.
Also - if you find yourself faced with a bug where you cannot unmemorize a skill in your skill book, try removing the action from your actionbar and it should then allow you to unlearn it.