My inventory is filled with books but I have no where to store them yet (was guessing you could store stuff in the temple at some point) so I was curious if they were worth the pain in keeping them all. I have read them all and a couple could pertain to quest's but for the most part they are the informative / lore ones.
Just sell them, they would end up in your tower storage and start to collect dust in your coffer, unless you like the idea having all the books there is no real point of keeping them.