Once you have read that type of book (that are not quest related) they can be sold or destroyed. I hung on to a few unique books (diaries, etc), but sold any randomly generated books that can show up more than once (the various 'history of' series, etc).
Thanks. I'm surprised at the vast amount of books I've already found, and they really seem to serve no purpose. That will be a lot of free space!

What about Farmer Jackson's Diary? I already spoke to him/killed him, then spoke to Captain Rodney and showed it to him, but it's still in my inventory.