Just for fun and adding atmosphere. There are 3 special books though you get a skill from by finding them.
And you should keep books given to you: Worshippers, Malcolm's account book (found), summoning book.
Kiya