Some books are just for atmosphere, some contain hints, etc, and some provide a history of various groups or regions.
Farmer Jackson's diary is in the quest item section of your inventory and will remain there, since you can not get rid of it (some quest items are removed after the quest is done, some are not). Items in the quest section are not counted towards your inventory capacity, though (nor are a couple pieces of equipment that are quest related, which you can not sell or destroy).