I strongly, STRONGLY disagree with having lore books give you direct stat bonuses. The information in the books should be reward enough for reading them. It'd be far better if these books actually provided some useful insight and knowledge that could be applicable in future situations.
Reading them could unlock certain dialogue options in future conversations, e.g. reading a note with instructions to do something signed by "T.L.", and you later encounter someone with the initials T.L., and an insight check could make that connection and unlock the hidden dialogue option.
Now I like your idea of the unlocking of a hidden dialogue option much better and to me would make it much more lore friendly
