There is a book, The Codex of Pestilential Thought, which gives +1 Intelligence and -1 Perception. I don't know if there is any way to reverse that (short of the
save editor, on PC, or CheatEngine).
Deals with demons can also result in an attribute penalty, but it sounds like you found a codex.