If you pick it up then drop it it's no longer a stolen book.
There's another issue with reading books that I find punishing: when you close the book it closes your inventory, too. So you have to open your inventory screen again to read another book. Say you pick up 3 books and start reading them, you have to keep re-opening your inventory screen each time you close a book. I'd rather that just stay open if I'm reading the book from my inventory.
