Read the book just named "Book" in his library. It opens that up.
What about those 4 patches covered by sunlight in his garden? They click when you walk through them. Is this some other puzzle?
Honestly, I have no idea how that one works. I used barrels to hold them all down, but nothing seemed to happen. Felt like it was bugged or maybe just meant to mess with you...
:\