Actually, it wasn't my idea; somebody else found this. I briefly tried to see if he could be attacked, but didn't think to talk to him repeatedly to get him to turn hostile (the first couple of quests you can deliberately annoy him, but I don't recall being able to do so repeatedly, and didn't try after getting the quest for the flowers).
