Congrats!
Just to add my two cents, I didn't find it all that hard to kill him either, but in my game he never summoned anything to fight me. My arrows never did any direct damage, but I had a lot of points in piercing, and just whittled away at him with piercing damage until he died. He must be immune to poison, as I had a lot of points in that as well, and don't remember him ever turning green. Only used about 20 potions as far as I remember, but he missed me a lot because I had high avoidance. When he did hit me it hurt a lot, though. At least 1/4 of my health with every hit. Anyway, sorry to hear others aren't having such an easy time of it.