Not easy but i did it.
As i left the Temple entrance i made a left, there is a ramp that goes down, it seems to always spawn there. I split the group, put both of my mage and warlock up top above the Ramp and took my warrior and cleric down right up against him.
From there is just nuke him, stay close with melee, his jump does way too much damage. Actually it ran away the very first time and went into the ground at 17 HP, then i found him again somewhere else and finished him off.