After getting my ass kicked a bunch of times, I found a quick and easy way of dispatching the bastard.
Level 1, Polymorph Spell... I turned him into a frog and then he hopped off and disappeared into a wall. From there he made his way to the axe room where I found him again before he teleported away.
Then I was free to kill his minions and collect the items I needed without even breaking a sweat.
