Once you are strong enough....meteorstrike over and over and over....he folded against that for my mage. But yeah, if you need, level up a few more times and go back for him. I don't think I tackled the vampire until about level 15-20...somewhere in there.
