Teleporting him out of the group if it even would work, wouldn't help as in the End he would still remain and you can't kill him.

Physical Aoe also doesn't work


But I have to admit that I didn'nt bother much as i just solved it per a conversation option.