It seems, there's no satisfying way for a decent character.
I tried to stop him, but all the Droughs side with him instantly and they make short work of my party. Especially the sergeant deals a ridiculous amount of damage.
So I sided with him, which I didn't want to do. He got pushed into the lava, so I didn't even get his loot and still, I had to fight the whole dwelling. My party barely survived.
I tried it with my warlock, Shadowheart, Karlach and Tristarion. All level 5.