Easiest method I've done is
just throw an Invisibility potion on him when you open the door, but before the cutscene. The kamikaze summons won't even move from that spot and you can just run the Duke straight to the ladder.