Instead of triggering the initial event where you first encounter Dallis the Hammer I choose to take the ladder on the side of the structure. Dallis and Bishop Alexandar will both stay in place so long as you don't escape Fort Joy.
Both Dallis and Alexandar will only go to 0 HP. It seems if you bring Dallis to 0 she triggers and they will both exit on her next available turn. If you bring Alexandar to 0 he will heal back up but dropping him again triggers the Dallis/Alex exit.
The fight was possible with the blood rose elixir exploit and the insane power of a buffed Warlord character.
https://www.youtube.com/watch?v=PKwL7p3xK_s&feature=youtu.be