I figured it was bugged so I reloaded and gave the evil overlord back his soul.
It feels like that part never got finished.
There's dialogue options, the skeletons spawn, but then he just disappears. The quest doesn't update either so you don't know that you failed it, it seems like it is still active.
But entering and re-entering does nothing. Nor does using his soul stone thing./