There's a secret entrance out of the basement, which also needs to be cleared. That's the way they're supposed to escape I bet.
But even after I cleared that route, the prisoners wouldn't leave, still complaining about it not being safe.
I gave up at some point and consider it to be bugged.