If you want to leave with no violence then the easiest (if you don't find a tactical retreat scroll to let the ladder down) is to go through the fire-slug cave. You can sneak from there out of the prison without talking to let alone fighting anyone.
The problem is you will then be in the marshes at level 3 with shit gear. You'll need to gain some XP (go and see the dragon) and get some cash (do the maze) before you can go back to the fort finish them off.
