Do you mean the north west gate, behind the King Crab? If so, there are no doors there (a clue that that gate would be the easiest to leave by). The gate west of the harbour should have doors.