It is hitting the gong (after some required quests) in the council hall that triggers the one way trip, but you can go to the hall and still complete current quests and areas.