No there is a path going down left exiting the abandoned fort. If you go straight out of that it walks you into the purple spore mist and the minotaurs. If you go left down the ramp it triggers the bullette. If you are sneaking you can avoid triggering the minotaurs agro even if you are not in the cone of visibility (as they agro just a bit before the red cone intersects with you, so sometimes they see you at the purple mushroom spore mist).
(just realized you said goblin fort, which is what I just described)....
The very first time I ran into the mino's i jumped a character down the pit at the phase spiders because I accidently won by pushing the matriarch down the hole and wanted to see if i could go down there and find a body to loot. Imagine my surprise on finding the underdark!!! From there I was able to jump off the side next to the flamable mushrooms and sneak out, again you can avoid a fight from that approach....I am not sure of any other ways to get to the underdark at the moment. Goblins or the spider cave.
