My guess, they probably made their way down to the river, crossed near the bugbear cave, and were spotted and aided by the druids. The toll booth was ransacked by goblins first a little bit ago, then the gnolls moved in, so there might have been a gap that allowed for their caravan to escape their patrol.
This is all speculation, I don't think it's unbelievable for refugees to rough it off-road.