One of the books explains that Halsin saves them from the gnolls. He guided them to the grove presumably, as Sozz said, down by the river where you can't cross but it is assumed others can because Scratch and his master Gromwick likely went that way while escaping the gnolls.
It is also implied that the goblins have only recently claimed Bogrot for themselves. The tieflings arrived 2 months before the game began, so it is possible the goblins weren't in control of the ruined village at the time.
