At the mine, where you get a cutscene as you approach and are warned not to enter (the opponents outside may be fine, but are a much higher level inside the tower building), follow the main path to the left (north), probably back the way you came, and you should easily be able to see the path to the bandit's hideout curving off to the right. If you are approaching from the north, there is a hill blocking the view, and if you are focusing on opponents, it is not terribly difficult to go right past the side-path.
Welcome to the forum.
