Did you find the 'dungeon' in town? There are a couple entrances inside the town, and one just outside.
Dig up the largest pile of dirt in the graveyard.
Leave the cave for now, and look for smaller groups or lower level opponents.
Did you pick up a couple companions in town, if you are not playing characters with the Lone Wolf talent?