Sounds like your still early in the game, try attacking from a distance with bows, or, if you can manage it, lead 1 of the men away from the other group so you can tackle him on his own.
Make sure you have plenty of healing potions, and take them till you thin them down to 1 or 2.
If you have a summoning doll, send that in on its own, and keep re-using it until you weaken/kill some of them
You could try luring them into some traps if you have any in your inventory.
Have you got any spare skill points you could put into increasing your health?
