what difficulty are you playing on? have you escaped the actual fort yet? explored the rest of the island?

i don't even return to do this fight until i'm well beyond their level because of how much there is to do in the first act