This fight seemed impossible without the perfect setup and I don't understand the purpose of the 2 npcs helping you, when they are only lv1 and almost do nothing at all.

In the end my mate and I cheesed this fight. We found a greatsword in the hidden pirate treasure which was irreparable and was way stronger than a normal greatsword, it had around 420-440 damage. So we fully buffed my Elf Paladin who then proceeded to 3 hit kill the Bishop/Worm with 600+ damage hits and 1 hit kill everything else.

But after that fight it got really messy and buggy. After all enemies were killed, Gareth rejoined our party, but was hostile towards 3/4 of our party members (everyone but my Elf Paladin), so I had to kill my own party members to end that fight. After that the story continued as normal.