Luck does play a big part in it. I beat them with a level 3 party but then in another playthrough my level 4 party got trashed. The initial initiative rolls are big and the enemies might miss or hit on their first turns. Having some high dexterity characters that go before most of the enemy units is really helpful and someone with a really high AC so they miss.

And I like that there are a ton of options for this encounter.