Magic missiles solve the problem of hag's illusions (and I have 2 casters, gale and astarion as arcane trickster), and astarion is fast enough to save Mayrina in first turn. Shadowheart was only casting Guiding Bolts, and my ranger (I took battle master's skill at 4th lvl) frightened Ethel, so the second turn was the last one for her.
Although I do think because BG3 is lacking the Shield spell, magic missiles are much more powerful than they should be all the time.
And fear shouldn’t provoke fleeing and cause an opportunity attack. In fact frightened creatures seem to not do anything other than run. Not that this is the reason you killed her but just something I noticed in gameplay.
I would assume they are using the fear spell, which says that a target frightened by the spell MUST take the dash action and move away from the caster on each of its turns unless it can't move. The frightened condition is closer to what you describe in that it gives the frightened target disadvantage on ability checks and attack rolls while in the presence of what frightened them and they cannot move closer to what frightened them.