She can also easily take out two PC's in melee in a single turn if she teleports in melee range. So why doesn't she do this right away?

This is another example why combat in BG3 sucks. It's about gimmicks rather than character builds and abilities.

Larian designs overpowered monsters, and then creates a gimmick you can exploit to win. I really despise this design. And it only works once. Replaying these encounters when you already know the gimmick to beat them is dull. Just like once you figure out how easy and OP "backstabs" and high ground are, you use them every turn all the time.