But is it?
What if the attacker is male and the female is a cleric of Lolth? Hell any female? Most males aren't Zaknafein and he even he kept up an outward veneer of respect..
Even away from that sort of set up, a Drow wouldn't attack on sight unless it was damn sure it would win. In many cases, even confidence of a win wouldn't be enough, they would also want to be sure of leaving no witnesses (that matter at least).
The Lolth loyal Drow absolutely are cruel and vicious but they are also scheming and very far from stupid or rash. This encounter could definitely do with some work from a lore/world building perspective..
For this specific encounter it wouldn't matter I believe. The petrified Drow have no idea if you are involved in their state, they would only remember up to the point of petrification, not anything during, so they definitely would attack. Even if this did not happen, I am fairly sure most Drow encountering them would at least let them become weakened against the Spectator before trying anything, any survivor could possibly be questioned. Stepping in to help, even if female, would be strange to the others. Not to mention most people tend to have a full party, traveling with non Drow would make you look really suspicious since they clearly aren't slaves.