A specific place has been named, although the reference does not require it at all. This is where the reasoning begins.
The Drow are tightly tied to the slave trade. Given early idea - it could have been a reference to Astarion's past. He could have done business with the Drow.
Given that he says "the Drow could have learned", in the scene with the mushroom servants, he could have known this first hand, their method and considered insufficiently effective.
Perhaps the whole Astarion story is a reference on the original story of Astarion.
Menzoberranzan has a trading quarter where other races are allowed to make deals. It is recommended to have guards and guides from the locals, pay for protection. Only the really rich or powerful non-drow can afford it, but you can do it. It also benefits the economy of the higher houses of Menzoberranzan, it is encouraged. But going outside the neighborhood is strongly discouraged. That's according to the lore of D&D.
Astarion was wealthy, and could theoretically have thorogh business with the Drow. In Raphael's House of Hope, before the portal to Menzoberranzan, Astarion has a line (unfortunately, I can't find a close save to quote verbatim) that makes it clear that he doesn't want to go to Menzoberranzan. Whether he doesn't want to go to Menzoberranzan because he knows about Drow society and its customs, or he doesn't want to because he has had some, perhaps not the most positive, experiences in the past, remains a mystery.
