Hey OP you are absolutely correct. And kudos to all the others who've corroborated this point. Yes in RAS's books there are a number of drow hanging out on the Sword Coast, but they all took a very long time of traveling to get there, in many cases years of travel. And yes, portals are a possibility, but portals are not that common and tend to suffer a lot of oversight by the powerful NPCs of the Realms.