It seems like there are two different things here:
1) How you behave in dialog.
Your options should totally be driven by your life experiences and knowledge - background and class should absolutely give you more dialog options. Race is more of a stretch but I could see an argument for common cultural experiences (assuming you grew up under typical circumstances, which is questionable for adventurers).
2) How others behave towards you in dialog.
This could be influenced by the things you say and things that people can figure out about you by looking at you or hearing you. Race is certainly a factor here. There might be hints about your class, but that's less clear. Background probably shouldn't be a factor outside of very specific cases where there are visual cues (e.g. if you're wearing your captain's hat, they might be able to guess that you're a sailor). But no one outside of Sherlock Holmes is going to just know that this 50-year-old adventurer was an urchin when they were 12 by looking - if they figure it out, it's because of how you behave in dialog (which is driven by #1, above).
I'm in about totally agreement with that. It's an important distinction to me.
There are still a few racial "what you say" things that make sense. Like an elf saying "hey, how come I'm dreaming, I usually don't" during the dream cutscenes. (Also, who can an elf by conveniently sleeping when Astarion comes for a bite ? WFT ?!). This is biology, more than culture. But overall, this should be rarer than class or background, that say more about your personality, and thus what someone like
could say.
And as you point out, nobody should see you as a former Criminal or Sailor, but if you're a Drow, you can expect people not trusting you.
