Thanks for re-stating your argument, I think that helps me figure out the core of what you're saying (though I agree this is more a thread for the general section). I think you have a good point about saying "it's fantasy so anything goes" isn't a great argument, but I also think that your base claim about what stretches the bounds of believability in the first place. I also think that a lot of those genre conventions are just conventions and not essential characteristics. I think your argument is better aimed at specific settings as opposed to fantasy as a whole. I think that on a broad level, because it's fantasy anything can be permissible, but to use your Duergar example, I'd also say that sure, anything CAN be possible but based on the established lore for this setting, that's not how it works. I think that's fair especially when physically duergar have one real visual defining trait.
I also think that the "It's fantasy" argument also gets used as much as it does out of frustration that somehow acknowledging identities and diversity that did not originally exist in the genre because it was established at a time when such things were not acknowledged in the society at large is somehow harder to accept than fantastical magic and inhuman entities. I'm sure plenty of people make the argument sincerely, but in my experience it's not the norm.
Thank you for engaging in good faith. I understand that the argument is born of frustration in many instances and I get that there is a difference of opinion in terms of what exactly hurts suspension of disbelief. The difference between characteristics and essential characteristics is also very difficult as in the boat formerly owned by Theseus but the pragmatic view of knowing wins here which is why I mentioned sophists and chairs earlier. Using a kind of appeal to extremes argument by saying that a genre has changed slightly over time and it's generally not always easy to identify essential characteristics ergo fantasy is just whatever and now every fantasy realm is California with a medieval paint job is not a good argument (I'm strawmanning but allow me the rhetorical flourish).