I've had discussions in the past about whether or not it is using meta-knowledge to target the guy in robes/clothes because he's "obviously" a wizard or sorcerer. After all, there are many things he might be, including a merchant, a party scribe or a fancily-dressed rogue.
The consensus is generally that the world of D&D is what it is, and the creatures within it all live by the metaphysical laws that surround them. Therefore, most intelligent folk will know what adventuring parties consist of, and therefore that the arcane casters are liable to be the ones in the lightest (or missing) armour. The same argument goes when you have players comparing their levels and classes. If these things are fundamental to the way the universe operates, then why would the people not be aware of it?
It's an interesting way of looking at the game. I prefer to operate my games without this "meta", but I can see why others might follow that logic.