Okay slight tangent but I have always been so annoyed by the fact Halflings are called Halflings when they aren't half anything. It just irks me in a very particular way that I know is stupid and pointless, but I still can't entirely ignore.

And I think the thing about charisma that makes the racial ability scores thing break down for me is that, as others above have pointed out, charisma isn't objective. Strength, dexterity, attentiveness, even intelligence to a certain degree are all objective. No matter who you interact with, those things won't change. Even if people think you're not strong, smart, etc, you will always be precisely as much that thing as you already are. But charisma is very, very subjective. You can be super charismatic in your own culture, but all the things that make you so in that culture could be the height of impropriety in another. And if you're saying halfings or half elves get their charisma bonus from their looks, then that shouldn't apply to creatures like Dragonborn, who would have entirely different standards of beauty. Or hell, if they meet a blind person, that charisma bonus shouldn't apply there either. UNLESS we say that their charisma bonus is magic. And since charisma can impact the weave and magic, I think there's room to say that that's the case.