Why?
A hill giant has a cha of 6. You are some peasant and encounter one in the wild. He clearly does not like your presence and roars, while raising his club. Are you scared or not?
If Volo the Bard, with his civilian clothes, happened to be closeby, picked up a little stick from the ground and also stared at you menacingly, would you be scared now? More than by the giant?
It depends on what intimidation is used for. If it is me being clever and threatening you with revealing some secret etc sure, it is a cha skill.
But my wizard is not scary because of their charisma, he is scary because from the magical gear he has and the lightning coming from his staff, you can clearly see he is a powerful wizard, so his int. My fighter is not scary because of his wit or because he will outsmart you and embarass you in front of your group, he is scary because he has a long sharp stick of metal and enough muscles to hurt you with it.
Most skill checks should be dependant on the situation and what you are actually doing with them
Intimidation checks are tied to the charisma attribute. It is as simple as that. You can be proficient in intimidation and accumulate terrifying presence with levels by raising your proficiency. Otherwise, you need to be charismatic to exude that presence. Charisma does not equal cunning or guile. Your example has some validity, because in the dnd sessions, from my experience (though limited), a DM can give you "story" advantage in a situation where your frightening appearance happens to be a part of the world logic. For example, it makes sense for an unarmed peasant to fear an armed fighter. But to intimidate a seasoned mercenary you'll need some extra effort - and that effort draws from your charisma score.
