This has always been my thought as well. I never considered races in D&D as analogs of real world races and physical limitations and advantages seemed logical to me. People demand and seek immersion but it's jarring to me that a halfling with strength 20 can lift and carry the same as a half orc with strength 20. But clearly I am in the minority here.
It is a bit unrealistic, but what is the solution? Should halflings be limited to a maximum strength of 16 or 18? Or be allowed to have STR 20, but have disadvantage on all STR checks (including str-based weapon attacks)?
These changes would make the races more 'realistic', but they come at a high cost for player options.
The answer is yes. There should be limits. If people seek immersion then the answer is yes. If people want to min max at the expense of immersion then the answer is no. The choice really comes down to what the player really wants.