At least in Europe, that's the history of many major cities' districts. Paris used to be an island in a river. Many of its modern-day core arrondissements and later banlieues started off as such gate towns.
As a european, this is not the case for any town or city I've looked at the history for.
Even for cities that were built around a castle and thus had the similar situation as Baldur's Gate with having a wall separating the inner castle and the outer farm land... The entire thing is still considered an extention of the city. Those living outside the castle walls were not considered a separate town.
In fact, this is more true of Europe than say America. Since the entire thing about why European cities are so well built is because they were organically expanded, so that access to facilities remained easy as places built up over time.
But I said nothing about graphics.
You mentioned:
getting lost in a frameless pretty painting, or endless loading times/other performance issues.
These are issues of graphics. Not map design.
Loading times and performance issues are based on graphical intensity. (Which is why games with low graphical intensity, such as Minecraft, can have massive worlds with low performance requirements - To the point that there are mobile versions of Minecraft)
A "Frameless pretty painting" is based on graphic design.
