Probably someone else has already pointed out this scenario, but here I go.
I realized I hate this unlimited fast travel on the first version of the game, during my first playthrough.
I was at the Goblin Camp, doing my commando-like quest to quietly infiltrate and eliminate the three bosses of the horde. I killed Minthara, the Goblin priestess and only the big hobgoblin himself was remaining.
I triggered the encounter and managed to survive through it, but now all my abilities and resources were depleted, I was low on health and with a whole garrison of goblins now hostile to me. I had to think of a clever way to get out of that mess... and then it hit me: I could have just fast travelled away.
I didn't do it, because it was no fun to me, but the fact that the option was legally available bothered me soooooo much it killed my immersion. Why bother to learn magic or to carefully plan a heist or an assault when you can just pop away if you get stuck or mess things up?
I understand many people may not be bothered by this but I think is a serious game flaw, detrimental to immersion and the overall challenge of the game (at that time the infinite long-rest option was still a thing, so the game difficulty was below 0).
I would love for fast travel to be like in The Witcher 3: you can fast travel only if you are near a fast travel point. It also makes sense since you should be in front of a teleportation gate (rune) to travel to the other side.
I can accept the existence of teleportation scrolls for when someone is really stuck and the last save game is maybe hours before, but they should be really rare and/or expensive.
Anyway, these were my two cents on the matter
