It's a hard sell not to have fast travel in a modern open world kind of game. Maybe you're patient and don't have a problem walking around, but they have to fight for gamers' attention and keep them engaged. Simply removing the fast travel would be a terrible play. However, they could remove the portals but keep the mechanic so you can assume the party just walks there off screen if the portals are a problem. But they're already used for introducing Gale and they do act as a reason why you can fast travel to these particular points but nowhere else.
