The problem is, where does that convenience factor end?
In dangerous areas. If I'm in the Spider Lair or Hag's Lair or wherever, and I can't Fast Travel, Fast everything is locked.
Think about it. Why not allow Fast Buy and Fast Sell, etc.? Why make players Fast Travel, run into the grove, talk to the merchant, listen to that tiefling trio talk about how the guy cares about their lives, their futures, slowly and painstakingly drag and drop from one window to the next? If you can rest an hour at the click of a button for short rest, and if you can fast travel instead of manually jogging everywhere, why not speed up one of the most boring aspects of the game, endless item management?