Gameplay Solution: I definitely agree with your concerns about TP'ing from within a dungeon or cave. That seems bizarre. I also really like your suggestion that travel to the campsite should have a notification alerting us to how much time has passed (the "you travel for 2 hours" suggestion). That makes a great deal of sense. If party members have higher survival checks, they could speed up travel times, but we could also get hopelessly lost if none of the party members knows how to travel in a forest setting.
Story Solution: I really like that idea, of having a quest to find a campsite! I think that's another great solution. We could make the campsite accessible before the goblin battle at the tiefling camp entrance.