Even if Larian eliminated fast travel from non-camp locations when implementing contextual camps, you wouldn't be required to double back??? You can just push forward to the next camp, and again you'd be able to fast travel from camp to camp. (Presumably they'd also untie conversations from long rest locations, or at least prevent the skipping of conversations because a new one triggered.)

And if you're finding the game so hard that you're needing to backtrack to long rest, then that's easily solved by difficulty options.