If you can get any one character to a location, then I believe you can go to and return from camp with them selected to drag the entire party with them to the difficult-to-reach location.
Yeah this seems awful to me. It feels like a cheat or exploit.
