If you fast travel to camp then fast travel out, it will only travel with the character you have selected.
If you fast travel to camp, then choose "leave camp" it takes your whole party and puts them back where you were before fast traveling.
Ideally, at least in my opinion, you should be able to fast travel out with the same group you had going into the camp. If you were a full party, 1/3, 2/2, whatever, should be the same group. Seems the camp forces your party to be unlinked, and doesn't remember what you were if you fast travel out (unless you choose leave camp).
