Does anyone else have the problem where you go back to camp and regroup your party that when you say "leave camp" or "go to X map marker", only the ones that were also in your previous party come with you? This wouldn't be too much of an issue except that when I go back to camp to select the ones that fell behind, the map locations are greyed out. It makes game play very difficult.
So I think I figured out a way for this not to happen. Basically, ask the party member in the world map to leave. Then after he leaves group, port to camp and invite the character you want. I have noticed doing this, I no longer get that crazy bug of the NPC porting to some crazy place when I leave camp, or not being able to port him out at all.