Hey this is definitely a bug, my friends and I have run into the same problem. We were only able to fix it once, but it happened again once someone crashed. The way we fixed it was have the host go to camp, remove every character from the party, then send invites.