I think getting into Multiplayer to begin with is the problem. We've never had a problem once an invitation was successfully accepted and we could see each other in the Multiplayer lobby.
We still have occassional glitches, but it seems to resolve after just trying a second or third time...usually