I can get to to work..occasionally. Best shot seems to be if all players end the game and then try using the session menu to add them.
It's weird though. Frequenty when they attempt to join my game they join some random other persons game.
Also I have had random other people join my game even though I have it set to friends only, I assume they are randomly landing in my game incorrectly the same way.
