I tend to forget to change the connectivity setting, because it sadly does not get saved.
So if a random joins, it reminds me to change that.
But honestly, I have no idea, why people join randomly a game. This is not like Diablo, where you have your own character and take it with you.
