I agree, I have been on the other end of this (no offence). While I didn't use any "interesting adjectives", it was quite surprising to have a stranger pop into our game and take control of one of our characters (it took a bit to notice).
I feel like an options menu when you start a multiplayer game would solve a lot of these issues, i.e. it could ask if you want strangers to be able to join your game. Rather than having to search through the options to get to these settings.
This is an alpha build so I expect/hope cleaning this up is in the plans.