You can turn off randoms. Though its never stated to newer players that you can do this, which should change (it should ask you when you start a multiplayer game). You can switch between: Anyboy, Friends, Invites and Nobody.

[Insert something funny here]