You will have to first go into game options and enable your drop-in co-op. Friends only should work for what you want.