The Fab Five quest is basically the same quest you can get from another NPC, so no; you won't be missing out.

@ LeBurns; You could simply let your toons agree with each other? Or did you turn off dual convo's/picked an AI.