This is generally how I play. In fact, I quite often kill potential party members I don’t plan on recruiting. Usually Lae’zel (she is a bloodthirsty Gith), Shadowheart (worships a dark goddess), and Astarion (pulls a knife on you when you meet him). Wyll and Gale I just ignore.
If you plan playing this game through a lot (as I do), there really isn’t any reason to grab a bunch of benchwarmers you aren’t planning on using.