Hmm. I thought you could reject everyone from the party. I admit I haven't tried, but I thought you could. Even if you accepy him in the party at first, I thought you could tell him to go after you find out he's a vampire.
Then, there's a monster hunter in the bog. You can tell him Astarion is at your camp. So I thought they had several ways to get rid of him.
Or maybe your point is that you cant just tell someone to leave at any point in time. I will say, I agree with that. I'd like it if you could tell anyone to go at any time.