I think the main Tadpole, the one that you start the game with, eats the other tadpoles. It's part of Illithid biology where the tadpoles eat each other if left to their own devices. So instead of multiple Tadpoles, you just have one very hungry one who eats the others, and since it has more nutrients is able to more easily infest the rest of your brain. This is how I think it is operating. Too many Tadpoles might just plain old kill you, the Tadpole needs the host body alive to complete the ceromorphosis, it will defend its host from other foreign parasites so it has room to grow.