I don't think any sensible person would like the idea of a parasite in their brain much, and it's unknown what using those powers does to their health. And the magic involved, perhaps you can be mind controlled or killed at any time. Hence not using the powers and getting it removed at the first opportunity. Save for perhaps Astarion, but I never have him in my party because the whole vampire storyline is so cringe.

I will be very annoyed if the tadpole can't be promptly removed because Larian decided for me that I need fun powers, and that's all the tadpole is without any drawbacks.