It already has - there is a conversation between Gale and Wyll, where both complain, that they lost a lot of their abilities, since they got the tadpoles.
Plus Gale lost the favor of the goddeess of magic.
My headcanon is that Wyll is simply a liar and has exaggerated his abilities and feats. If he actually did do those things he accomplished, it's only by Jar-Jar-Binksing his way through any combats.
For Gale, it's feasible that his condition turned him from whatever level he was into a 1st level character.
It's possible that the Netherese Destruction Orb is constantly eating at his magic, making him much weaker. He has to build up his magical stamina to simultaneously both feed the orb and cast anything other than 1st level spells. Also, giving him magical items to absorb makes the Orb less "hungry," allowing Gale to use more of his own magic.
