It is said a few times in the game, that the tadpole swapped their abilities and they had to relearn them.
So no matter, if Wyll fought hell beasts, Karlach was the hero of the blood war, Gale the next Elminster or Astarion practicing lock picking for 200 years, they were temporarily disconnected from that and had to relearn. In Gales case, the orb is a second problem hindering him and as Rag mentioned, Astarion was a slave and I doubt, Cazador was about training his spawn.
