It doesn't have to go slowly at all, just look at the average height now and 100 years ago, genetics are such a big part of it. It's like deseases some are genetic, 1 person has a genetic "flaw" that causes a desease and it can be passed on generations. In case of wisdom teeth, this is my theory: teeth health has improved a lot over the last few centuries, where before wisdom teeth grew on a later age because other teeth were in bad shape (a theory). Now genes pass on less deseases of the teeth, so the need for these wisdom teeth is no more, and slowly they are dissapearing.
In history you'll see Homo Erectus or whatever first started to wear clothing (rags) before it started loosing hair, back then they had different, less, sharper teeth (lot's of raw meat).
Back then these adaptions went slowly, not because one home erectus wore clothing they all did it, it took many centuries for all of them to learn that behavior and so did the hairloss. It took a less long time to improve teeth health care, so it will in this theory take a lot less time for wisdom teeth to dissapear.
