No. Why NOT Know-It-All? It's a point in INT in exchange for a Disposition drop. Does Disposition really matter all that much if this character isn't going to be doing the bartering? Besides, it's not like mages get a lot of good Talents. The only thing I have to watch out for is that taking Stench on top means almost a -50% drop.
Taking it at char creation is important because you need 8 INT for a success chance on your basic spells, and the Speed Mage has more things than just INT to spend attribute points on.
***
I've started the experiment. My other Source Hunter will be a sword + board fighter, to tank and take the heat. I'm now level 3 with a full party.
The biggest issue so far is that other than Burning Touch, you need to be level 4 before any of the touch spells become available, and they are pretty expensive, too that early on. So given my starting skill selection (Fast Track for Movement, Walk in Shadows for escaping, Oath of Desecration for my fighter) all my Speed Mage can really do is cast Staff of Magus on enemies. I picked up Minor heal to help.
Luckily, I've discovered that you can easily get to level 3 by visiting Hiberheim and returning. As for the money, well, I guess I'll be robbing Cyseal of its stuff.
This build may take a few attempts to figure out. I couldn't figure out what other skills I'd want to take, so I ended up just taking 5 skill schools: Witchcraft, Air, Water, Earth, Scoundrel. Even I know that's not optimal. I'll give it a go anyway.