I think this is a fair ask. I like witchcraft too, but am way more selective about its spells. There's no pressing drive for me to put it up to 5 without some truly nifty talent on the horizon.
I'm similarly picky with marksmanship skills--I only find a handful of them worthwhile in execution, but I upped that to 5 pronto just to get quickdraw.