Yes, it's viable. However, my experience is that I was better off without it eventually, and the way health scales with so many things, cutting it into less than half is a bad idea. Lone Wolf has synergy with health boosts that shouldn't go to waste IMO.
I actually take full man-at-arms on my lone-wolf mages, just for the durability talents. It's odd but you have the ability points to spare and it gets me access to:
Weather the Storm: 50% magic resistance.
Picture of Health: 25% extra vitality
Pair those with Fortify and Absorb the Elements, and your Lone Wolf mages are tanks. You eventually get enough speed that you really don't need the extra AP and you'll struggle to survive in some situations with such low health.
It's a fun higher risk play style though.
