If you have a wand + shield mage, it's probably better to put points directly into Intelligence and whatever school of magic your mage is focusing on and use a wand with the same type of damage. So Pyro + fire wand, Hydro + water wand, etc. After a certain point, your mage will be focusing exclusively on spell-casting and you'll only equip the wand for the bonuses it provides. At that point, all the points you've put into INT and your respective school(s) of magic will pay off in increased spell damage. If you're really concerned about weapon damage, you could always carry a staff and put points into the Two-handed skill.