Spells really dont do more damage than normal attacks from rangers for example but yet you have long CDs on them. It wouldnt be as bad if your wand attack would do as much damage as from a normal bow or weapon attack but it doesnt, it does way less damage.
On top of it mobs have resistances which lowers your damage even further and thanks to the new armor/shield system there is nothing which reduces physical damage.
If you dont care for achievements I can recommend the fast casting mod for mages which halves the CDs for damage spells.
You are already limited by AP so I dont get why Larian didnt balance the game around having low CDs on damage spells like lightning bolt, fireball etc. ? Its bad gamedesign if you always have to use each round different spells. And often enough you are not left with a good alternative for 1-2 rounds, doesnt make sense and is not fun.
