I feel totem were too strong to be honest, and that's a shame since I was hoping the summoner to be more about summoning a creature you control (or a small number or weaker ones like a few skeletons, ... I know this won't be the case) rather than actual turrets.

I don't mind the turrets it's just they drag enemies' attention, deals damage etc. so...
Even if only one a turn the enemy has to destroy each every turn making them spent a bunch of spell or attacks for it.