I am told that level difference matters hugely in this game. Specially during lower levels. That might be it- maybe your magic is getting resisted. But I would not know in details because I am not there yet.