Hahaha, I have the same problem.

With my d-pad, I have to really press the buttons. Sometimes when I'm casting Way of the Ranger, which is on the left d-pad, I'll accidently un-summon my creature.

In one of those Flying Fortresses, that can kill me really fast. I should probably just put Way of The Ranger on the top of the d-pad.
When I first made my priest, I wanted to have an army of summons. When I found out I couldn't do that clearing out the Broken Valley area, it kinda pissed me off. So when I got my Battle Tower, I got my point back. It was the principal of the matter, haha. But having all three summon spells, and using them based on the situation seems pretty smart to me. Someday maybe.