There's probably nothing stopping you, if you unmemorize your earth/fire skills. Although you could just hire someone else like Lhose who starts out as Air/Water by default. Later on you can unlearn the wasted points.
However, I will say that I'm finding earth/fire to be much kmore effective, damage-wise, than air/water, that's more for disabling enemies after you strip their magic armor.
