It also effects your ability to use high level elemental skills. Some pyro skills for instance are level 10 and require a lot of action points to use. The higher your pyro skill the less AP it takes to use the spell.