AP limit is critical for cases like 'wait for the enemy to get close, unleash 12 AP worth of damage'.

It's not like spellcasting causes Opportunist attacks anyway, so there's no issue delaying those until after you made space for them(be it moving yourself, or delaying the turn to cast after the target exhausted his pool).