Actually this is happening a lot. It's happened about five more times since I posted this. Elemental affinity is always accounted for in the AP cost (you can see the -1 in the tooltip), but I can't cast it, and it costs me 2. It's really frustrating when you plan around it.