An interesting option. It didn't occur to me to try to attack the crystals. I like that the game has such easy-to-miss details.
That is likely because the UI does such a poor job in highlighting targetable objects. I still have to find a way to destroy the webs in the phase spider lair, because I just can not target it.
That should work simply by selecting e.g. a fire cantrip (Gale) and hovering over the web, which should display it as a target with, I think, 4 hp or something.