It's on and off. It's almost as if the game is forcing a spellcasting roll regardless of whether my counterspell is matching level or not. So sometimes it passes, sometimes it fails.
Unfortunately, the combat log is supremely unhelpful. All it says is that my character cast "Failed Counterspell". It doesn't even show a roll for spellcasting check though!

I'm using Mystra's Spells, from the in-game mod store. But as far as I know it just adds some more spells rather then changing ones that exist.