I don't actually see casters being underpowered.
Especially if you lean into the barrelmancy Wizard and go for playing the environment over just direct damage.
The only argument is "the enemies have more health", and sure, but so is Home Brew monsters. Honestly the Monster Manual isn't the end all be all of monsters. Many DMs tweak their own versions to give a bigger pool of variety and challenge.
Goblins already have a +2 to their dex saves, making them resilient to fireballs anyways. (And Sacred Flame, but Sacred Flame doesn't really come into it's own until lvl 5. But man do I wish it was Wisdom based and not Dex based in 5e since it's Clerics only damage canrip)
And the "affects HP" spells. Those are based on current HP, not max XP, so you throw those in after the fireball went off.
