You miss, you miss. Nothing happens, but... if I recall correctly... the charge for smite is still used. I might be mistaken on that. I had a Vampire Spawn Minotaur Paladin running as one of the two custom support characters to my main (which was a Werewolf Sorcerer Gnoll). The other Support Character was A Goblin Bard that had been a Goat for 10 years.
I only lay all of this down to give you an idea of how easily I could have lost track of some of the finer details.
"Starting at 2nd Level, when you hit a creature with a melee weapon Attack, you can expend one spell slot to deal radiant damage to the target, in addition to the weapon's damage. The extra damage is 2d8 for a 1st-level spell slot, plus 1d8 for each Spell Level higher than 1st, to a maximum of 5d8. The damage increases by 1d8 if the target is an Undead or a fiend."
The charge is not used up unless you use it, rather you can only use a charge if you hit.