If you're multiclassing from Paladin, mechanically Warlock will work better, because it is also a Charisma class. Spore Druid uses Wisdom for its casting stat, so you would have to be careful not to use spells that rely on saves if you use Druid spells, as you won't have the right stats for it.
