I'm not sure if it happened because I switched classes or because I let Astarion kill me from blood loss and used Withers to resurrect me, but either way I no longer have Loviatar's blessing. Annoying but not game breaking.