Sounds like your paladin broke their oath. Have you been to camp since? There should be an oathbreaker knight (or somesuch) hanging around. You can pay him gold to re-do your oath or change you to an oathbreaker paladin (not sure what in the full game as I only played a pally in EA)
