I've had this same "bug" in multiple different runs:
After getting my collar moved by either Nebora or Leya, and then having guards attack me for not having my collar on, Ifan would join the side of the guards trying to kill me. Even if i had a positive reputation with him before from helping the elf girl.