Maybe its tied to quests decisions or something. Did you kill any npc's during your playthrough? Did you side with thr goblins maybe? Did you take a deal from 1 of the factions to help you get rid of the parasite?
I inmagine its caused by something, right?
If it's related to something the player did, make it clear. Otherwise it's just a bad game design. I shouldn't have to guess why my party members are attacking me