I restarted my game using a new profile. I did some messing around Fort Joy and here's where I narrowed it down (for me at least).
Tested killing certain NPC, stealing outside loot, stealing from vendors, not stealing, and a few other things. The moment this bug occurred was after initiating the fight with the Fire Slugs in the caves. Loading a quicksave that I created right before the fight carried over the bug, loading a regular save that was a few minutes earlier DID NOT carry the bug over.
I was able to continue my game normally without dealing with the slugs (I did the Red Prince marriage pass) in combat and I haven't had any issues at least for now
