LOL, forgive me Larian, how could I have ever doubted you!
Not a bug. Well, maybe a bug, but nowhere near as bad as I thought. The NPCs aren't mad at *ME*, they're mad at my Djinn. I had one out for the fight and it lasts a long time, so it was still around. Dismiss the Djinn, problem vanishes. Odd, not sure why, but I got around it so sorry for all the posts.