Yep, it's especially jarring after their quest, where you have a character who essentially just stated "Fuck our people. Thank you for showing me how rotten we are. I hate our commanders now"... And then they *still* attack you on sight if you are flagged as someone who left the Fort.
Yup, that's why I pretty much flagged that as a bug. There's no logical reason for them to attack you in any way.
