I made a post on this as well. I've discovered two things:
1) The imp familiar, if currently summoned, seems to consistently make allies a the camp become hostile
2) If you teleport to camp using a waypoint during the day (as opposed to using the campfire button), allies can sometimes be hostile
