Yeah, it's pretty much reputation + modifiers per person.
It's increased by doing quests for them, and lowered by giving bad offers, being caught stealing, etc.

Mostly I notice it doing it affect the prices you pay & get during trading.