In the short-term, I believe it just adjusts the "attitude" of any other friendly NPCs you talk to. This means that it is then harder to convince them using the Charisma mini-game, and they'll give you worse prices when bartering.
Long-term, I'm not sure. There might be certain characters that refuse to interact with you at all depending on your Reputation score, or events that play out differently depending on whether your Reputation is positive or negative.
