If you are telling me that I've been paying the entire game massively inflated prices just because my main character isn't charisma based (and In fact it's a barbarian with 8 CHAR) then I would ALSO count that as a shortcoming of the game having a stupid implementation for this stuff.
Yeah, they use the charisma of the person speaking to the npc. I checked the prices between my party members before selling at first vendor. Also, you can take out gold you would get from selling or give more when buying to increase vendor favor. I did this for first two vendor's since its just them for a while.
