The devs really need to just make an option:

Use Talker's:
This setting uses whoever started the conversation.

Use Highest:
This setting always uses the highest Skill/Save in the party.