I don't know if this is a bug or a hidden successful persuasion check, but I ran into Gavin (or rather, he came up and initiated conversation first). He insisted on talking to me and me alone. Despite the fact that I was traveling with two other people at time,they were right there, our portraits were linked together, I told him "yes, I am all alone", and he believed me.
If that was a successful bluff, then I guess it's working as intended. But it rather seems like he should have gone "no really, I mean alone." and won't talk until your companions are some distance away.
