My fiance and I just encountered this exact same problem while playing. He could join conversations fine until we got to the harbor and began having conversations with the townsfolk. I tried it without him in the game and I had the same issue. My character can join his character's conversations, but his can't join mine whether we're coop or I have it loaded without him in the game.
A fix would be greatly appreciated.