I had this just yesterday with a laggy host after he loaded a slightly earlier save, but it was fixed by me simply leaving and rejoining. It was pretty early into the game, we had just arrived in cyseal, but it never happened with another friend when we played for 10+ hours. So it's related to bad connection i guess.
