Okay I might have figured out what went wrong. So when he says "Be right back!" He leaves to sacred stone to get friends to come back and fight. However I had already wiped out the whole village, so there are no friends for him to gather. I ported to sacred stone and sure enough he's on the other side and initiates a fight as soon as I arrive. So I guess I did kinda bug it by killing the sacred stone villagers. Now to figure out where in Rivellon to find more star stones. arg
