your ping will be very different to theirs is why it happens
i.e, the person with the lowest or steady ping should always host

Luke Skywalker: I don't, I don't believe it.
Yoda: That is why you failed.