The first time you leave the imp village you are summoned by a necromancer. This delay added to the generation of the battlefield the first time you enter can make it take a little longer than usual. Try walking out of the village to get summoned, and then enter the battlefield from the place you are sent.

If you leave the village by trying to enter the BF, do not re-enter and exit the BF from where you are sent. You will be returned to the village, thereby skipping the location you are sent to (missing experience, loot and teachers).