How long are you waiting? The BF is generated the first time you enter in each act, which can take awhile. In act 2, the first time you leave the imp village you are summoned by a necromancer; if you try to leave by entering the BF, that causes an extra delay.
Try just waiting awhile, and if you are trying to get to the BF from the village, try leaving the village first, then enter the BF after the necromancer's task.