The key from the prison is actually the level 1 BF key. Did you cross the lava yourself, and enter the imp village? That may cause problems. After freeing the 2 imps from the prison, did you try leaving the village and get summoned by a necromancer? If you try leaving the village by entering the BF for the first time that act, it takes a lot longer than normal (though still shouldn't be an hour). If that is the case, try walking out of the village to be summoned, and then enter the BF.
