The BF dungeons are opened depending on the level of access you have (shown when you mouse-over the 'enter BF' icon in the main game, and whenever you click on a BF key). If you have found the level 2 BF key for a particular act, then the level 1 and 2 dungeons should be open, etc.
The temple will be opened by a later main-plot quest. Just keep playing through the areas that are accessible and you will get there.