If you walk back to the lava where you entered the imp village, you should get the option to return to the citadel island.
When you find a higher level BF key, it opens all dungeons up to that level, even if you missed earlier keys. You could just skip the first key and wait until you find the second.