Only the BF is possible without reloading an earlier save. The hermit will still be there after getting back from the shaman's realm, so if level 2 of the hammer spell is good enough, then there would not be any reason to revert to an previous save.