I've had this bug multiple times. Wait till it throws you into lava heh.
Most humorous time it happened it ported me to the Manni wizard dudes lair and I had no idea where I was since it was so much later in the game.
I just shrugged and said it was because my wizard's callous use of magic was ripping holes in the fabric of the universe.
