They are not hidden under something in your inventory, after talking to the imp in the room west of the gong?
If you do the inventory swap again it will work now. If you load the save with the other inventory files, you may be able to click on one of the teleporter stones to find one of yours. I tested a save with two sets once, and the last moved stone of each pair becomes the dominate one, though if the game checks your inventory first and notices two there, it would not teleport you to your own. If that is the case, you can just keep/use the new set.