When you use the pyramid to teleport out, keep it in your inventory and use it rather than drop it on the ground.