There is a tutorial message about the teleporter pyramids being returned to your inventory the first time you pick up the blue one (though it mentions getting too far away, rather than specifically changing maps).
You can prevent this (for now) by placing a pyramid in a container (can still teleport to it that way). However, this can cause problems if you use the other pyramid on a different map (you could be teleported out of the level bounds, or somewhere you may not want to be).
