You might have better luck leaving it outside the entrance of the dungeon, 4verse. During the beta it became pretty clear that some of these dungeons are placed on the same map as the outdoor maps, but waaaaaaay off in a corner somewhere where they're not normally seen or accessible except through a transition point (like a hatch on the ground, or other prop).
If it's measuring the actual physical distance between the town and where the dungeon is placed in reality (rather than pretending it's right underneath the above-ground entrance), it's little wonder the range is being exceeded and returning the pyramid to your inventory when you teleport out of a dungeon, across the entire map, and back to town.
