Don't drop and use the ancient pyramid. Throw one across the swamp so it lands on the island; stand on the west side of the swamp, click and drag the pyramid, hold the shift key down, bump the screen focus east with your mouse, drop the pyramid on the island (this may take a couple tries depending on your sight range), and teleport past the swamp.
Alternately;
Teleporting Anywhere, via hex editor 
