The teleporter stone on a pedestal (triggered by picking up one of 3 keys) in the room with the gargoyle statue and crystal bag is fixed in place, and can not be removed.
You can get a set of teleporter stones relatively early in act 2, though you loose them once you enter act 4.
