It is by design that the teleporter stones are removed from your inventory on entering act 4. The imp historian said he would be back for them, though there is no in-game cut scene to explain that that is what happened.