Open your inventory pane and stand close to the teleporter stone on the ground. Left click on it(and hold down the mouse button), drag the teleporter over your inventory pane and release the mouse button to drop it in.
You will need to use this technique to pick up letters, chests and other item who's default action when clicked on is not to be picked up.
