You lose your teleporter stones temporarily when you enter the castle, but you get them back. There are a couple small places where they will not work, but that's about it. There is no connection to or limit from the teleporter pads.