Throw one teleporter stone deep in the castle and use the second stone to teleport inside.

Just avoid the gate guard.