Also, if you run into the castle, then click inside again immediately after the conversation with the guard ends, they will not be quick enough to stop you a second time.