You can also just wander in. After the guard tells you to get lost, click inside the castle right away; you will run in and he will not react fast enough to stop you a second time.