You've already been to the castle by invitation, and visited the ghost of the duke? The graveyard remains locked until after you've been to the castle.

You can get a key to the graveyard / Nericon's garden (same key opens either) from Walker's back room (see here and here) or by file swapping;
Lost Items - guide and directory of inventory files

or bypass the gate;
Teleporting Anywhere, via hex editor