Yes, but only in the cellar of the building in the north west corner of the complex (the hatch is in the northern room in the guard's quarters), where there is no anti-magic shield. You can also toss chests out the front gate, or place them on the wall to get later from the outside.
You may want to put any excess loot you have in a chest to toss out the front gate, to make room in your inventory for what happens after you leave the castle (you will not have your teleporter stones for a while).
