Did you get the key from Blake the merchant (in the North-Western house of Ars Magicana)? That is the only key to the garden.