The gate will be opened after you have the elven council member quest, which is after entering the castle.