Okay! After a few days of waiting someone posted that the fix was to kill Jaheira before entering the lower city. It flippin worked, no delay at all now in the lower city. What a crazy fix.