I got the overwhelmed by the Absolute near the Goblin Camp when I arrived with Sazza. The Guardian sleep wasn't the next sleep after that. I cleared out the camp except those outside, fast travelled to the Grove to report in, had the after-party sleep, went back to finish off the goblins still outside at the camp, went back to the Grove to sell some bits of gear then back to camp for a long rest (I originally intended to go to the Underdark next) and I think this was when I got the visit from the Guardian.
While we were near the gith patrol I clicked on Lae'zel by mistake and she started talking about her dream.