Yeah, I have also left her behind because of how she derails the story, there's the whole thing with Wyll, and personally I also didn't love how the player dialogue during her big scene was not in any way adapted for Avatar-Astarion or Gale - you'd think both of them would have some better lines than the standard Tav ones.
From the way Wyll talks about it, I had assumed he would make dinner for the whole gang - and that would have been such a nice chance for a proper group scene with all the tadfools, possibly making use of the cozy sitting-area in the Elfsong.