I very much agree. I've had times I knew I had a nighttime story scene that I wanted to trigger, only to have to sleep 5 times to trigger it because there was a pile of events queued up that hadn't happened yet. You just don't need to Long Rest enough to trigger most of the scenes in a timely way unless you're playing incredibly poorly.
