Second playthrough, Chaos side, I have noticed that if I take too much time looting corpses and crates, Gobelins will clean the grove without me (even kill Wyll, but there's still some events that require me to be there).
It is true to say there is a time and space (exploring or taking some path) mechanic even without triggering night/sleep.
IMO, outside the quest update notification that triggers after a night/sleep and/or quest/event completion, it is just some minor/extra things.
Depending on your play style and goal, you may want to use your saved points or have multiple playthroughs.
Actually, it may be a good challenge to finish the game with only your two short rests, relying only on magic scrolls and potions.
