This is one of my complaints. Resting at camp messes with the narrative flow of the story. I get it and understand it, but there were times in camp where I didn't want to go to camp because it felt like I would get pulled into a story hook that I didn't feel my PC was ready for just yet in game.
glad im not the only one feeling like this for the narrative.