The problem is, the campsite itself is immersion breaking. There is no conceptual grounding on how the hell we teleport from the middle of a dungeon.. to the surface level... sleep for 8 hours then teleport back and the monsters are just chilling in the same place.

This is D&D, there is every kind of magical craziness that you can imagine here. Is camp a pocket dimension? a demiplane? do we legitimately teleport there with some Gale magic (even if hes not in group, maybe he summons you to camp). The immersion is shattered because there is no explanation of what is going on when we go to camp. Pocket dimension would make the most sense lorewise... but just explain what a pocket dimension is so it feels less immersion-breaking