Now think about how many dialogues you will need remove from the camp to somewhere else. There are no problems with the rest system, I believe there are problems with the dialog system for which you need to rest.
> if you want to kill the goblins, all of them, its extremely difficult to kill them all without long resting
This is not difficult at all if you use food and a short rest. We can also have one fight in the camp, if you go with goblins. This is just the case when you have not rested, and still this is not impossible fight. You should also explain "dungeon" in more detail. Goblin camp? This is an open location. The Underdark? No. Conditionally we have two dungeons in my understanding:
1. Owlbear Cave
2. Hag's Tea House
Idk about Owlbear. but I think we can't rest in Hag's Tea House. So maybe there's a difference.