Everything past the bridge and in the underdark would probably be way more doable at level 5.
Considering you'll usually reach level 4 at the end of the goblins/druid/tiefflins quests, I guess that's how it's designed to be.
But to be honnest I think those encounters will be very easy at level 5^^
This will be hard to manage all along the game but it could become less a problem at higher level.
It will takes way more time to level up so we'll probably clean "a map" map with 2 or 3 levels rather than 5. This, only if they don't increase xp rewards to make us level up faster.
Actually in one playthrough I made lvl 4 before leaving blighted village.... Still had Wetlands, Goblin Camp, Crossing the bridge, Underdark, etc left to do
