Apparently I'm supposed to be close to level 5, but I'm level 3 close to be level 4. I can't progress because everything is higher level and I'm getting owned.

what I'm missing? I defended findal, the injured druid, fought the owlbear, saved tiefling kid from harpies, saved arabella, cleared the crypt from undead and bandits, collected 4 soul coins, defended the Grove from goblin scouts, recruited the ogres, cleared alchemist cellar, saved the gnome, what the hell I'm missing?

I can't fight spider matriach because she's level 5, and I can't recruit Karlach because Anders is level 5 and he one tap her, I can't enter the swamp, I can't clear the goblin fortress, what the hell I'm missing?