I seem to have missed a lot, the hag, gnolls, Anders and the goblin bosses are dead and I'm still lvl 4.

You might just have done some other stuff in a different order, or taken different options that don’t maximise XP. If you kill only when you absolutely have to, for example, level 4 at this stage sounds about right. I wouldn’t worry about it, as I’m sure there’s still plenty of content you’ll find if you keep exploring, especially below ground.