Do you have Pet Pal? You're giving up XP by not having it which could lead to you being under level for certain things.
I'm about to leave Cyseal and just hit level 10, but unlike Beta I often found myself equal or higher level for fights, whereas beta I was constantly 1 level below my opponents.
But yes, you shouldn't have to hit the pre-cave groups before level 5 if you've done enough in town/picked up the XP from quests in accessible places.
