You need to explore more. There are plenty of places you get XP from without fighting anyone. If you sneak around aimlessly you can get to level 3 without fighting anyone at all if you wish. Eventually.
Maybe so, but that only reinforces the point of feeling like you need to scrounge up every morsel of XP to make progress, or feeling like you need to do things in a very specific way.
I'm getting old, I don't have the patience to wander around aimlessly for hours. I have better things to do with my time and I'd be more likely to consider that a flaw with the game than "working as intended".
If you mean the 600 XP in the frog cave you can find for standing in a random spot, I'm actually going to recommend the devs remove that and tie it to opening the chest in that cave.
You don't need to pull it out. You can break it if you don't stumble across items that give you a strength boost. In any case there are numerous other ways into the prison such as up the ladder behind the necro saleswoman or through the cave with the fireslugs all of which give you varying amounts of XP, are not mutually exclusive and don't require you to fight anyone
See, that's a neat tip. I didn't know about the spear.