If you mean the 600 XP in the frog cave you can find for standing in a random spot, I'm actually going to recommend the devs remove that and tie it to opening the chest in that cave.
There is that place, yes. There is also
going into the back corner of the room with the caged dog, teleporting into the broken cell where you get the necklace (by the Houndmaster), going into Orvids Chambers up the ladder, into the fire-slug cave and so on.
I'm getting old, I don't have the patience to wander around aimlessly for hours. I have better things to do with my time and I'd be more likely to consider that a flaw with the game than "working as intended".
I don't know what is intended at all. You can leave the prison at level 1 (you then level up to 3 immediately). You can then just go straight to the final fight after stealing a purging wand and try it at level 3.
You most likely won't win at that level unless you are remarkably lucky but you can skip almost all the content if you want to.
I'm terribly slow and being very old have lots of spare time... I've never finished this bit in less than 30 hours so god knows how long the whole thing will take me. Weeks probably.