No you aren't. There is a massive block between level 3 and 4.
At level 4 you can get (buy or steal) all the skill books you wish but prior to that they are very limited. It is fairly easy to get to level 3 inside the fort but level 4 less so.
There is an advantage to staying in the fort - you can kill Dallis and her pets which you can't if you leave. This gives you a lot of XP which makes it possible to be level 8 before the last fight rather than half way through..
If you don't care then you can leave the fort whenever you want and get XP from the following (with no fighting required except against fossils)
Enter the compound XP420
Go and see the retard on the beach and get his ring. 0XP.
Dig up the hole next to him and pop down and up for XP720.
Go and give the retards ring to his mum/dad (not sure which gender) XP600 and get their key.
Pop down to the frogs cave (sneaking again) for XP600 at the back. May as well go and see Witherspoon while you are at it. XP720 I think.
Sneak through the path with the fire slugs and up into the fort. Go into the guys private chambers for another XP720.
Pick up the teleportation skillbook. Go and see the teleport guy and tell him you know all about it - XP600.
Sneak out of the fort for XP 3600
Sneak over to the east of the island, through the gargoyle maze, down the steps at the back and into the hidden cave. XP some thousands. Don't try to enter it further yet as you'll die.
Go back and steal the purging wand from the witch.
Refuse to give it to the dragon (XP600). This is a morally bad move and your computer will scold you for doing it.
Kill the now fossilized Dragon XP3200. This is morally even worse (and you need to use piercing arrows to do it) but at least you are now on level 4 without actually fighting anyone who will retaliate.
You can also get another XP1600 by
grassing up the lizard on the beach and then helping him not die while he kills his assassin 'Silence' but as that involves fighting I left it out.
It is also a bit morally dubious but as you'll never kill all Griffs gang at level 1 I suppose it is OK.
Better than killing a fossilized dragonknight anyway
I found that if I could get to the village in the last DOS the rest of the game was pointless as I'd done the hard bit.
I now feel if I can get to level 4 the rest of the game (well this act to be fair) is pointless.
And before I get picked up on my incorrect use of words again by @stabby when I say pointless I mean possible. At level 3 you can't finish this act unless there is some trick I don't know about. At 4 you can. You have to cheat:cheese a bit but it is possible.
So for me if I make it to level 4 I'm done (it is possible after that) and I'll try another iteration.