Once you get your stuff back and leave that small area, there is only the one required level (the exit is to the far west). The lower levels, reached by reading the scroll to open the stairs, are optional; when you get to the bottom (it sounds like you are close) there is a shortcut back up to the first level.