Sometimes the portals to the different levels are so close to a wall that you will only find them if you pass directly next to them. You will not see them on your map overview either. I usually mark every area that I go to in the dungeons just because of that problem.
I have yet to find a dungeon (probable have done more than 50 of them) that did not have all the portals, but I have found a couple with small areas that you could not reach due to all access to them being blocked.
All you can do is look around and go down every corridor again.
