Fast travel in dungeons also should be removed.
You still would be able to take long rests, but would have to travel long way in and out of dungeon every time.
I don't think long rests should be penalized to the extend that you suggest. Maybe on higher difficulty settings, but still.
I don't want a game I play to have busy work like having to run out of the dungeon. Hell, in DnD campaigns leaving a dungeon can be as fast as saying "we leave the dungeon". Which is effectively "fast travel"