I agree that the Berserker is particularly powerful if exhaustion is not a thing. I also think the idea of getting a level of exhaustion after going down could be an interesting addition (I even use it as a homebrew rule in tabletop D&D), and like you said, could add more use to clerics.
I also understand that at the moment, there is so much food in the game that you can basically rest as often as you like, so this would need to be tweaked in order to make sense in tandem with an exhaustion system.
Maybe we don't need all 6 levels of exhaustion implemented - maybe one effect that covers exhaustion is enough to balance the Berserker. It would need to be tested, of course.
