I am not buying the argument that it breaks the balance.
At 10th level, a Warlock has just 2 spell slots. A Wizard has 4+3+3+3+2 = 15 total spell slots. If you allow infinite long rests, there is absolutely zero balance here.
Yup. The long rest system right now kind of sucks. Should be far more limited.