Yeah... but because it is this way in 5e it will stay.
There is also other reasons. If you get it like you say. It change the game balance. It makes multiclassing, much much more powerfull and makes remaining in the single class less powerfull and that is the main problem.
I understand your suggestion (it makes multiclassing more simple and more powerful), but it would open a worm can so big that I hope that this does not happen! IMHO
