An easier fix, that would be more in line with how druid is supposed to function in 5e would be to disable inventory completely while shifted and treat it as if you had an empty inventory, as when a druid shapeshifts the items they have on their person fuses into their new form and becomes a part of them until they return to normal.
I wish they just had a thumbs up, but this is the ideal method. If you are wild shaped, you can't switch inventory because it's part of your body. You would need to shift back to switch things out. This would be the logical approach.
