I mean, you have a left hand and a right hand, being able to carry a sword and a shield and play a lute is pretty ridiculous. A sword and a shield and a torch is also pretty weird.
There should certainly be penalties in combat for having to juggle more than two hands worth of stuff. But during exploration, I totally get wanting to make torch use simpler. What if, instead of having a melee loadout button and a ranged loadout button, you can just have some number of loadouts that you can use for whatever you want. So maybe you've got presets that you can swap between quickly:
-a longsword that you can wield two-handed
-the same longsword, but wielded one-handed with a torch in the other hand
-the same longsword, but wielded one-handed with a shield in the other hand
-a torch in one hand and a shield in the other hand
-a longbow
