I don't think this is a good idea.
BG3 already ignores somatic components, why would it not ignore material ones then.
Somatic and material components work in a PnP because you describe what you do loosely. A video game can never make a equipment/inventory system work as seamless as it does in a PnP. Especially not with the basic inventory system we have now.
So why even try.
Another one who never plays D&D eh? normal components belong in the pouch, while extravagant ones can fit in the existing inventory system all right.
Not sure why you would assume that, considering material components can be used individually, especially when you play 1/2 or 1/3 casters who typically don't have a lot of spells.
Anyway, just to re-iterate my point: I think if a player had to unequip/sheathe a weapon (in a sword & board/two weapon case) each time they wanted to cast a spell then that would slow down combat. So if we don't care about the somatic component, what is the point of enforcing material components, if all you do is have it in your inventory.
Theoretically, we could make a component pouch/focus an item you need to equip in a weapon slot and swap those out. But then some classes like cleric/paladin can have a holy symbol on their shield/as an amulet. Martial classes like Arcane Trickster are 1/3 casters and then would need to have access to a ranged weapon, dual finesse weapons, and a focus? But this is getting way more complicated than it needs to be just to give somebody a reagent pouch.
Plus, how many times do you actually describe using a focus/component in a game. In my experience players gloss over the fact they are tracing signs and holding a focus and just say what they are casting. Sure the first time your sorcerer casts Fireball they describe it, but do they do that the 20th time they do?
So why enforce animations and/or equipped item management systems for something that typically get's glossed over in 5e.