Where does the shield go if it's not equipped? Usually, it hangs on the character's back, or it should. Now, in most video games, that shield that's supposed to be there isn't shown, but it should be.
So? Having a heavy wooden object hanging of your back doesn’t exactly help you - if anything it should add negative AC. By the same token an unequipped weapon might do damage to an attacker cause “hey, you turn around and the mace smacks an enemy in the balls by accident”.
The key point is that you're likely wearing other armor. E.g., "Having a heavy wooden object hanging off of your back
[while you're wearing plate armor] doesn't exactly help you."
It probably helps more for characters wearing lighter armor, but on the other hand having a large object on your back will negatively impact your mobility, roughly canceling out.