Haha I actually completely forgot about the statboosts as I play with my mod which completely disables any skill or ability boosts from armor and weapons. And things like critmultiplier already only apply to the weapon you attack with. Tried that by modding a shield to have x2 crit damage and it didn't do anything when attacking with my sword.
Maybe just add offhanddaggers which don't have alot of statboosts and increase dodge chance. Like in reality. (parry dagger, noone would use a normal dagger in the offhand in combat)