So ... should I start arguing that stealing (which I assume many enjoy) should be removed from the game because its effect on game balance is detrimental to my characters who don't use it? Or should I continue to ignore the guides that say you have to steal and quietly shoulder the bit of added difficulty that comes with playing a virtuous character? And to appreciate that not stealing would have no meaning if stealing were not an option?

The price of freedom is always going to be individual responsibility. Part of role-playing is choosing what sort of character you're going to play and choice only exists in the presence of options. To my mind "don't like it, don't use it" is pretty basic to RP.