[color:"orange"]Why would it bother you so much, if you are just as guilty?[/color]
Is that the question? But then you assume, or even presume, that it does bother.
'You are just as guilty' - you are prejudicing a moral issue by implying 'guilt', on either side actually. Now, does this guiltfeeling come from within you, through your personal ethical standards - or is this some obscure moralizing that society has imbued you with? (Hard to distinguish at times, I concede)
What is 'bother'? - Hurt pride? Deceived trust? Loss of 'possession'?
Of course, if it would not bother you, the whole argument would be futile. You would be relieved to have 'shared guilt', and as such be able to switch from defending to accusing position (Even if only for yourself). I am not arguing that that would benefit the relation...
I personnally lack the experience of being caught in the act. I have been cheated, but found out (was told) only at a much later stage, when the crisis in the relationship not only had been overcome, but was at a stage where 'forgiving' was highly probable (or is this 'I had won' male vanity?). And I have been the one someone else (a stranger to me) was cheated with (I am talking of a 4 year relationship, not a spontaneous fit of lust). Did I feel 'guilt'? Not really - Did she? Well, let's say it was a burden in our relation, that we carried.