Githyanki are VERY proud and arrogant people. If a Gith finds out you're wearing Gith armor, the assumption should be that you killed a Gith and stole their armor.
Maybe they are too proud to acknowledge that you might have killed a Gith? So they just stare, sweat and behave like assholes while not acknowledging the armour. 😊
