It can help to summon the ogres if you have the horn. They soak up some of the heavy attacks that would otherwise be going to your party.
Disarming the githyanki helps. Sarth isn't much without a greatsword.
Maybe wear the minotaur's amulet, the one that gives resistance to slashing damage at the cost of a vulnerability to bludgeoning. Since none of the gith are using blunt weapons, there's no drawback to the item in this fight. And it will give some resistance.
Use your potions when you can in the fight. An invisibility potion, even after the fight starts, can save the day. A void bulb can pull together several of the gith if you want to get them in a group for something like a cloud of daggers.