Height grants Advantage to attacking and defending.
Advantage works out to be on average approximately +4 to a roll. That's pretty big when you're rolling a D20 - that's an additional 20%.
The system for player AC and player health is 5e standard, and doesn't take into account the extra +4 to hit enemies will get from having high ground, nor the extra damage you'll take from fights dragging out longer from the -4 to hit when players are on the low ground.
Try that Harpy fight from the low ground and see how that feels.
