This fight is extremely easy if you only send one character down to talk, and keep the rest of your party at a height advantage. If you just King of the Hill it, and give the party member who went down a misty step scroll to get back to high ground, you will have no problem taking them out just like King of the Hill combat works in all the other fights. Heck, I soloed this fight with Astarion using a high ground ranged attack + hide combo, where none of the enemies knew where I was after my turn, so they just skipped their turns, rinse and repeat.
That being said, I also really agree that it was silly for the dragon to run away. I tried to murder hobo this encounter the first time I played and just started attacking them from on top of the gate, targeting the dragon first, and I was surprised that it just ran away. I was expecting it to come kill me, for the game to punish me for trying to fight a grown dragon at lvl 4. But without any context from a conversation with the gith, it made no sense that the dragon immediately leaves when it's attacked. The gith didn't even know who the attackers were, yet the dragon still just goes "nah man" and runs away lol.