The damage you deal and the damage you take is still level dependant, not to mention the stat points you are missing. Vitality and dex around 60, plus your character level of 36 makes for a bad combination of the ending fight.
If i remember correctly, Ygerna and Behrlin are around leve 38-40. The best weapons in FoV are also for 38+.
I'm afraid you simply started the ending sequence a bit too soon. You should have a few more levels and better stats (via charms and enchants as well). Some more skills above level 5 would also help in dishing out damage.
