Do a search on the forum for the word "Fergus" (search for the last 3 months). You will find lots of posts and strategies on handling him. It seems that sometimes Fergus can be one of the toughest fights in the game.

Good Luck.