kill all of the enemies + rats first.

1st time I didn't kill any rats and got defeated fergus pretty easily
2nd time I killed the rats first and won pretty easily

It might just be the difference you need.
Also don't bother about the imp just have your Hero + DK have a go at fergus at the same time.