I wouldnt say impossible ...
But in some cases its quite clearly unfair.
I just recently finished my Durge playtrough ... and there i accepted Orin's challenge to a 1v1 duel ...
The 1v1 fight has to be the hardest variant. I bearly got through on DUrge without slayer form on Tactictian and it was all thanks to my mummy, 2 ghouls and the Dance Macabre.
So Wizard Necromancer may be a tactic to any who are struggling. Since all your adds can join in the fight.