If you fight the Spectator with the drow the difficulty drastically changes.
You have a chance to fight him alone, but you have to get to the solution of the problem before the fight: it's kinda a reward for a smart thought