As the Spectator will activate the Drow statues, making this a very tough fight, I found it useful to destroy as many of them as possible before taking on the Spectator itself. You need a blunt weapon (at this point you should have the named Mace from the Drow leader in the Goblin camp) as they are immune to most types of damage. Sneak around smashing as many of them as you can find before activating the Spectator by moving a character down to the treasure horde. You get experience points for smashing the statues, so nothing is lost. You can also use this time to position your party for the coming fight. To get down to Spectator area, don't forget Feather Fall and good old Misty Step. To get out of the area, you can return to the waypoint in the fort.