I discovered this somewhat by accident. I had saved Sazza, gone into the camp, rescued Halsin, and killed the 3 leaders. Later I went back and discovered that the goblins in the outer courtyard were not gone, but hostile. At first I got overwhelmed by numbers.
Then I noticed the upper level around the area. The question was how to get there without starting combat. I found that you can go to the NW of the windmill where you rescue the Gnome. Go as far NW as you can, then you can jump down to the area below
(you will take some damage from the jump - my chars took about 10 each.) Just get everyone to the lower area, then go to camp and rest overnight. From there you can walk across a stone bridge over the river, right to the upper level around the goblin camp
courtyard. You can take out a few of the goblins using stealth without alerting the other. However, when they are alerted and start combat, they can't get to you. Just park each char less than 1/2 their movement from the balustrade, then move up to the edge,
shoot the goblin with the best % to hit, and move away from the edge and crouch. They never see you and usually can't target you. I took out all of them and the Big Guy taking almost no damage myself. I did this after the 3 leaders, but it might help someone
if they want an easy way to clear the courtyard.