I think you should continue with your plan to rescue Zandalor (sewer entrance to the castle treasure room), then let him teleport you somewhere and clear that area....

You do not need to go to the Dark Forest first.