I got my rogue to sneak in and made my mage use invisibility spell to get past them. IIRC when you switch to a different area with a loading screen your entire group will magically join you on the other side.

Overall a bad area IMO. required a lot of meta gaming and save scumming.