Hmm... I'll only play on HM or HM-rules these days and I never had much trouble with this one as you can basically "cheat" it like many other self-same trials in similar games. Pretty sure DA:O had one that could be cheated this way. The "cheat" is to strip your entire party of all weapons, armor and equipment before you activate the blood-bowl.
I'll usually have them all naked just to be safe (not because I'm a pervert or anything) and then the duplicates will also spawn nude and gear-less. Cloned Karlach for example will only ever attack unarmed like that - if she even manages to attack at all because this fight will usually only last a turn ... maybe two. Just don't forget to re-equip all your stuff before combat starts, of course.
Another trick is to put everyone on sneak-mode, split up the party and sneak your two characters whose clones are at the rear into position behind them. That way at least one of your guys will be able to open with advantage plus you'll surprise the enemy party and gain a full turn on which they can't do anything. I'll usually have everyone but Tav sneak around to the back area and then open by attacking one of the two rear-most clones (they'll be positioned on a flight of stairs), depending on who is positioned where. You *probably* want to take out someone like Gale first because he's more dangerous naked and unarmed than, say, naked and unarmed Barb Karlach.
So there really is no need to incur Shar's curse, because with all that help, everyone should be more than capable of dealing with their own counterpart if you use the right attacks and are mindful of the enemies' resistances. Also note that you *can* interfere with another character so long as you don't damage a "strange" clone. You can have SH throw a water-bottle near clone-Karlach to make her wet for example. Probably shouldn't *hit* her with the bottle though..
As for the invisibility thing: I'm pretty sure you need to pass that perception-check (at least I think it's perception) once you get close enough to the combat area in order to detect the ambush - so be sure to put Guidance or even Owl's Wisdom on your guy with the highest WIS/Perception and/or highest chance to spot ambushes. There are also items that can help with this - both the Halberd of Vigilance and the Sentinel Shield are available at this point and both give advantage on Perception-checks.
