theres another way to go entirely and its how I usually play this bit.
Only mentioning it because you sound so fed up and its heavy spoilers so don't read if you want to retry the other route or just hate spoilers

.
Only caveat is you do need to have Minthara's Spider Lyre, so you'll have had to find it on her body in the goblin camp and looted it and not sold it (I think you can also get one from True Soul Nere in Grymforge). Or I believe you can do a hard speech check if you don't have it but not 100% sure on that.
Go into the shadow cursed lands from the mountain pass, a goblin woman will meet you tell you to use a torch and find the camp - its straight down the road you are on then bear left over a massive tree root, not very far. Little tip, if you have a speech with animals spell activate it before getting to the camp. One of the goblins is playing a rather nasty game of fetch with a rather loveable hyena and is the first goblin you will get close to, you can Paladin or True Soul intimidate the goblin to go fetch his own stick (he'll die, I have no worries about this being unfair as it was exactly what he was doing to the hyena) and then tell the hyena you are a friend and that he needs to be free of the goblins and he'll go as well (I really want to send him to my camp, he's rather adorable), thus reducing the number of enemies without starting a battle. Speak to the head honcho with the goblins and either tell him you have the lyre or pass a speech check. If you have the lyre attack them all quickly (usually about 4 of them). If not you will have to wing it a bit - I always have the lyre so I'm not sure how it works if you don't have it. Follow the quest direction to play the Spider Lyre or just find it in the common actions and do it from there. Kar'niss ( a drider) will appear from behind a tree fairly close by. Tell Kar'niss you killed the goblins (if you did) to honor the Absolute (he's weird and loves it), tell him that you aren't quite ready to leave and then suprise attack him (and any goblins) hard using cleric divine smite and anything else you have handy, he has an immunity move which you just have to push past just don't let up but if you keep at it he'll go down and he actually doesn't fight back very much if you're quick and organised. At the end of the day its one fight with one drider and 4-5 goblins as opposed to waves of enemies at the Inn and if you lose you can reroll without anyone else being affected. Loot his lantern, chat to and release the pixie inside it, she'll do a spell to enable you to resist the shadow curse which is auto transferable to any members of your party as soon as they walk into it for as long as you are in Act 2 and the bell thing she gives you will work if for any reason you go back to act 1 area and then return, use the bell and she'll redo the spell. From where you met the goblins head south west to Last Light. You are already protected so You don't need to speak to Isobel and no one will direct you to her. So you don't trigger the fight. Avoid her entirely in fact as you have no need to go upstairs there at all.