There should, perhaps, be a way to tell the game "Yes, I'm just attacking all the Elturel refugees wholesale right now" ... But if you do that at any point before the Druids act to expel them then you are, and I want to be clear about this regardless of your character's philosophical alignments - attacking a group of non-combatant refugees who are currently under the ward and protection of the druids circle, of which you are also a guest. Yes, all the druids will attack you as well. Many of them may have been swayed by Kagha's lies, and many may not appreciate the presence of the Elturel refugees in their grove, but that doesn't change the fact that they are, currently, under their protection.
A way to connect with the Shadow druids may be an interesting idea in principle, but every evil faction or enemy organisation cannot be made into an ally-able path or the game will never be finished - some enemies have to remain enemies - I don't think the option to identify as a shadow druid is really a tenable one going forward beyond this one event, in a way that makes valuable sense for the game.
Even Kahga doesn't really want to go that far unless forced to,
She is never forced to. It is her choice. That's kind of the point. You can SAY: "Oh, I don't *want* to genocidally kill this entire village of people, I really don't WANT to, unless I absolutely have to, but you see, they're in the way of my plans, so I HAVE to, they're forcing my hand!!" It doesn't make it true. You weren't forced to - you chose to. This is Kagha's position. She is never forced to do anything. When she murders a small child in front of everyone over petty theft, and then makes excuses to you about protecting her people, and victim-blames the dead girl with racist remarks, that's all her choice. No-one forced her to do it - she took that upon herself.
The point that there are limited supplies is due to the havok being cause by the Absolute goons and the need to lock up and preserve the grove until the threat has passed. Cant gather goods with goblin patrols and a bunch of Tiefs eating it all.
They specifically call out food shortages: This. Is. A. Lie. There is no food shortage and they live on a fresh water river. Kagha claims that the grove cannot support the refugees - this is a bare-faced lie as well. The grove is large enough to support the number of people inside it. If they were all druids or druid affiliates, no-one would be complaining, because the grove is actually very much large enough to support them all comfortably, and there is no food or water shortage at all. The refugees might need to learn to live natively, as the druids do, if they're stuck there for a while, but there is very much no problem with them staying... as, to remind, they were invited to do by the grove's leader, who knows what the grove can and cannot support, and were offered safety and protection by him until the immediate threat was passed.
Citing that it's too dangerous for the druids to go out and gather supplies - they aren't willing to because it's too dangerous - then completely undermines any degree of sense that the "harmonise with nature or die" argument might have once made, for putting the Elturel refugees out to be killed by goblins. IF it's so dangerous that the druids themselves won't go out, then they have NO grounds for using that argument to justify sending non-combatant refugees out there right now.
