This is a fairly early fight. Gameplay dictates it should be reasonably winnable. If you had to fight Kagha, a strong Olodan, two of her minions, several druids in the Grove AND more strong opponents, it might be a rage quit moment for many. The druids that fight on your side are pretty weak.

I think it makes sense for the Shadow Druids to fight in the Grove because they haven't given up on taking it.

But I like the idea of having some different Shadow Druids ambush you later in the world, especially since it is part of a broader conspiracy
to take the druids out of the coming battle.
There isn't much to Act 1a left by then, but I am sure they could track you to the Underdark. The Underdark doesn't have many challenges until you get to Grymforge.