That may well be but its never actually provided as the reason of mistrust. Nor did the possible lack of food.
Both which are solved by enabling the refugees to do what they actually want to do - hit the road to Baldurs Gate. Which they cant do because of the Absolute forces. That druids hilariously just dont care about and apparently cant even think about.
The fact that the stealing of the Silvanus idol is serious offense to the Druids is not in dispute. The reason why it was stolen and intentional removal of all obvious common sense dialogue options about it - is.
The druids dont just have to create food magically, and there is no need for "depletion of natural resources" - not in the least because Druids can teach others how to grow food without depleting the natural resources. They can use their magic and nature knowledge to help people grow food faster, better. There is a whole godamn river right next to the grove. And the point is - the situation is not an ordinary situation - and the refugees dont want to stay there.
The threat and danger to the grove literally dont have anything to do with the refugees. At best its a misunderstanding - with plenty of direct evidence against it, including the attack on the gates and all other attacks and destruction in the area. Including the attack on the Archdruid showing its not just goblins behaving as usual - only he keeps that a secret from the rest. Except Nettie but she doesnt say anything to anyone either so its irrelevant in this sense. The goblins are not to be considered a threat just because they are goblins - but because they are a part of attacks on the whole area and the grove.
Closing the grove in this situation would have very specific consequences - if the player doesnt intervene. Not any good ones. Its the opposite of what should be done and exposes the grove to eventual destruction.
Because its not just goblins behaving as usual. Not to mention that putting that kid in a jail is directly opposite to what Kagha apparently wants.
And so on...