They know where it is because there are goblins the first time you come to the grove. They found it, but it looks like Minthara's scout are even more stupid as they should be.
We have:
A group of goblins, which chased those mercs and came to the grove that way. They didn't know where exactly they are going and since we killed them they couldn't tell anyone.
A group of goblin scouts that's totally stuck in the tunnel under the grove. Because of traps and our possible involvement. If I'm not mistaken the goblin which talks to Minthara even refers to some scouts that did not return.
If we check the map on her table we can see that she had an idea about the grove position, she had problems with pinpointing the entrance.
And here we probably coming to more level-design, than narration itself problem: the entrance should be hidden better. Put some trees, bushes, maybe an illusion and that's it.