Your journey taking you to unexpected places is part of the fun. And looking for supplies that you need it's a legit reason to check out the place.
But, only on my 4th playthrough i discovered you can get a quest for the ruins in Druid's Grove by interacting with a journal, next to Arron Trader. But if your Tav is not the type to snoop into other people stuff, it's kind of hidden...you have to stealth to read it.
https://imgur.com/5Zp6zKh
