If you don't want to talk to you mushrooms, just don't talk to them. Or if you hate it so much you can kill them.
Their help is not necessary to progress through the game.
Explore the forest area and you will find the solution. If you are really stuck:
Go left up the ramp past the area where you fight a bunch of nectar drunk goblins. It's heavily trapped, that's why there is a Featherfall Scroll provided by the barrels. Perception potion recommended you if you don't have a character with decent perception. You can dig up the spell to remove the barrier in a hidden mound.
