That's exactly what happened to me on my first playthrough. I talked to the mushrooms, then I did almost all quests in the forest before finding that silly scroll. Some of the quests were really hard at that point, however later on the game just got too easy. I do appreciate the non-linearity of the story but still -- a few more hints would have been helpful.
