That sounds odd to me, I suggest you go get the quest again at the summon master and then visit the tunnels again after you got the book for summoning ingredients, and see if the mushroom has re-appeared.

//Inel.

Don't play when you have a mushroom phobia! o.O xD