Funny story. I was in the underdark near a collection of those green mushrooms. As I was plotting my path, my character would freeze, pull out her daggers, then stay put until the animation for sheathing her daggers was complete. I reloaded the save, and everything was fixed until I approached my mushrooms again. It took me a hot minute to figure out what was going on.
If your mouse hovers an item that can be attacked, like a barrel or mushroom, the game assumes that you want to attack it. Your character will stop what they're doing, pull out their weapon, and plot a path to the object.
Why is the default interaction for these objects "attack" in the first place? If I see an exploding barrel, the last thing I want to do is strike it with my sword.
It's counterintuitive as well. I was playing coop with a buddy, and he clicked on a barrel, assuming the default interaction was pick up. Nope. He got himself killed.