The way I understood that scene was that all the other druids were afraid of Kagha; for one reason or another. She was the leader in that moment so defying her would be against the rules. Some might have been genuinely afraid of her or maybe she was threatening for some reason hidden in another plot line.
The player character is an outsider, though. If Kagha was curious to see what you could do and lets the situation play out then all others would too.
