From what Halsin himself tells you after you defeat the goblins and recruit him the Grove had deep internal political issues running well before the nautiloid crash and the goblin incursion.
If you speak with the druids after you defeat the goblins and allow the tiefling refugees to depart peacefully, almost a third of them will voice their unhappiness that you changed Kagha's mind (joining the Shadow Druids) and that Halsin return is not going to improve things because they want complete rejection of outsiders
I think it's fair to say Halsin is not great at politics (and being an ex adventurer that might actually be reasonable) and a let issues fester so much that more than a few druids started considering joining the shadow faction. This is not to say Halsin is evil, he's just not perfect and considering his later actions he was probably not well suited for a leadership role.
