I found a while back, there are definitely times you don't want your or Gale's familiar out. Found that out on the ship. Haven't had an issue with Wyll's familiars, but I usually don't have them out or I use the Imp and make it go invisible.
Yeah, that's not the issue. I just killed the goblins and walked into the grove, used persuade to stop the fight with Zevlor, then went into my character to level up and Zevlor keeps interrupting me while I'm choosing a level 2 spell babbling about the assault he is investigating and attacks me.