I got attacked by Wyll after the fight at the grove entrance ended. Could be something to do with surfaces dealing damage and the neutral AI attributing it to the player because there aren't any enemies around?
I was able to recreate the problem if I go to a camp with a familiar summoned.. Whoever is not part of the group will attack the party.