Yeah, it's annoying. I could've sworn that at first in the demo, the game had a prompt that said "these guys don't attack you, are you sure you want to chop 'em up?" That helped.