It is in fact only out of combat that the auto thing happens, except maybe for enemies.