I don't know whether it's a bug or intentionally - I'd believe it if it was declared WAI - but the Ogres are very much a tactical nuke option. If it isn't a party member, it's on their 'kill' list, and they don't stop until there is nothing left but your party, that they are aware of.
