It's not really about time, it's more about shutting off your brain or not.
Anyway I guess you never played Blackguards ? Sometimes ennemies turns were very very boring because you had many ennemies and nothing to do during their turn (as in every TB games)... but that's not really what I pointed at.