Yes it is flavour, but it can be made meaningful.
I concur that my initial idea of having no defence animation for bad misses (i.e. far too low rolls to hit the target's AC) is not a good one. So, to visually distinguish easy fights from difficult fights, characters can dodge low-roll attacks regardless of armour. This way, if you see your tank dodging all the time, it tells you that this encounter is a piece of cake; and when they block / parry all the time, you need to actually pay attention to enemies' attack rolls - maybe you need to add some extra defence.
