But if you have 4 party members and 6 enemies, it makes for 10 turns per round. 6 party members and 6 enemies: 12 turns. But indeed, it might mean that you will feel more agency througout the fight, as the ratio of allies/enemies will be better, if they go with "buff enemies" rather than "add more". One problem I could potentially see with this is that buffing enemies in a meaningful way (not just pump numbers) requires more effort than adding more of the same... Or not, if the enemies are unique and would require new models/animations/voices. Still, rebalancing might be more complicated than it would seem. But I think we might have a better idea of this when we see the difficulty options, which are said to be more interesting than "buff numbers". Perhaps a solution would be "bump this type of difficulty settings by 1 for 5-man party and by 2 for 6-man party to experience a similar level of difficulty".
(I'm playing a bit of a devil's advocate here, I would strongly prefer a party of 6. I mentioned the "parroted" argument because it seemed like a legitimate concern, I will be happy to be proven wrong.)