The only justification Larian gave for 4 party size that makes sense was to prevent TB combat dragging on too much.
But as already pointed there have been plenty of other games that did six (or MORE) party members just fine.
Even specifically in the turn-based subgenre.
XCOM is broadly considered one of the most "fast paced" turn based tactical around and it goes with a six men squad.
Frankly, if the combat may turn into a "slog" is more up to the encounter design and how they'll manage and animate units, rather than a matter of having four party members.
