I think just adding 2 more party members would be a buff enough.
It's not even a matter of power, frankly. Six party members would make party composition better simply because they offer way more possible combinations, more room for non-essential roles and give you chances to put to use a way broader variety of interesting loot.
I'd be fine even with making fights twice as hard, for all I care about that. That's a completely separate issue, anyway.
