Mixed party setups are still the optimal parties, because enemies usually favor one kind of armor so you need to be flexible.

And characters can deal both kinds of damage. Archers easily deal both because of elemental arrows and traps. Scoundrels just got a huge buff and can strip a ton of magical armor (Gag Order). Summoners can easily deal both kinds of damage. And mages could always cause decent physical damage via Necromancy or by using a shield.