At first I was opposed, and annoyed, by this mechanic as well. Over time I got used to it and can see its positive influences. Having a hybrid party is not a downside at all. You need mages to take down magic armor of the warriors, while warriors take down the phys armor of mages. And both your melee and magic characters have access to CC abilities that can assist in both situations.
No, you do not. You just stack one damage type and kill the enemy. You literally don't ever need hybrid damage types for anything at all. In the first act, there are places where people have 100% magical armor or 100% physical armor but this is generally limited to act 1. After that, the armor values differ but the strategy of "stack damage and kill the enemy" is still the most effective way to play the game - as measured in "least number of turns to win the fight". So yes, having a party which is a hybrid of elemental damage and physical damage has a downside - it takes you longer to win fights. This isn't really debatable unless you're limiting your conversation to act 1.
