I like the core idea behind armor system but I personally don't think the current implementation is good for the game. Just like you said it forces you to focus on either Physical or Magical because if you do not, you have to go through 3 different buffers instead of just 2. Making almost all encounters more tedious than it needs to be and for what exactly? I can't see any reason other than just goofing around.
My solution to this problem would be to shatter both armors if one of them breaks. This would further encourage hybrid parties and would give a new meaning to skills like Chloroform.
Other than that I do not have much problems with armor system. I think CC part should remain the same.
This suggestion at first glance might feel like it might make the game way too easier but remember this goes both ways.
