i like the armor system fine, it could use some tweaking, maybe with more individual spells (of the non-cc variety) going through them
i think the real issue is the stat inflation at higher difficulties. there are a lot of ways to increase difficulty during combat that don't involve simply making the fight a longer and more monotonous version of what happens on lower difficulties. i'm going to assume that most people who hate the armor system are people who play on higher difficulties like tactician and dislike wasting turn after turn burning through bloated armor before anything interesting happens. this is a fair complaint, but again, i think this problem can be solved without overhauling the whole system. in my experience having separate armor stats for physical and magical (which, on most enemies, has one significantly higher than the other) has overall made fights interesting in terms of who focuses which enemies. i always bring mixed parties (usually 2 mages and 2 physical damage dealers, though it becomes more complex and less tied down as the game goes on) and, while fights certainly take longer, they're still manageable. if difficulty was achieved in ways other than simply making fights longer versions of their lower difficulty counterparts i think people would find the system a lot more to their liking
