Certain types of plate gives innate resistance bonuses and leather armors have innate initiative bonuses. There is a little bit of specialization there.

For the most part though, if you have the stats, wear what you want. The world is your oyster.