You only need to wear the complete dragon armour set once to get the experience and skill; after than you can switch back to whatever you want.