It'll certainly be a lot more immersive to see people not wearing armor/travel gear during their downtime, and lying on a bedroll in full plate (or attempting to do the elvish trance) is just plain unhealthy. The dream scenes would look less goofy also, what with no longer holding your lover's hand wearing a metal gauntlet.
Optional only. Best way to implement this would be for us to find more outfits and then we have the option to change into them if they are in our inventory (or pull them out of the camp chest) much like we can choose what to eat now. Would let those of us who sleep in armour continue to do so and those who wish to change also be able to do so, you just have to remember to do it.