I actually like this idea, why not pair it up with armor?
When your armor is gone your armor literaly breaks in combat, your sword literaly breaks too.
I like that, maybe also ahve a normal durability but realy thats mostly so breaking down doors has some form of limitation.
