Not sure if it is supposed to work this way or not, but durability doesn't seem to go down at all (or at least extremely extremely slowly) when using special abilities.
I just tested it with both melee axe and range crossbow. I shot several benches, barrels, tables, doors, etc to test. Shooting them with the crossbow with a regular CTRL + click attack lowered durability. Shooting them repeatedly over and over until destroyed with barrage or arrow spray never lowered durability once.
Same with melee. Destroyed 4 benches and 2 tables down by the entertainers. No durability lost with Rush and Battering Ram at all. But just 4 straight CTRL+ click attacks finally destroyed 1 bench and lost 10 durability on my axe.
I have the My Precious on all characters, so I can't say how it fares without that talent.