Enemies take 1 durability per hit... but other objects (like the tomb) take 3 durability per hit.

This seems to be by design to dissuade using weapons on them. However, it has the side-effect that spells can do it free anyway...