"Upgrade" is a very amibiguous term.
OP: It sounds like you're basing your complaint on the idea that if you were able to upgrade a piece of equipment in a certain way ("...like you can to metal ones"), that you should be able to do the same with every item. That's a big assumption (in this conversation) and I don't think it's necessarily true. In other words, I think it's probably the case that the statistics/attributes of some items cannot be upgraded at all.
This comes from my non-spoiler approach to crafting, which means I know I haven't discovered all the crafting methods yet, so I could be wrong. It could be the case that all attributes for all equipment can be upgraded in some way.
For example, leather chest armor can be improved at a mobile kitchen (camp fire + cooking pot). The "instructions" for this are found in an Adventurer's Field Guide (AFG). Doing so improves the armor value of the piece, whereas improving metal armor at an anvil improves its movement speed.
Whetstones drag and drop onto metal weapons in my inventory (I get the "not enough blacksmithing" message).
According to an AFG, moonstone chips can be added to wooden weapons (and possibly shields?).
As for cloth items, I haven't figured out how to improve these items, but I know there has to be a way. I have a feeling they cannot be upgraded statistically (i.e. more armor, more speed) but can be magically enchanted.
