Actually, there is a chance to get crappy dragon armour, though the pieces of that set are potentially one of the best in each armour category.
I would only use silver (large) charms now, and leave at least a few open slots for gold (very large) charms, which become more plentiful towards the end of the game (some can be dropped and one found in the Dark Forest).