What's crazy, leaving open slots for gold charms or trying to use at least silver charms?
I used one charm (gold strength, dropped by the dragon rider Zandalor killed) the whole game, but converted my loot to charms. At the end of the game I had;
66 gold / very large charms (found 6 before hitting the gong)
59 silver / large charms
24 red / medium charms
19 blue / minor charms
1 green / weak charm