Gems are not charms. Charms are the 1 which you mentioned adding few stats points, where as gems and ores are use for enchantments on your gears and weapons, such as adding 6 range defence, +15 melee damage and so on. You need to gain recipe for enchantments before you can enchant the stats you like.

[Linked Image]