Ore and gems are required for enchantments, which can be used to add various bonuses to equipment that have enchantment slots (Increased Damage, Life Line, Poison Aura, etc). You also need the formula and an enchanter (the first is available after you get access to the full Broken Valley, and is reachable from about level 12, or so). You can remove enchantments on equipment, as well, which will give you the formula if you do not already have it (you do not recover the ingredients).
Similarly, plants can be used to make potions, if you have the formula and talk to an alchemist (the first available is in the starting town in Broken Valley).