Gems are merely for selling them - a few are exceptions - are you still in Act 2? Gems don't embellish.

Odd gems => fire elemental eggs, you can sell them or fight the hatching elementals.
Diamonds => in the lava cave are merely a trail for the imp in there to get out. I left them on the floor, as the don't bring cash.

Crystals => can go into your crystal bag from Act 1 to increase your resistances. You can take out the weaker ones to swap them with the larger (more powerful) ones.