The gem is in a cavern in the Crystal forest.
If you are tired of searching for it look at the maps.

Spoiler






This is 2 maps from Hellfighter's site.

The cavern is located at n5 on this map :
Crystal forest south part

You'll find the gem at n2 on this map :
Monster cave

Barta