Just keep sending the gem gatherer out for diamonds... then sell them
A quicker method, which doesn't require you to constantly leave the tower and come back, is to complete Barbatos Quest and get his spell book. The spell book has three potions that he'll make for free. Have him make a spell from the book, wait 15 seconds, then ask him for the potion. Each potion sells for about 135 gold. Granted, your gem finder may come back with 10 diamonds at once if you're extremely lucky... but there's also a chance he'll still be injured when he comes back and bring back nothing.
I use a combination of both Diamonds and potions and have my dragon stone take me to a seller with gold.