I tried to get 4 with the first malachite vein in the depleted ore mine, but gave up and settled for 2 (didn't notice I re-loaded the wrong save until after I had deleted the one with 3 gems). The last malachite vein I tried, the semi-hidden one in the Primordial cave, I did get 4 out of, but if I had gotten 3 first I would have taken them.
I don't recall a chest by the veins, but the walkthrough states "There's a key hidden on a shelf straight south of the locked chest it opens."
