AFAIR ginseng is a little scarce on Sentinel Island, and in Broken Valley it is certainly less common than fatfern / fanny blossom / wisperwood. Before getting the battle tower I had 8 ginseng in my inventory, and had made a few potions a couple of times, so possibly used another 20 or 30.
