Yes, you can just click the icon to leave the BF; when you enter again the inventories should be different.
What version of the game do you have? (language and version number, shown in the bottom right corner of the main menu)
I think it was the 1.40 patch that change the BF merchant inventories to regenerate each time you entered the BF. The 1.43 patch added a small chance for the BF merchants to carry holy water (required to make permanent potions), so that change had to have been made by then.
If you have an original release version, the English patch can be downloaded
here (that page has a link to the German patch; don't know about other localizations).
