If you want more immersion, then ask for merchants that make more sense. A lost spirit in a hidden or underground library that sell spell books for all school of magic for example instead of so many merchants all over Fort Joy.
That is what we are talking about. There are far to many therefore ruining economy and immersion at the same time.
If stuff isn't reasonable scarce, high prices are not understandable and therefore will just feel punishing for players. But if stuff is obvious hard to get, high prices are more logical. And if you give options like quests to lower prices you even improve the play value.