Well I tried to play my save further as I wanted to see if the wizard being so angry at me would bufu the plot but alas, my sense of direction in Divinity games has always been awful so I finally gave up trying to find the wizard and started over.

This time around the Wizard was more than happy to trade with me, so it must be as mentioned above something wrong with that game mechanic that Larian should look into if they've not already.

I didn't do anything different in the new game from the old game... Picked up rain scroll, picked up various shells and allowed the drunk guards to escort me to the first fight zone, finished fight, picked up loot and then went to the towns front door to meet the wizard.

Oh and in my opinion, what should happen is that the prices should be increased to accomodate the anger that the NPC feels towards the player so that unless the anger is so bottomed out (like -50) then the prices will simply reflect that but trades would still go through unless as said, the anger towards the player is bottomed out at like -50 or -100 or whatever is the lowest possible amount.