I think this should have been resolved already? From the v.1.047 patch notes:

-Fixed the repeating voice problem that drove everybody bananas
-Muted banter voices while in dialog or trade

Link: http://store.steampowered.com/news/?appids=230230

I only played a little then and was not much in town but I thought it was reduced now....