They didn't bother me but I did feel compelled to go to the Maxos temple asap. I just ignored her till I had done most of Broken Valley. The only voices that did annoy me was your character saying 'ahh, I have been poisoned etc' every few seconds during large fights.