That does sound annoying. I've not had that happen.

If you're able to replicate the issue and have a savegame and haven't already done so, this definitely sounds worth sending to Larian (the Report a Bug link on the launcher will take you to the right place).

"You may call it 'nonsense' if you like, but I've heard nonsense, compared with which that would be as sensible as a dictionary!"