I found this mod, seems to have a fix for it:
https://www.nexusmods.com/baldursgate3/mods/998