im having huge issues too. no crashes or slow loading textures tho.
from early access, to official release and up to august 23rd i was playing on ultra settings, dlss balanced and my fps were 85-110 depending on the area.
In lower baldurs gate my fps were between 55-75 with some drops here and there to jut annoy me but nothing major.

After the "fix" from patch 2 and the hotfixes the game in lower baldurs gate (for me) is literally unplayable. dropping to 10fps for a pretty long period of time (4-5 seconds) very often to the point i get nausea. vomit inducing bad.
in other places everything is seemingly normal, but in baldurs gate the games performance is taking a huuuuuuuuuuuuge nosedive that it shouldnt with a 5800x3d and a 3080ti... especially since it never did it before the patch and the hotfixes.

oh i always used vulkan (swapped to dx11 to see if there is any difference, but spoiler alert : it was the same).