Happens in DX11 as well. Seems to be related to changing areas. Randomly after changing area the game gets laggy. I guess that assets unloading/management is broken, creating an asset memory leak.

Only solution right now: Save and restart.