I have the same issue. I'm running with DX11 (Vulkan just doesn't work with HDR) and I get smooth framerates for a while and then it just falls off a cliff. It isn't the environment as I can save in a spot with bad framerate, close the game, relaunch it and reload which results in smooth framerates.