I think it's some kind of memory/event management bug that was introduced in last patch. PC users found out a fix which involves deleting thousands of events that clog up the game and create those huge delays we are experiencing. Unfortunately us PS5 peasants can't do anything but wait for a fix, which i'm sure Larian is working on as this is a known issue. I'd be happy with a rollback tbh.