Thats bcs this game is loading whooooooole map at once.
Still they should have figured out a way to correct this as long as the game has been in Early Access. Even Skyrim which is a lot bigger game with more resources to load takes next to nothing to load.
Thats bcs Skyrim works differently ...
In Skyrim only nearby surrounding is loaded ... if your pc is slow enough you can even see textures (and sometimes even characters) getting loaded.
So if you are for example in Riverwood, game dont have Whiterun loaded ... bcs its purely single player game ...
Here, if you are at beach, game need to load everything ... including Waukeen's rest, Goblin camp, Hag's Teahouse or Gnoll Encounter.
Reason is possibility of Multiplayer.
Since every player can go different way, game needs to chceck whole world state at once sinve every action any of them do will affect the others and bcs game provides info about their deeds in real time.
So i would not expect this to change.
Now mentioning this ...
I wonder how much performance would timed events take. :-/