Larian Studios Forums Divinity - Original Sin Divinity Original Sin - Mods Some notes on character script variables

It's already common knowledge that object variables (wether global or local) keep their values (wether assigned as initial value or from scripted processes) until a new value gets assigned, but are never 'emptied' to null or 'reset' to their initial value (s. Some tips for programming charScripts/itemScripts and Object Scripting Optimizations, thanks to the Tinkerer for the confirmation). Especially the need to optimize scripts, not least to improve saving and loading times (s. Ameranth's thread above), provokes further questions of how/when exactly the engine saves variables and if there are differences between global and local objects (there are!), so finally: if removing or changing object variables in scripts for existing games is possible. Spoiler:

I did some testing with characters to get a better understanding (should be transferable to items). Here are the results:

Global characters

Both local and global variables of global characters are saved in the save file regardless of the level they're located in and the level the player is in when he saves the game, and regardless of if their script was initialized on level load after adding the variables (or ever before) .
Whenever I added a variable with an initial value to the 'Base' character script and added a reaction that has this value of this variable as condition, saved and exited the game, changed the initial value of the variable and checked for this value in the reaction: when I loaded the save every global character in every level of the game remembered the original value and refused to perform the action, although the scripts of most characters were never initialized. So: Once you save the game, all variables of global characters are set in stone and can't be changed in or removed from the script any more.

Local characters

Variables of local characters are saved differently:

Local and global variables are saved in the save file when the script was initialized on level load; there's no difference between local vars in VARS and global vars in INIT in terms of when they are saved. It doesn't seem to matter if the game is saved in a different level or in the same level they're located in, as long as the script was initialized before.
Variable values of local characters can be changed by script call ('Set' within the same script or 'SetVar' for variables in other scripts) as long as the player is in the same level; the values will be saved, regardless of the level the player is in when he saves the game.
Note: Local characters can't be addressed and are completely unresponsive if the player isn't in the same level. Their scripts also don't react to events in the EVENTS section (OnCharacterStatus, OnGlobalEventSet, OnCharacterEvent ...). Only scripts of global characters do!


Now, what happens if we remove a variable from the script?

Both local and global variables can't be used when removed; if a specific value is checked as a condition it will always be true, regardless of the required value. Though they're still remembered by the game (once they were saved in the save file), even if the game is saved without them later and loaded from this save. When I reintroduced these variables in the script it didn't take the new value I gave them but took the original value the game remembered. This seems to apply both to local and global characters.

What if we remove the whole reaction, along with all VARS, and save the game?

The local variables are still remembered. When I reintroduced the reaction the original values of these variables were still remembered.
Only reaction priorities, which are similarly remembered as variables (so changing them in the script doesn't apply to save games, they must be changed with a call), can be changed by removing the reaction, saving and adding the reaction with new priority.

Conclusion

All in all, I see no way to remove variables from the game once saved in the save file. The only case where variables can be changed or deleted in the script is when variables belong to local characters and weren't loaded yet on level load.

It would be great if these observations could be confirmed by those with more insight!

Great sleuthing, Abraxas!


Global characters:
On a related note, my development revealed that OnInit() events of global characters were fired multiple times in rapid succession (I noted 3 to >10, depending on PC) when a new playthrough begins. Thereafter, OnInit(), along with variable re-initialization, appeared to occur at least on level load; potentially even after re-encountering an object after having left it beyond a certain radius.



So these variables are saved permanently, yet they appear to always be reinitialized on level load. I've had a few cases during development where this became a problem, because there is no way to maintain a "flag" variable within a local object's script (that will at some point be walked away from).



More specifically, I believe a player needs to be within a short range of local characters (roughly 35 meters?) to have their scripts function.



Can definitely confirm this, I dealt with this problem a number of times.



I had no idea... good to know!



I'm interested to know if removing the object via a scripted Destroy() (I think it's called Destroy) call will remove the variables from the save. Likewise, does death or being offstage affect this whatsoever? I'm relatively sure that death does not, as I've tested that scripts continue to function from the corpses of deceased characters.



Thanks for the hard work Abraxas, very informative!

Good to know for some cases where execution amount matters.


Note sure if I understand you right. Can you elaborate a bit?


From my testings this only applies to the character's BEHAVIOUR (and that function of the engine which the OnActivate / OnDeactivate comments call the 'script controller': controlling priorities, usage, conditions, execution etc.). The EVENTS section remains ready to receive events and execute actions, regardless of if the character is active (in player range) or not.
Local characters, as long as the player is in the same level, are able to answer events, e. g. ('LUC_MineQuarrel_Cultist_02' is a local character, the event is added to the 'Base' char script):


LUC_MineQuarrel_Cultist_02 always applies WEAK on Player1 and adds a pumpkin (to make sure no other character answered), no matter how far away he is in the level. Global characters answer from every point of the world, regardless of the level.


Thanks for the Destroy hint. I regarded it for testing and, indeed, it makes a big difference! Details in the following post (can't add it to the initial post without losing the ability to edit it; I hope this obstacle will be changed in the nearer future!).

That is correct! And it's still the same in DOS2.

If I understood your post before, local object's variables are forever kept in the save file once they are added, even after the player leaves their level. Yet, the variables reinitialize on level load. So, if the object has a script with an event like this:

The function will potentially be called more than one time, because _Flag will reinitialize to zero, even though the game may have saved _Flag as 1. This would even be the case if _Flag was instead %Flag--defined as a global variable instead.



Thanks for the clarification!



Looking forward to the post I have a system in place within Epic Encounters to take advantage of this--deleting characters who have died during combat-- but I was afraid to start deleting creatures on a broad scale due to the potential for deleting an NPC that would be used in some way after death. As a trial, I instead set them OffStage, and there haven't been any bug reports with just that for a couple months.



Hey Tinkerer Thanks for the confirmation!

All right. This time I followed Ameranth's murderous order and killed around one thousand NPCs. I didn't do this lightly, but it was necessary.

I operated with 'CharacterDie' (which should be identical with the common way to die in Divinity Original Sin) and with 'CharacterDestroy' commands (as Ameranth suggested) and had a look both on save file size and variable entries in the save files themselves.

First a few notes on the variable entries in save files:

As far as I see, all variable names are saved in the 'globals.lsf' file you get when you unpack the .lsv file:

Global variables have one single entry for a specific variable name (e. g. 'PrevIdleAnim'), regardless of how many characters have a variable of this name.

Local variables have one single entry for every combination of variable name and reaction/event/scriptframe name (e. g. 'CrawlingInfestation_Removed.spiderLevel' with 'spiderLevel' being the variable name).

Every variable name entry gets a 'MapValue'. By counting the appearances of this value in 'globals.lsf', where all global characters are saved, and in the levelcache files (also part of the save .lsv), where local characters are saved for each level (e. g. 'lucullaforest.lsf'), I could see how many of my unique variables were removed from the game by killing or destroying characters. Since every character had these variable names only once in their scripts they had only one entry with the MapValue of this variable (so amount of removed MapValue appearances = amount of removed characters = amount of removed variables of this name).

I only added global variables to characters. I suppose that local variables are removed along with global variables when they're getting removed. Spoiler:

Here are the results:

I: 'Base' script without new variables, saved in Luculla, at the level entrance to Cyseal

All Luculla chars alive: 6,33 MB save file size
Local LUC chars dead: 6,35 MB (+ 0.02)
All LUC chars dead: 6,50 MB (+ 0.17)

-->Instead of decreasing the save file size, it even slightly raised the size for local characters and notably increased it for global characters (which probably came from story progress, triggered by the death of important characters; I got journal entries and XP as reward).

Looking into the save files:

The amount of variable entries didn't change. All characters remained in the save file (can be found via GUID).

II: 'Base' script with 40 new global variables, saved in Luculla, at the level entrance to Cyseal

All LUC chars alive: 6,51 MB
Local LUC chars dead: 6,53 (+ 0.02)
All LUC chars dead: 6,68 MB (+ 0.17)

As we see: introducing 40 global variables into the game (which were saved for all global characters in the game and for all local characters in Luculla, since I never leaved Luculla during this testing session) increased the save file size by 0.18 MB. This difference didn't change after the death of all Chars in Luculla. It didn't approximate to the size of testing session I. The death of all global and local characters of Luculla should be enough to at least reduce the initial difference slightly (since I didn't kill the global characters of other levels, a difference, of course, would always remain; but here it didn't change anything!).

Also changing the level and coming back, saving the game didn't reduce the file size. There was no cleanup of dead characters.

Looking into the save files:

Same result as in I: The amount of variable entries didn't change. All characters remained in the save file.

The script of all characters were still ready to receive events and execute actions.

III: 'Base' script without new variables, saved in Luculla, at the level entrance to Cyseal

All Luculla chars alive: 6,33 MB save file size
Local LUC chars destroyed: 6,16 MB (- 0.17)
All LUC chars destroyed: 5,58 MB (- 0.75)
All LUC chars first dead, then destroyed: 5,75 MB (- 0.58)*

* As clearly can seen from a comparison with 'All LUC chars dead' (6,50 MB) and 'All Luculla chars alive' (6,33 MB) from test session I, the difference of 'dead + destroyed' to 'destroyed' in session II comes from the story progress, only triggered by scripted death, not by scripted destruction of the character! The story death of characters obviously adds 0.17 MB to the save file in both cases: 'dead' and 'dead + destroyed'. Without story changes both operations would have produced the same save file size.

As we see, the destruction of all local and global characters of Luculla significantly reduced the save file size.

Looking into the save files:

The amount of variable entries for global characters was reduced (I didn't study this in-depth, but I guess all entries for the destroyed global characters of Luculla were removed). All variable entries for local characters of Luculla were removed. All destroyed characters were removed from the save file (checked for a few characters, they were all gone).

IV: 'Base' script with 40 new global variables, saved in Luculla, at the level entrance to Cyseal; some previous results added to the list for comparison

All chars alive: 6,51 MB
All LUC chars dead: 6,68 MB (+ 0.17)
All LUC chars dead + destroyed: 5,87 MB (- 0.64)*
Global LUC chars destroyed: 5,89 MB (- 0.62)
Local LUC chars destroyed: 6,33 MB (- 0.18)**
All LUC chars destroyed: 5,70 MB (- 0.81)***
Local LUC and local Cyseal**** chars destroyed: 6,14 MB (- 0.37)

* this is 0.12 MB higher than without 40 additional variables; they should come from the global characters of other levels we didn't destroy

**same file size as without 40 additional variables per char!! We destroyed all local characters that contributed the additional 40 variables to the game, so we also destroyed the new variables!

*** the difference to 'dead + destroyed' (0.17) again comes from the story which the death command triggered

****similar save size reduction as local Luculla chars; there are more local chars in CYS (166 vs. 119), but Cyseal chars didn't contribute additional 40 variables per character (which increased the save file only very slightly, probably by around 0.02 MB).

--> We see: the numbers of IV also sum up quite well. The reduction sum of our 'local chars destroyed' (- 0.18) and 'global chars destroyed' (-0.62) matches almost perfectly (just a matter of rounding numbers) the 'All chars destroyed' sum (-0.81). The numbers also seem plausible when we compare them to our other tests (I-III).

Conclusion

It seems indeed possible to remove variables from the game, if we script a 'CharacterDestroy' call. Since this removes the character completely from the game, this is a very dangerous (if not 'deadly') task one should probably avoid for story relevant characters. But maybe a cleanup of local characters (which are never resurrected anyway, as far as I know) could be an option to keep the game and the save files tidy. Especially scripting heavy mods could profit from this in terms of saving/loading times. Though I'd be crazy if I recommended this to you.

Possible complications haven't been investigated yet.

When you set the value to 1 it should remain 1 and be saved as 1 until you change the value with a script call. It seems the engine is able to recognize new variables during the initialization and saving process: variables of characters that are already saved for this character (or already loaded) won't be reset to their initial value; the engine will never read the value from the script again, only when the variable is yet unknown to this character.

Edit:

Thanks, Tinkerer! Always good to have certainty.

@Ameranth: Setting characters on- and off-stage is the last thing to test, I guess. Would be more safe than destruction, I suppose. But will it have the same power?

Some nice research here!

Comments:


This is most likely caused by the Osiris tracking databases that save (in multiple tables) which character GUID-s were killed, etc. They apparently don't clean up the entries when the character is destroyed, hence the difference.


If you think about it, it's quote logical. The game doesn't remove a character's handle after its death, as it has no way of knowing whether scripts will manipulate (revive/etc.) the character after its death. This is also the reason why there is no automatic cleanup/GC, as it would subtly break things if the player happened to be at the wrong place at the wrong time. When a character is destroyed, it is removed from the CharacterManager (and its items from the ItemManager) completely, which also removes the variables, hence the save size reduction.

A possible optimization would be adding a flag to the RootTemplate to indicate that the character can be safely destroyed without consequences, which would allow the engine to clean up the majority of characters. I'm not sure if characters lingering there forever is a problem in practice though, a roughly ~1M save size growth [by killing all characters on all maps] will likely not matter.

Hmm, this seems contrary to what I've experienced. I don't have access to the files at the moment, but I recall that scripts which I had designed to spawn additional creatures on combat start (just once ever), would spawn creatures on combat start every time (after feeling), regardless of a flag variable.



Indeed! Thank you again for the effort Abraxas, it's nice to know that a strategy like this is possible, whether or not it would be safe/would require significant testing before implementation.



Certainly would be a nice addition.


Probably the largest benefit of such cleanup, apart from save file reduction, is that the scripts would no longer need to be loaded in/their events would no longer need to fire. Much of the original scripting of DoS shows how critical this can be; as an example, rats and other critters have scripting that includes a very frequent, permanent, timer. Even if the creature is dead, this timer remains.



Doubtful that it will, though it certainly seems to prevent script events from firing, which is helpful for performance in any case. Also, it was intended to find any bugs that might be appear if a creature needed by a script was "missing."

Note that the behaviour of local variables in events has changed in DOS2: they are now reset to 0/null every time an event is entered.

The reason is that (also in DOS1) local variables used as parameter were assigned whenever an event was called, even if the conditions were not matched. This mainly caused problems in case you had a single handler for multiple events. E.g.



Unlike what you may (and I did) think, in DOS1 this does not work. The reason is that whenever a CharacterEvent gets thrown, _Char will be initialised to the character. Only then the engine will check whether the event name was "Event1" and if so, it will execute the handler. It's similar for ItemEvent "Event2". This means that as soon as you have a character event that's not "Event1" followed by an ItemEvent that is "Event2" (or vice versa), inside the handler _Char/_Item will still have the value of the previous non-matching Character/ItemEvent that got thrown.

To avoid this problem in DOS2, all local event variables that have not been initialised by the event you are currently catching will always be set to 0/null. As a result, assigning initial values to local variables in events is now also forbidden (unless it's 0/null).



Even with global characters?


Hi And you're welcome!

Thanks for all your comments and interest!


Indeed, local characters don't seem to contribute much to the save file; global characters are the majority (a bit concerning thinking of all the events they might receive in modded games and the amount of events Osiris receives from them) which leads me to a question I always ask myself when I add events to character scripts:

What exactly does the engine do when an event occurs, e. g. OnCharacterStatus? Does it look through all scripts of all global characters and all scripts of all local characters (of the current level) and check if there's an OnCharacterStatus event scripted or does it already know which characters to check if this event occurs? And how much of a difference is there in terms of performance if there is 1) no OnCharacterStatus event in any of the scripts, 2) one OnCharacterStatus event in the scripts of every character which doesn't match the conditions (e. g. not the required status), so no actions are performed, 3) one OnCharacterStatus event in the scripts of every character which does match the conditions and performs one action, 4) ten OnCharacterStatus events in the scripts of all characters which don't match the conditions, 5) ten events in the scripts of all characters that do match the conditions...
In short: How much effect on the game's performance do I have to expect whenever I add an additional event to the 'Base' script, which all characters share; is it more efficient to have one OnCharacterStatus event block for several possible status events, checking the status inside the ACTIONS part and executing further actions if it is equal to certain statuses, rather than scripting several OnCharacterStatus event blocks? And how many events can I add until there is a notable effect on the amount of hardware resources required to perform this?

And now the last, short test:

Does setting characters On-/Off-Stage affect the save file size?

There doesn't seem to be a difference. Although I created a whole ghost town in Silverglen, even set all Luculla characters off-stage, there was no save file reduction. The size remained 6,33 MB (without new variables) / 6,51 MB (with new variables). Saving, reloading and saving didn't make a difference either.
And one related note to this: even off-stage characters will still receive events! Global characters from every point of the world, local characters from the same level. They respond like they do on-stage, which raises the question of this command's use: to me it seems there's no other use than keeping them away from world interactions (on the stage), but not from communication.

Edit: Maybe Osiris cares. Has to be investigated, yet.

Preface: my answers are for the DOS2 engine, but I assume it's similar in DOS1.


The former.


I can't give you exact or even ballpark numbers, but what happens in the iteration above, is that every time a completely matching event handler is found, it gets executed. The parameter matching checks themselves seem to be quite light at first sight.


Probably, yes.

Thanks, Tinkerer. Very appreciated! Many details I can only guess.

This is excellent information, thanks for the insight!