Larian Studios Forums Baldur's Gate III Baldur's Gate III - Help Tips & Tricks Information on Karmic dice?

I'm wondering if anyone has any solid information on the Karmic dice system?

Ostensibly, it seems that the intent of the karmic dice system is to define a stochastic process that is essentially the same as dice rolling, but that satisfies the 'law of small numbers' (short sequences of rolls have the same distribution as long sequences of rolls). In other words they want to implement the gambler's fallacy. Anyone familiar with probability theory will likely be thinking of a Markov chain as the appropriate process to achieve this, and ways to implement it are immediately obvious.

One way to simulate from a process with this property is to keep a running dice probability, so that your 20 sided dice for example initially has a uniform distribution, [1/20,1/20,...,1/20], and each time a roll occurs the distribution is updated. It's not hard to choose transition probabilities such that in the long run, you're equally likely to turn up any number as any other, but you're unlikely to see long streaks. One way is to redistribute a portion of the probability of an outcome uniformly among the others whenever it turns up - if the proportion is 1, you never see the same roll twice in a row.

The disadvantage of this method is that, while you're less likely to see the same number come up repeatedly, you are, in the short run, no less likely to see groupings of similar outcomes (perhaps anything over a 15 is a 'good' roll, and you don't want to see a sequence of 5 'good' rolls, such as 16,17,18,19,20). This can be overcome by categorizing rolls into, for example, quartiles, and redistributing weight between categories, rather than individual outcomes. This is very easily achieved, and can be demonstrated to have uniform distribution in the long run. Here is some MATLAB code that does this:


Now, all of that said empirical data collected by akdavidxy on Reddit (https://www.reddit.com/r/BaldursGate3/comments/zwqaem/psa_having_the_karmic_dice_setting_turned_on/) seems to suggest that this is not what they are doing. In fact, it seems that the long run distribution is not even uniform, which in my view would be the most fundamental property of the natural distribution of dice rolls to preserve. Not only is the distribution not uniform, the mean isn't even preserved! Forget preserving the long run distribution of the dice rolls in short sequences, it's not even preserved in long ones!

If this sample is representative of the actual implementation, it would seem that they've missed the mark by a very wide margin. Perhaps then, the intent is not as stated above, but rather to fundamentally change the distribution of outcomes (it's getting a bit unfair to call these dice rolls at this point) in the game. Weighting everything more toward success, and less toward failure. Perhaps the 'law of small numbers' interpretation of their intent is completely off the mark.

Personally, I'm a big fan of dice as a source of randomness in games. They're simple, and all of the information you need to understand how them is readily available. This allows you as a player to make informed decisions about how you play the game. When I saw the karmic dice system, I assumed they were doing something similar enough to what I've described above, based on their description - basically the same as dice, but with the gambler's fallacy built in. Reducing the probability of 'lucky/unlucky streaks' compared to natural dice rolling is not a bad idea, and since it's all happening on a computer (where it's easy to implement) why not? Worth a shot at least, might make an overall more enjoyable experience (although lucky/unlucky streaks themselves can be quite fun, so that's debatable). Turns out it's not doing what it says on the tin? Anyway I would love to hear more from anyone who knows anything about how this system is implemented or what its actual intent is.

Thanks for reading.

I've done a fair amount of analysis on that reddit post and in general the karmic dice system in the past few years. In summary:

In Early Days (e.g., Patch 4) the Karmic Dice System reduced streaking. A roll of 1-10 would be much more likely to be followed by a roll of 11-20. However, it was inconsistent with a normal distribution. Since then, it was updated to reduce the # of low rolls - using an equation that gives results similar to Rolling With Advantage.

They implemented Karmic Dice because their original default rng system was...terrible. It had the opposite problem: low rolls would be preferentially followed by other low rolls, and vice versa. However, analysis of later patches shows that they stealth fixed this problem, and the Non-Karmic Dice System now produces a roughly uniform distribution of effectively random rolls.

Previous thread on this topic: https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=838587
Looks like my conclusions were:
And more detailed conclusions are here:

Thanks for bringing this to my attention.

For a long while now, I've had in the back of my mind the project of doing some analysis of rolls of the Karmic dice. Since I want to learn Python at some point, that will be a perfect mini-project for it.

I haven't looked too closely at it. I am just a bit dubious of the need to translate statements about dice rolls and their probabilities into statements about ACs and probabilities, when dice rolls is the topic of interest. But anyway, it's nice that akdavidxy provided their raw data, i.e. the full sequences of dice roll. That may save me some rolling.

I expect other folks also interested in this will likely make mods to help with the effort. I'm imagining a mod that just sticks the outcome of 100 rolls into your combat status bar as a way to generate data quickly

An asterisk is that Larian has seemingly made stealth updates to their rng systems with certain patches, so the data from the reddit post may be out of date.

And it is possible that they coded Karmic Die so that it takes the enemy AC into account - e.g., Karmic Die tries to ensure that you on average have a 65% to-hit, regardless of your to-hit bonus and the enemy AC. (Although this effect is also achieved by rerolling misses).

I might have---or be able to find---various raw datasets (most of them pre-patch 10) that I've collected from others if you want..? Let me know if so and I'll try to make time to search for them.

That would be dope !


That is very nice of you, thanks. I'll ask you when I get round to doing it. Sleep tight though. I've been thinking of doing that for at least 2 years now, and I still haven't started ...

Interesting stuff. I'll write a detailed response when I get home this evening, but I would be keen to get my hands on the data you mentioned. Interesting that you tested against a normal distribution - were you thinking they might have used a normal distribution to get more average and fewer extreme results?

I have a bit to say about the chi squared test and non independent samples, but I'll write that when I'm in home.

Ah, no, I was just being a bit of a dummy with words. I was using the non-technical definition of normal, as in "normal to what you would expect for [that distribution]." I.e., uniform, Advantage-like, or Disadvantage-like.

I'll see what I can find, raw data wise.

Here's what I got. I don't have pure, roll-by-roll raw data for ~half of the datasets. I didn't know or want to figure out how to statistically determine if a data set showed a trend...just if it was overall consistent with expectations. I calculated the average (and expected standard deviation of the average), along with the chi^squared statistic to compare against the appropriate 95%(?) critical value of 31.44 for 19 degrees of freedom.

Different tabs correspond to different data as labeled. Most of it is pre-patch 6, so extremely outdated.
https://docs.google.com/spreadsheet...7XatKhIeRf_jr4ewe5ug/edit#gid=1759356055

Some of the raw, roll-by-roll data should be present in these threads:
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=838587
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=723265
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=765262
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=766281
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=767538
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=823234
https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=833936

...good luck.

Awesome, thanks for that!

This is why I don't like the naming of the normal distribution. What I wanted to say about the chi squared test is that it only tests whether the frequencies of individual outcomes match those of the theoretical distribution, which says nothing about the order in which those outcomes occur. That's great if the sampling process is drawing independent and identically distributed observations, but not so much if it's a stochastic process (or if its time-series data, if you prefer). When you're considering streaks (which is what Larian claim to be doing with the karmic dice), the order in which observations happen matters.

As an aside, from this stochastic processes perspective, it's interesting to look out for critical values that are 'too good', which may suggest that your sampling process isn't IID (independent and identically distributed). See here for graphs https://imgur.com/a/pjNf9HR

You can see the intuition behind this by imagining a coin flip - you'll get a much better fit to a uniform distribution by first flipping your coin with 2 heads on it, then flipping your coin with 2 tails on it, etc, etc. In fact on any even number of flips your empirical distribution is going to be exactly the distribution you 'expect', and on odd flips it won't be far off. If you were to perform Pearson's chi squared test, the test statistic would be 0 (exactly, for an even number of flips), but you'd be super skeptical about it. You might argue that this sequence isn't even a random variable, but I can flip an actual fair coin to decide which of my two rigged coins to start with, and now it's a genuine random variable. The marginal distribution of heads/tails outcomes is genuinely uniform, but the joint distribution of the nth flip given the outcome of the n-1th is either [1,0] or [0,1], and never [0.5,0.5]. This example is pretty extreme, so you get literal 'suspiciously good' values of the Chi squared statistic, but in more realistic scenarios you'll never realize that the system is rigged in this way from a Chi squared test. You can see from my graph that my rigged dice simulation does better on Chi squared tests than the IID uniform one, but not so much so that you'd notice.

What you want to look at is the autocorrelation, shown in the second plot in my link. Here it's very obvious that my rigged dice aren't IID uniform, and this is what we would LIKE to see from the karmic dice. You can formally test for autocorrelation using the Durbin-Watson test, but at this point it's a little academic. The karmic dice data doesn't even have the right mean, so who cares if it's autocorrelated or not...

Also, be careful with multiple testing. I see in one of those threads you posted "BUT! If we combine your unweighted chart with @Saberem's unweighted data, we get a Chi^2 of 32.26 which is GREATER than the 95% confidence value of 31.14. Thus, we can conclude that Larian's unweighted dice rng is NOT random." - you need to correct for multiple comparisons. If you test enough times, you'll eventually get a test statistic greater than your critical value by chance (unless your sampling method actually IS biased). Applying Bonferroni correction and assuming you performed only two tests, the critical value would be 34.17, and since you got a Chi^2 of 32.26 you would fail to reject the hypothesis that the data followed a uniform distribution.

The non-karmic dice data looks good. If you wanted to scrutinize it I'd be looking for autocorrelation - you can see from my graph that for samples of size 300, autocorrelations stronger than about 0.2 in magnitude would be suspicious. I'd be very surprised if there was anything wrong with the basic dice rolling implementation though. Good random sequences are precisely the ones that look bad to the layman - if the players didn't think it was a bad implementation, that would be a strong indicator that it actually was. That is exactly why Larian have tried to implement the karmic dice, they want to make a version of dice rolling that the players think is fair, rather than one that actually is fair, and those two things are mutually exclusive. It's a shame they seem to have done such a bad job of it...

Edit: I should say, there's no real reason the system wouldn't be 'fair' in the sense of justice, but it wouldn't be 'fair' in the technical sense of 'fair dice'. In some ways it's more fair that low rolls tend to be followed by high ones, and vice versa. Whether it's more fun or not is another question. Fewer people would experience the feeling of good or bad luck. Personally I enjoy that, but I've also spent most of my life studying probability theory so I'm about as biased as a karmic dice...

Nice. I'll leave any autocorrelation tests up to you!
I wouldn't say any of my statistics were "too good," although idk exactly what value would indicate that. The lowest ones were still in the teens. Also, isn't some of what you're talking about above with the coin covered by degrees of freedom? A coin only has a single DoF.

To be clear, I actually combined the raw data for that analysis, which should be equivalent to a single dataset with a larger # of rolls, no?

Actually, the original base RNG system *was* bad. If you look at the 1st plot in the "Niara Data" tab, clearly low rolls were being preferentially followed by other low rolls, forming this sinusoid pattern. No one actually did a statistical test on the level of correlation though. And originally, their Karmic Die system alternated between low and high values, as shown by the 2nd plot.

Again, this has since been changed and their base rng system seems fine, whereas the Karmic Die system seems to have been modified to result in higher chances to-hit, rather than flipping between high and low rolls.

Agreed, it was really just an aside because I love talking about probability, it's of little practical value beyond better understanding the limitations of the tests like chi squared. You're right that the coin has only 1 degree of freedom, but this doesn't cover the phenomena I'm talking about.

Not really. It's equivalent to a larger dataset with more rolls internally, but multiple comparisons correction is about procedure external to the test. The idea is that when you perform the test on data that is genuinely drawn from the distribution you're testing (i.e. the kind you want to not reject), there's a 5% chance that you incorrectly reject the hypothesis that the sampled data follows the theoretical distribution. Multiple comparisons is about recognizing this fact, and adjusting your testing procedure to reduce the chance of this error (Type 1 error). If you keep testing the same, or essentially the same hypothesis with different datasets, eventually you're going to reject it - you expect to incorrectly reject it 5% of the time after all. The Bonferroni correction essentially brings you back to 5% chance of incorrectly rejecting across the set of tests you've performed. In this instance, you're repeatedly testing whether the dice are fair using different datasets, but if the dice really are fair then that test should fail about 1 in 20 times. You can calculate the probability of making a type 1 error for two tests as

P(Type 1) = 2*(0.95)*(0.05) + (0.05)^2 = 0.975

Generally the formula is

P(Type 1) = sum_{i=1:n} n!/[i!(n-i)!] a^i*(1-a)^(n-i)

where n is the number of tests and a is the alpha level of the tests.

Better yet, calculate it as 1 - the probability that you don't make a Type 1 error (the most frequently used trick in all of probability theory, no doubt) to see that

P(Type 1) = 1-(1-a)^n

Anyway, by the time n = 20 you're at a 64% chance of type 1 error (falsely rejecting the hypothesis at least once). So if you've tested 20 different dice rolling data sets, there's a 64% you falsely rejected the null at least once. Bonferroni correction for n tests basically brings that back down to 5% across all your tests by decreasing the nominal alpha level of each test. Get to n = 100 and there's a 99% chance you make the error.


Yeah that is quite striking actually. The lag 1 Pearson's autocorrelation of 0.16, which is high, just outside of the 95% CI suggested by my simulation [-0.1582,0.1582]. DW test p value is 0.0232, but considering that we picked the bad looking dataset of many that have been tested, I wouldn't get too excited about that. What really does stand out is that the lag 2 autocorrelation is significantly larger at 0.22, which is well outside of any confidence interval you might like, even if you've performed 100 tests. Across my 1 million simulations of 200 dice rolls I didn't see a single result even half as extreme as that. I'd say seeing that kind of increasing autocorrelation (with lag) from a sample of a uniform distribution would be a genuinely rare event. I'm still not convinced that the most likely explanation is Larian's RNG being faulty, but there's certainly something going on. Not really much for the data collection side of things, but I might just grit my teeth and collect my own sample from the game over the weekend and do some tests.

The thing is, is that it's not hard to do a good job of simulating samples from a uniform distribution in this day and age. Any standard pseudorandom number generator implemented in essentially any computer language will do a good job - when new ones come around they get tested to hell and back by people who take this kind of thing extremely seriously, so to put a Bayesian spin on things, my prior belief is very much that it should be fine. You've piqued my interest though.


Edit: xckd comic on multiple comparisons: https://xkcd.com/882/

Whoa that’s some intense overview …it just takes the rough edges off …without it there seemed to be an adversely high chance of outlier results i.e rolling four ones in a row …

I thought karmic di ce might make things a little to easy but to me after hundreds of hours in early access I just find them more balanced.

Not scientific but that’s my two cents.

The only thing you need to know about karmic dice is how to turn them off.

I think it was possible Dice was seeded in BG3 and that they fixed this state. That's something that occurs in RPGs with RNG.
That said, I would prefer if Karmic dice was more advantageous to us, and I dislike how enemies apparently hit you harder with Karmic dice turned on.

They do? I haven’t noticed that…mind you I turned on karmic dice due to the huge numbers of miss miss miss even with high hit chances - to be fair it was a while ago there may have been some fiddling during EA

I was looking for any information for karmic dice all over the internet for days now and it looks like there is barely any. It surprises me very much that a core mechanic, and option set to ON by default, has zero description and no mentions in interviews, news, panels or whatever beside ages old patch note where it was introduced. The only bit of info there is is a vague reddit comment from community manager that "there is a bug, it will be fixed". What bug? What is the fix? How karmic dice work in detail? ¯\_(ツ)_/¯

IMHO I would like an option for karmic dice to be enabled for player only and only for open world checks. I do not care much about fights - there are million ways to win any, but in open world checks you are either roll good or you suck. It is my game, there is no GM, I play single-player in my free time - make an option to let me enjoy my story how I want it please without save scum?

How?

It's likely the previous implementation (before the fix mentioned by the community manager) was simply adding/subtracting an escalating value to dice rolls based on the amount of consecutive low/high rolls, by their own internal definition. It's not statistically sound, but it's easy to implement, would comply with the request from the design team, and technically fits the bill of "making the dice less streaky".

Example of what I mean:

1. Roll 6 (low): add +1 to next roll
2. Roll 4+1 = 5 (low): add +2 to next roll
3. Roll 16+2 = 18 (high): add nothing to next roll since low-roll streak was broken.

Could also be some simple variations on this, e.g. reducing bonus' absolute value by 1 after a streak is broken (instead of resetting it to zero), or increasing the bonus by a larger value if the previous roll was super low/super high, etc.

Source: I know how game dev is usually done lol

But it will be possible to turn off karmic dice? I really want my 1d20 to be a regular 20-sided dice and not some funny math construct. What's the idea in showing a dice roll then, if the result is some behind-the-scene construct anyways? And the very idea behind karmic dice as I understand it goes against my idea of playing with dice. Means you cannot really be a lucky bastard or the opposite, as compensations will be made on following rolls. Not everything needs to be fair. It is not chess we are playing. Leave my dice alone please.