[Feedback] The Bless spell and Larian's use of player data for the EA development - 14/02/21 08:01 PM
Larian gave this interview, which explained that data show that people are uninterested in buffs, that "you cannot sell a Bless spell to people, it's boring", and that people want to see fireworks and damage.
Passage copied below for those forum readers who don't want to read the whole interview (though you probably should). Colouring is mine.
I feel that just arguing about Bless would too narrow, like the small tip of a big iceberg, and I would to be a bit more general. I'm not going to go maximum generality and discuss Larian's vision and what they think is fun. I have my own tastes and vision of what a 5E-based combat system should be, you have yours, Larian has theirs. That's just tastes and they can't really be discussed or argued. (Well, actually, I've just come up with things that can be discussed, but I'll keep them for elsewhere and not derail this thread in the very first post). What I want to discuss though is Larian's use of data and their development process.
1) Why are buffs so little used ?
Larian's data shows that players don't use buffs, and that includes Bless in particular. Why is Bless not more used ? Let's try to see what explanatory factors there could be.
So Larian's current implementation butchers Bless, partly and directly through a bad implementation of the spell, and partly and indirectly through homebrew rules, which significantly affect the balance of value between the various options available on a Cleric's turn, and which go in the sense of making Bless less appealing. This list is not exhaustive though. I could think of adding a couple more.
Now, out of all the possible factors above, which ones are likely to have the greater explanatory power ? I certainly wouldn't pick the last one as the obvious one. Very far from it.
I would have no problem if Larian just spelled out their vision and tastes : "we like fireworks and damage, we think that Buffs/Debuffs (and the Fighters and Rogues, and basically the Attack action, etc) are boring, so we'll make sure we have lots of fireworks and damage in BG3". But here they don't do that. They advance the thesis that players want firework and damage, and use the buff data as supporting evidence. The thesis may or may not be true, but the current data are not good enough to support it. This is bad use of data. There's nothing wrong with having an agenda/vision/goal, and there's nothing irredeemably wrong about incorrect reasonings, but I'm always uncomfortable when I feel there's a chance that the incorrect reasoning which supports a goal wasn't incorrect completely by accident.
Before I move to more about the use of data, I just wanted to consider the following question : are buffs/debuffs any popular in 5E ? According to data from DnD Beyond, for Clerics, Shield Of Faith is the 4th most memorised spell and Bless is 9th. For Paladin, these spells are 2nd and 3rd, while Divine Favour is 4th and Heroism is 10th. For Druids, Entangle is 3rd and Faerie Fire is 7th. I'm not too familiar with DnD Beyond, and I don't know the profiles of the players with an account there, but I would assume it's a mixed crowd. The data also doesn't how often the spells are actually cast. But by and large, I would say it doesn't sound as if buffs are under-selected in 5E. And if they are not, there seems to be a marked difference between the balance/meta/ecosystem in 5E and in BG3. Is that because the TT players and video game players are significantly different in their wants ? Or is that because the BG3 makes buffs less attractive ? I think it is a somewhat interesting question.
2) More generally, what can the data say, as of now ?
I am somewhat worried by the way Larian seems to use their data. In good part, this connects with another thing that has been puzzling me, which how primitive a build they have let us play with and how massively-early this access is.
Given how far from a vaguely-complete game system the game is at the moment (I'm just talking about the systems and mechanisms, the story content doesn't matter), most of the feedback that Larian can gather, both from what players explicitly say and the player data, is probably of limited value. But I'll focus on just data.
Player data show how players behave. This behaviour depends on everything they experience in the game : all the dialogues, all the rules, and ultimately all the parameters one could think of. It also depends on things outside the game.
As a consequence :
a) Seeing as many parameters, rules, etc will be significantly changed by the time the game reaches the near-completion, fine-tuning phase, whatever behaviour patterns are observed now are quite susceptible to evolve.
b) Seeing as many elements in the game (notably combat-and-adventuring rules) are inter-related in complex ways, it is not always trivial to find the most contributing cause to an observed behaviour.
c) The fact that this is an Early Access has an effect on both the type of players who are currently playing and how they are playing. This may introduce a bias in the data.
A couple of examples.
- If NPC dialogue lines are skipped at a high rate, it might be because the voice delivery is too slow compared to how fast people can read text. Or it might be that most EA players have already heard some lines half a dozen times already.
- If people are massively casting spells, it may be because they look so flashily good. Or because they are over-powered. Or because the absence of any restriction on long rest makes spells massively available.
I'm certainly not saying that nothing can be learned at the moment, but I'm surely saying that one must be cautious with how the data is analysed and interpreted.
Everyone is collecting copious amounts of data these days. But it's not the size of the data that matter, it's what you do with it. If you have a kitchen filled with all the utensils you could think of and the finest ingredients, and you place someone who has never cooked in that kitchen, you shouldn't expect amazing dishes to come out.
Larian surely uses the data they collect in many ways, but from what they have communicated, what do we know ?
I want to believe that Larian has smart people working for them. And that as a result Larian's development process is well-informed and efficient. But judging by what Larian communicates, I don't stand super impressed.
3) Development process.
It sounds to me as if, should one want to change an observed behaviour, one should have a good understanding of its main causes, and turn the relevant dials, instead of pulling levers at random.
For the sake of example, suppose that players die at an above-desired rate against the 3 Intellect Devourers on the crash site. There could be several causes. Maybe they have too much HP. Maybe players have at most one companion at that point but there'll be a second one available in the full version. Maybe it's because the game currently does a bad job at teaching its mechanisms, like the fact that, unless you Hide, enemies will see you and trigger the fight even if you don't enter their vision cone (which mean that new players may have a harder time starting the fight from the safety of the harder-to-access high ground). Decreasing their HP isn't necessarily the obvious way to go (nor is it necessarily the worse way to go, it depends on many things).
Less theoretically, if the data show that people are re-loading way too often on some conversation skill checks, it allows to spot a behaviour which is undesired by the designers (I guess) and the players (I believe). But is it because the DC are too high ? Is it because there is a contradiction in proposing high-Charisma companions like Wyll but requiring the players to talk using their PC if they want to gain Approval ? Is it because they tied the Approval to the outcome of the roll, instead of the choice of the player ? Is it because they locked some story beats behind skills checks when, perhaps, it shouldn't be so ? Turning down the easiest-to-access dial (the DC) isn't the obvious way to go.
Larian didn't say that they flip switches voluntarily-at-random. But starting to balance things, any thing, at this early stage of the game (personality of the evil companions, DC of conversation skill checks), without a complete system (all companions, rules for resting, improved conversation mechanisms, etc) and a good understanding of what causes an observed behaviour, sounds close enough to flipping switches at random.
Also, I'm a bit puzzled by some of the design goals.
Sometimes, when there are several options, they should probably all be equally appealing. Other times though, you can't expect all options to be equally used. DnD has over 300 spells, including about 50 level 1 spells. They are designed in such a way that many are situational. You might use a given spell once or twice in a campaign, but the times you used it were situation-changing and memorable. There's nothing wrong with that. DnD seems to give players problems and give them (especially Wizards) a box of tools : up to them to figure out what to use when. Larian seems to want every spell to be the popular one who's used all the time.
Besides, suppose Larian manages to make all spells be the most popular, and thus used about as often. If they want to have each spell used 5% of the time ... that means there can be only 20 spells. If they implement 100 level 1-and-2 spells and we want to click them all all the time, that means they're all used 1% of the time.
So this whole goal of wanting to make us click on every spell sounds quite dubious to me.
Anyway, in short, this interview really didn't make Larian sound good at using EA data and developing the game, in my view. Of course, with enough iterations, they can perfectly converge to a great-to-play version of the game.
Passage copied below for those forum readers who don't want to read the whole interview (though you probably should). Colouring is mine.
Originally Posted by Wireframe, Dec 2020
I feel that just arguing about Bless would too narrow, like the small tip of a big iceberg, and I would to be a bit more general. I'm not going to go maximum generality and discuss Larian's vision and what they think is fun. I have my own tastes and vision of what a 5E-based combat system should be, you have yours, Larian has theirs. That's just tastes and they can't really be discussed or argued. (Well, actually, I've just come up with things that can be discussed, but I'll keep them for elsewhere and not derail this thread in the very first post). What I want to discuss though is Larian's use of data and their development process.
1) Why are buffs so little used ?
Larian's data shows that players don't use buffs, and that includes Bless in particular. Why is Bless not more used ? Let's try to see what explanatory factors there could be.
- Bless is nerfed by bad implementation. In 5E, Bless allows you to choose up to X targets within a ball centered on the caster. In BG3 you choose an AOE and the creatures closest to its center receive the buff. This includes neutrals (and enemies ?), your allies don't have priority. That means in BG3 you cannot cast Bless on your companions who are already in the thick of the melee or tactically positioned around unsuspecting soon-to-be corpses.
- Bless is made impractical to cast in highly 3D terrains. The AOE is a 2D disc, not a 3D ball, so if you start having your party at multiple levels of altitude, it becomes a lot more difficult to obtain what you want from Bless.
- Bless gives an average bonus of +2.5 to attacks roll (and ST) and costs a spell slot. Advantage typically means a +3 to +5 bonus to attack rolls, and the current Backstab and High-Ground rules pretty much hand out Advantage each turn, without costing a spell slot. This decreases the value of Bless.
- Bless requires Concentration. In BG3's rules (as of Patch 2 at least, but bear in mind that the data mentioned in the interview was probably from October-November), elemental arrows are a double source of damage (the arrow + the extra). Which means that when you're hit, you have two chances of failing a Constitution Saving Throw (effectively having Disadvantage). This is made worse by the fact that the elemental damage is guaranteed, even if the arrow doesn't hit. Also, Joe Goblin is routinely equipped with fire grenades, i.e. mini-fireballs, which are guaranteed to deal damage and force the caster to do a ST. Not to mention the fact that a fire surface will deal damage at the beginning of the caster's next turn. Overall, this means Concentration is harder to maintain and it devalues all Concentration spells, like Bless.
So Larian's current implementation butchers Bless, partly and directly through a bad implementation of the spell, and partly and indirectly through homebrew rules, which significantly affect the balance of value between the various options available on a Cleric's turn, and which go in the sense of making Bless less appealing. This list is not exhaustive though. I could think of adding a couple more.
- The fact that long rest is handed out for free, anywhere, anytime, means that there is less incentive to be efficient in combat. Winning with 50% HP left or 80% HP left is all the same when you are 4 clicks away from full life.
- The under-developed combat log isn't good enough to make new players (of the log-reading type) pick up all the rules and see how powerful Bless can be.
- There are people who nowadays spend a significant part of their time in BG3 purely play-testing. I don't have a normal player's behaviour when I just want to go to location X to try new dialogue options, and will happily abuse Backstab and High-Ground in those fights that I cannot avoid, which I consider "solved" and which present no interest to me.
- There are people who stream their gaming. They want viewers to come back. Moving images (including flashy animation) are probably better at retaining viewers' attention than, well, everything else you can put in a video. I assume that streamer ... stream, which means live shows, which means you can't do jump cuts when editing. So they'll favour fireworks.
- Bless is not flashy and players just want fireworks and damage.
Now, out of all the possible factors above, which ones are likely to have the greater explanatory power ? I certainly wouldn't pick the last one as the obvious one. Very far from it.
I would have no problem if Larian just spelled out their vision and tastes : "we like fireworks and damage, we think that Buffs/Debuffs (and the Fighters and Rogues, and basically the Attack action, etc) are boring, so we'll make sure we have lots of fireworks and damage in BG3". But here they don't do that. They advance the thesis that players want firework and damage, and use the buff data as supporting evidence. The thesis may or may not be true, but the current data are not good enough to support it. This is bad use of data. There's nothing wrong with having an agenda/vision/goal, and there's nothing irredeemably wrong about incorrect reasonings, but I'm always uncomfortable when I feel there's a chance that the incorrect reasoning which supports a goal wasn't incorrect completely by accident.
Before I move to more about the use of data, I just wanted to consider the following question : are buffs/debuffs any popular in 5E ? According to data from DnD Beyond, for Clerics, Shield Of Faith is the 4th most memorised spell and Bless is 9th. For Paladin, these spells are 2nd and 3rd, while Divine Favour is 4th and Heroism is 10th. For Druids, Entangle is 3rd and Faerie Fire is 7th. I'm not too familiar with DnD Beyond, and I don't know the profiles of the players with an account there, but I would assume it's a mixed crowd. The data also doesn't how often the spells are actually cast. But by and large, I would say it doesn't sound as if buffs are under-selected in 5E. And if they are not, there seems to be a marked difference between the balance/meta/ecosystem in 5E and in BG3. Is that because the TT players and video game players are significantly different in their wants ? Or is that because the BG3 makes buffs less attractive ? I think it is a somewhat interesting question.
2) More generally, what can the data say, as of now ?
I am somewhat worried by the way Larian seems to use their data. In good part, this connects with another thing that has been puzzling me, which how primitive a build they have let us play with and how massively-early this access is.
Given how far from a vaguely-complete game system the game is at the moment (I'm just talking about the systems and mechanisms, the story content doesn't matter), most of the feedback that Larian can gather, both from what players explicitly say and the player data, is probably of limited value. But I'll focus on just data.
Player data show how players behave. This behaviour depends on everything they experience in the game : all the dialogues, all the rules, and ultimately all the parameters one could think of. It also depends on things outside the game.
As a consequence :
a) Seeing as many parameters, rules, etc will be significantly changed by the time the game reaches the near-completion, fine-tuning phase, whatever behaviour patterns are observed now are quite susceptible to evolve.
b) Seeing as many elements in the game (notably combat-and-adventuring rules) are inter-related in complex ways, it is not always trivial to find the most contributing cause to an observed behaviour.
c) The fact that this is an Early Access has an effect on both the type of players who are currently playing and how they are playing. This may introduce a bias in the data.
A couple of examples.
- If NPC dialogue lines are skipped at a high rate, it might be because the voice delivery is too slow compared to how fast people can read text. Or it might be that most EA players have already heard some lines half a dozen times already.
- If people are massively casting spells, it may be because they look so flashily good. Or because they are over-powered. Or because the absence of any restriction on long rest makes spells massively available.
I'm certainly not saying that nothing can be learned at the moment, but I'm surely saying that one must be cautious with how the data is analysed and interpreted.
Everyone is collecting copious amounts of data these days. But it's not the size of the data that matter, it's what you do with it. If you have a kitchen filled with all the utensils you could think of and the finest ingredients, and you place someone who has never cooked in that kitchen, you shouldn't expect amazing dishes to come out.
Larian surely uses the data they collect in many ways, but from what they have communicated, what do we know ?
- They told us before EA that they have the tools to create heat maps of where people die, save, re-load, etc. Which is all nice and good ... depending on how that information is used.
- They told us how un-tastefully common our choice of a Human Fighter was. This was done in jest, I found it funny. (As an aside, it's completely consistent with various data from DnD Beyond.) It also implies that they look at how frequently various options are selected. Totally fine.
- They told us that 25% of players sided with Minthara. Which means, again, that they're looking at how often (story) options are selected. Maybe 25% was a bit lower than they expected, I don't know. They're probably very correct in saying that most players will lean toward good more than evil. It's also probably a factor that not all players realised that siding with Minthara was an option, and that those who realised found that path to be little motivated and poorly handled. Also, once you've watched the sex scene with Minthara once, you can spare yourself the Battle In The Grove in subsequent runs, whichever side you're on, as it is painfully slow due to the very slow AI. It's probably hard to predict how that number will be when siding with Minthara gets fuller (... I mean, I've always wanted to assume that this content wasn't really ready and finished).
- They told us that players don't use buffs ... I've commented on that already.
I want to believe that Larian has smart people working for them. And that as a result Larian's development process is well-informed and efficient. But judging by what Larian communicates, I don't stand super impressed.
3) Development process.
It sounds to me as if, should one want to change an observed behaviour, one should have a good understanding of its main causes, and turn the relevant dials, instead of pulling levers at random.
For the sake of example, suppose that players die at an above-desired rate against the 3 Intellect Devourers on the crash site. There could be several causes. Maybe they have too much HP. Maybe players have at most one companion at that point but there'll be a second one available in the full version. Maybe it's because the game currently does a bad job at teaching its mechanisms, like the fact that, unless you Hide, enemies will see you and trigger the fight even if you don't enter their vision cone (which mean that new players may have a harder time starting the fight from the safety of the harder-to-access high ground). Decreasing their HP isn't necessarily the obvious way to go (nor is it necessarily the worse way to go, it depends on many things).
Less theoretically, if the data show that people are re-loading way too often on some conversation skill checks, it allows to spot a behaviour which is undesired by the designers (I guess) and the players (I believe). But is it because the DC are too high ? Is it because there is a contradiction in proposing high-Charisma companions like Wyll but requiring the players to talk using their PC if they want to gain Approval ? Is it because they tied the Approval to the outcome of the roll, instead of the choice of the player ? Is it because they locked some story beats behind skills checks when, perhaps, it shouldn't be so ? Turning down the easiest-to-access dial (the DC) isn't the obvious way to go.
Larian didn't say that they flip switches voluntarily-at-random. But starting to balance things, any thing, at this early stage of the game (personality of the evil companions, DC of conversation skill checks), without a complete system (all companions, rules for resting, improved conversation mechanisms, etc) and a good understanding of what causes an observed behaviour, sounds close enough to flipping switches at random.
Also, I'm a bit puzzled by some of the design goals.
Sometimes, when there are several options, they should probably all be equally appealing. Other times though, you can't expect all options to be equally used. DnD has over 300 spells, including about 50 level 1 spells. They are designed in such a way that many are situational. You might use a given spell once or twice in a campaign, but the times you used it were situation-changing and memorable. There's nothing wrong with that. DnD seems to give players problems and give them (especially Wizards) a box of tools : up to them to figure out what to use when. Larian seems to want every spell to be the popular one who's used all the time.
Besides, suppose Larian manages to make all spells be the most popular, and thus used about as often. If they want to have each spell used 5% of the time ... that means there can be only 20 spells. If they implement 100 level 1-and-2 spells and we want to click them all all the time, that means they're all used 1% of the time.
So this whole goal of wanting to make us click on every spell sounds quite dubious to me.
Anyway, in short, this interview really didn't make Larian sound good at using EA data and developing the game, in my view. Of course, with enough iterations, they can perfectly converge to a great-to-play version of the game.