Originally Posted by Limz
Sure, just keep in mind that the data points I provided are what Larian has revealed to us they could have more and while your criticisms are valid keep in mind that the more data you have from player behavior the more accurate the aggregate profiles become. In each example you give if you have more data you can build a countervailing argument; in an area with high death you can reasonably ascertain that a percentage of the players died because of stupidity due to the actions they took leading up to the encounter starting from the beginning of the game.

And to address 'Most people taking the "good" path doesn't show them how to fix the "evil" path.' is not necessarily true because the game offers multiple scenarios/tests that allow for an 'evil' outcome and it's simply just mapping those, discarding which points were least touched upon and improving it then sending it out for another round of testing. You can pretty much create accurate profiles of what an evil player wants with enough data points by simply running them through as many scenarios as possible or if you already have enough data. Again, most social media platforms kind of do this already without ever receiving direct feedback from you in the form of a forum post or a reddit post etc.

But barring that then sure, I am willing to say that if you can't have rich data then go ahead and rake through all the forum stuff.

Sure, with enough data and enough iterative rounds of testing, any problem can be identified and fixed. However, Larian's limit is time. They cannot, for example, make small changes to the "evil path", compare the resulting in-game data (how many players chose this route) to that from the previous version, make changes based upon those results, and repeat until convergence. They'll get through, what, 3 iterations using this method?

Feedback guides this iterative process of improvement, speeding it up. Players can say why they do or do not like this "evil path," and Larian can incorporate this feedback to quickly hone in on what needs improvement the most, and in what direction. To produce an equivalent product, using feedback and data collection should result in a fewer number of changes compared to using data collection alone.