Originally Posted by mrfuji3

Sure, with enough data and enough iterative rounds of testing, any problem can be identified and fixed. However, Larian does not have unlimited time. They cannot, for example, make small changes to the "evil path", compare the resulting in-game data (how many players chose this route) to that from the previous version, make changes based upon those results, and repeat until convergence. They'll get through, what, 3 iterations using this method?

Feedback guides this iterative process of improvement, speeding it up. Players can say why they do or do not like this "evil path," and Larian can incorporate this feedback to quickly hone in on what needs improvement the most, and in what direction. To produce an equivalent product, using feedback and data collection should result in a fewer number of changes compared to using data collection alone.


That's why I said if you can't have rich data you can go ahead and rake through the muck that is the forums or whatever you feel is necessary and why I am willing to concede the point on those grounds.

So, I think we can agree that if your tests aren't good enough and you don't have enough time, then you can't have rich data and have to rely on secondary sources more heavily in some cases depending on context.

As an aside for things like combat, etc., I don't think player feedback is as important as player behavior.

I think though once you hit the second round of iteration you might be able to construct better tests and scenarios and lean less on external feedback as you understand player behavior better.

I also don't think that Larian will release the entire game so they'll probably lean heavier on feedback in the first few rounds and then less so after constructing better tests/scenarios.