Larian Banner
Previous Thread
Next Thread
Print Thread
Page 2 of 4 1 2 3 4
Joined: Oct 2020
A
enthusiast
Offline
enthusiast
A
Joined: Oct 2020
I don't think that reading all the feedback is insurmountable. I wrote a summary of 250 steam reviews https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=704051#Post704051
That took me about an hour and a half. Multiply by 120 = 180 hours. About a weeks work for 3 people to summarize it and 2 more days to compare the top-50 complaints with their ingame analytics.
There should be results by now. I think they should have included at survy a few hours into the game like THQ did for the Gothic playable teaser.


I sometimes use thought experiments. I don't necessarily believe in every idea I post for discussion on this forum
Joined: Oct 2020
S
enthusiast
Offline
enthusiast
S
Joined: Oct 2020
Originally Posted by ArmouredHedgehog
I don't think that reading all the feedback is insurmountable. I wrote a summary of 250 steam reviews https://forums.larian.com/ubbthreads.php?ubb=showflat&Number=704051#Post704051
That took me about an hour and a half. Multiply by 120 = 180 hours. About a weeks work for 3 people to summarize it and 2 more days to compare the top-50 complaints with their ingame analytics.
There should be results by now. I think they should have included at survy a few hours into the game like THQ did for the Gothic playable teaser.


I’ve seen your job in your post and I really appreciate the effort but that’s NOT how market research is done, unfortunately. Statistically speaking you’ve selected a random sample out steam and took the responses that was given spontaneously and in a open ended questionnaire. What you’ve done we can translate to IDI (in depth interviews) but without any structured guideline (which is highly not recommended). That can be used to actually build a questionnaire and run the market research with proper samples and defined stratification of the respondents (age, gender, location etc). Without that everything will have bias and have no statistical meaning.
That’s why I’m saying that Larian should search for a professional approach instead of taking a trainee to gather information on forums.

Last edited by Sludge Khalid; 21/10/20 05:43 PM.
Joined: Oct 2020
A
enthusiast
Offline
enthusiast
A
Joined: Oct 2020
Originally Posted by Sludge Khalid
Statistically speaking you’ve selected a random sample out steam and took the responses that was given spontaneously and in a open ended questionnaire. What you’ve done we can translate to IDI (in depth interviews) but without any structured guideline (which is highly not recommended).

Guidelines would limit the set of responses. If you are out fishing for the most comprehensive list of input no limits should be applied.
You can then take items of interest gathered from such a wide sample and develop a more limited approach and take various customer groups into consideration.

Originally Posted by Sludge Khalid

have no statistical meaning.

That is not entirely true. Items with extremely low frequency are to be considered irrelevant.


I sometimes use thought experiments. I don't necessarily believe in every idea I post for discussion on this forum
Joined: Oct 2020
S
enthusiast
Offline
enthusiast
S
Joined: Oct 2020
Originally Posted by ArmouredHedgehog
Originally Posted by Sludge Khalid
Statistically speaking you’ve selected a random sample out steam and took the responses that was given spontaneously and in a open ended questionnaire. What you’ve done we can translate to IDI (in depth interviews) but without any structured guideline (which is highly not recommended).

Guidelines would limit the set of responses. If you are out fishing for the most comprehensive list of input no limits should be applied.
You can then take items of interest gathered from such a wide sample and develop a more limited approach and take various customer groups into consideration.

Originally Posted by Sludge Khalid

have no statistical meaning.

That is not entirely true. Items with extremely low frequency are to be considered irrelevant.


I’ve said guidelines are highly recommended, not essential.
Again, as someone who work with this for living, your action of grouping and summarizing the trend topics was one of the most productive things I’ve ever seen in this community.

But I’ve to disagree with this statistical point of view of gathering unstructured data and combining because it simply doesn’t work to fish conclusions. Is it excellent to gather assumptions? Yes! Definitely! With assumptions gathered and questionnaire properly structured, then you run an actual survey.

Open ended comments lacks in depth without a guideline because there are several nuances of “people” in the world. Some folks need to be explored by instigating their feelings with structured questions because they are too closed.

Again, my suggestion is that the company use their resources that I’m sure they have due to the amount of sales and get to know about their customer satisfaction level.

Nevertheless thank you for sharing your results.




Joined: Oct 2020
S
enthusiast
Offline
enthusiast
S
Joined: Oct 2020
Quote

They do both, they've shared some metrics they have built into their system, we don't know how deep or how much they have, but if they do a good job here they could have a treasure trove. They also at least in the past had a person gather all the feedback at all forums and they would analyze, sum it up and then present that to the team for consideration. As far as them getting back to us, they have at least once already, a little under a week in. Historically they have been pretty good at this, no they aren't one to act like they are working with you hand and hand with amazeball updates, but I felt they've been pretty good overall. We'll see, perhaps they won't update us, take the money and run and build a place next to Gave in New Zealand
.


Are you 100% sure?

Read this text (don’t need to read all of it. Go directly to the part called “The big issue”

https://www.google.com.br/amp/s/sil...e-flaws-of-divinity-original-sin-ii/amp/

I’ve already said this, the only thing Swen can relate about client feedback is in regard of the weapon durability in DOS2. Is that serious? Is this how you handle your customers?

Last edited by Sludge Khalid; 21/10/20 06:46 PM.
Joined: Sep 2020
journeyman
OP Offline
journeyman
Joined: Sep 2020
with the next update i expect some major bug fixes, fixes to 5e DnD rules, maybe some tweaking with amount of containers to loot or/and reducing some explosive barrels(?), nothing major tho. first major update i hope we will see beginning of december or right after new year, that will be the update that we might see some (among other things) minor recommendations being implemented. new contented additions (able to play original character or new classes, i'm taking bets on the side lol) anybody's guess, but i'd say not earlier than next year. and i as well would love to see Sven giving some PR of how the work is going or pictures of their mansions building in new zealand as someone pointed out smile

on side note i completely forgot the time line for DOS2, i remember that in EA we were able to play Sebile and Fane was never introduced in EA

Joined: Oct 2020
A
enthusiast
Offline
enthusiast
A
Joined: Oct 2020
I agree, some people are in need of some kind of stimulation or the occasional "now that you mention it" type of question. I do not think that the two approaches (with and without guidelines) are mutually exclusive.
Can you point me to any good studies comparing the outcomes of both approaches? I am somewhat obsessed with mathematics and therefore highly interested.

Let's say 20000 people mail in suggestions to the president of x. Two of them mention that their most important concern is that the flowers next to the parking lot in their village should be blue ones. Biased sample or not, the chances of extremely infrequent complaints being actually among the most important ones in the nation are next to 0. Biased or not, noise and signal can be identified.


If I were Larian I would use it as the base for a survey taking more personal circumstances into consideration.
I guess the point of this thread is that people have invested a lot of time into feedback and they are waiting for Larian to acknowledge their awareness of some of the issues mentioned.


I sometimes use thought experiments. I don't necessarily believe in every idea I post for discussion on this forum
Joined: Oct 2020
journeyman
Offline
journeyman
Joined: Oct 2020
I think hoping for certain gameplay changes/features to be included in upcoming updates could be risky. The only thing we know for sure is happening with every patch/update is bug and crashfixing, hoping for certain specific features to be finished by this early on in development could just be setting yourself up for disappointment.
I do hope they'll give us some sort of roadmap in the future, or at least let us know for sure what suggested features will be included in EA vs Full game (or at all), but we're not even a month into EA at this point so that's going to be a looong while.

Joined: Jan 2020
member
Offline
member
Joined: Jan 2020
I think larian is too busy squashing criticizm to actually care about this game, mods just lock forums and call people trolls who critique how poor the game currently is.

Joined: Oct 2020
old hand
Offline
old hand
Joined: Oct 2020
Originally Posted by Emulate
I think larian is too busy squashing criticizm to actually care about this game, mods just lock forums and call people trolls who critique how poor the game currently is.



If youre referring to the troll who just got their thread locked, then you stating justifiable actions they have taken isn't really helping your argument.


What is the problem you are solving? Does your proposed change solve the problem? Is your change feasible? What else will be affected by your change? Will your change impact revenue? Does your change align with the goals and strategies of the organizations (Larian, WotC)?
Joined: Sep 2017
old hand
Offline
old hand
Joined: Sep 2017
If feedback would be 'squashed' then it would be taken out of sight entirely. It's purposefully left up as all feedback is valid, but people's behaviour isn't always acceptable. Thus, moderation is required.

As for feedback review from Larian, it's still barely two weeks in. The priority right now is to fix all the technical issues some players experience so that everyone can play it properly, so that they may actually get genuine reason for constructing feedback of their own.

The amount of feedback already has been crazy (in a good way!!), so it'll take some time to process and sift through. Could even make a few arguments as to why taking some time with it is a positive thing, as feedback should be evaluated and analysed properly, and not just be knee-jerk reacted to at a week's whim. That's paying due respect to feedback, in my opinion.

Tuco #711136 21/10/20 07:24 PM
Joined: Oct 2020
apprentice
Offline
apprentice
Joined: Oct 2020
Originally Posted by Tuco
A common rule experienced developers tend to embrace while dealing with large amounts of feedback is that "your audience is usually very good at identifying problems, but fairly terrible at proposing practical, viable solutions".


We can't provide good solutions for them because we don't fully understand their limitations. That's information they're not going to give us. We can try, but if we give solutions that sound good, but one tiny part of it is impossible to do because of some quirk of the engine, or some rule that a higher up has imposed for some reason. Then it's no good.


Joined: Oct 2020
D
addict
Offline
addict
D
Joined: Oct 2020
Originally Posted by Orbax
Originally Posted by Emulate
I think larian is too busy squashing criticizm to actually care about this game, mods just lock forums and call people trolls who critique how poor the game currently is.



If youre referring to the troll who just got their thread locked, then you stating justifiable actions they have taken isn't really helping your argument.

Considering hes the OP from that thread I think thats exactly what hes referring to.

Good to hear that Larian has crazy amounts of good feedback smile we will keep it coming im sure ^_^

Joined: Oct 2020
S
enthusiast
Offline
enthusiast
S
Joined: Oct 2020
Originally Posted by ArmouredHedgehog
I agree, some people are in need of some kind of stimulation or the occasional "now that you mention it" type of question. I do not think that the two approaches (with and without guidelines) are mutually exclusive.
Can you point me to any good studies comparing the outcomes of both approaches? I am somewhat obsessed with mathematics and therefore highly interested.

Let's say 20000 people mail in suggestions to the president of x. Two of them mention that their most important concern is that the flowers next to the parking lot in their village should be blue ones. Biased sample or not, the chances of extremely infrequent complaints being actually among the most important ones in the nation are next to 0. Biased or not, noise and signal can be identified.


If I were Larian I would use it as the base for a survey taking more personal circumstances into consideration.
I guess the point of this thread is that people have invested a lot of time into feedback and they are waiting for Larian to acknowledge their awareness of some of the issues mentioned.



If I could suggest methodologies based in actual market research and what’s the difference between them I’d say I would go both AdHoc based (which is a Latin word for “to this” - used in cases that you have at least a slight comprehension of your marketing problem)

Now in adhoc you’ve both quali & quanti researches

Quali (in depth interview or IDI) (which is the survey that you’ve done) - is a research made with a small sample size and it’s commonly used for having an overall idea about the marketing problem . The way you conduct it is by using a guideline and the interviewer stimulate the interviewee to answer specific questions. It should feel that you are being interviewed by a tv reporter. Usually the respondents of that interview are specialists or playmakers because they have a broader vision of the problem in general.
You don’t usually want common folk to answer that because they are more narrow minded and as the sample size is small that would only jeopardize the research.

In Quantitative research it’s a different matter. Because the sample size matter and in that kind of research you already have the key buying factors of a product. After structuring the questionnaire you would run the survey and given the attributes considered highly important to the customers you will have a concrete estimation of their weight. USA elections surveys are quantitative for instance.

Now going to your presented case: 2 out of 20000 in sample size complained about something. Is this sample size representative to say that this is not statistically valid? This statistically is insufficient to prove that the matter is not representative. The correct way to approach whether this is relevant or not would be to ask for 2000 of them (because the error margin of that sample is around 2p.p) If they would like to have blue flowers instead.

Sorry for the misspellings. I’m not native and I’m on my cellphone right now


LARIAN run a damn market research!!!!!!!

Joined: Sep 2020
journeyman
OP Offline
journeyman
Joined: Sep 2020
Originally Posted by Sludge Khalid
Quote

They do both, they've shared some metrics they have built into their system, we don't know how deep or how much they have, but if they do a good job here they could have a treasure trove. They also at least in the past had a person gather all the feedback at all forums and they would analyze, sum it up and then present that to the team for consideration. As far as them getting back to us, they have at least once already, a little under a week in. Historically they have been pretty good at this, no they aren't one to act like they are working with you hand and hand with amazeball updates, but I felt they've been pretty good overall. We'll see, perhaps they won't update us, take the money and run and build a place next to Gave in New Zealand
.


Are you 100% sure?

Read this text (don’t need to read all of it. Go directly to the part called “The big issue”

https://www.google.com.br/amp/s/sil...e-flaws-of-divinity-original-sin-ii/amp/

I’ve already said this, the only thing Swen can relate about client feedback is in regard of the weapon durability in DOS2. Is that serious? Is this how you handle your customers?


Larian Studios came forth last year on their forums stating that they would be reading and taking into consideration all criticism and feedback, and would be incorporating it. Come release, and not only were massive complaints of the community, long-standing bugs, and other issues left unaddressed but major, sweeping changes were dropped in the release version with no public testing
this is the paragraph that might be worrisome

Joined: Oct 2020
A
enthusiast
Offline
enthusiast
A
Joined: Oct 2020
Originally Posted by Sludge Khalid

Now going to your presented case: 2 out of 20000 in sample size complained about something. Is this sample size representative to say that this is not statistically valid? This statistically is insufficient to prove that the matter is not representative. The correct way to approach whether this is relevant or not would be to ask for 2000 of them (because the error margin of that sample is around 2p.p) If they would like to have blue flowers instead.


There are thousands of single or dual mention items that you would have to ask people about. Would it be better in terms of the construct validity? Yes Is it feasible? No
You are talking about validity in the context of a test. As the available feedback does not probe for anything in particular I wonder in what way the critereon of test validity is applicable to it at all. It applies when you are measuring something specific.
What we have can provide the basis for probing specific items.
I think we agree that a survey should follow the unstructured interview
Originally Posted by Sludge Khalid

Sorry for the misspellings. I’m not native
No problem, I am not a native english speaker either. In some ways my Latin is better grin


I sometimes use thought experiments. I don't necessarily believe in every idea I post for discussion on this forum
Joined: Oct 2020
S
enthusiast
Offline
enthusiast
S
Joined: Oct 2020
Originally Posted by ArmouredHedgehog
Originally Posted by Sludge Khalid

Now going to your presented case: 2 out of 20000 in sample size complained about something. Is this sample size representative to say that this is not statistically valid? This statistically is insufficient to prove that the matter is not representative. The correct way to approach whether this is relevant or not would be to ask for 2000 of them (because the error margin of that sample is around 2p.p) If they would like to have blue flowers instead.


There are thousands of single or dual mention items that you would have to ask people about. Would it be better in terms of the construct validity? Yes Is it feasible? No
You are talking about validity in the context of a test. As the available feedback does not probe for anything in particular I wonder in what way the critereon of test validity is applicable to it at all. It applies when you are measuring something specific.
What we have can provide the basis for probing specific items.
I think we agree that a survey should follow the unstructured interview
Originally Posted by Sludge Khalid

Sorry for the misspellings. I’m not native
No problem, I am not a native english speaker either. In some ways my Latin is better grin


The things you would have to consider as it’s a qualitative report is whether this person is a specialist or not. That would be like weighing the opinion of someone who have gaming vision in BG3 context or just common folk. As I said, their opinions are statistically non-conclusive. Summing up : its not a yes but it’s not a no as well smile

Getting back to the subject, reading people comments seems unfruitful compared to actual research & both you and me are in agreement in that. As this is currently not being used, I’d suggest engaging in this topic and developing one. Guess googleforms do it for free as a platform. I’ll use my weekend to think about it. I’m not willing to spend my money further on this saga as I’ve already paid for the early access. If I find a free platform that accepts a huge amount of questions I’ll let you know.

Joined: Oct 2020
A
enthusiast
Offline
enthusiast
A
Joined: Oct 2020
Google forms does it. It is what THQ used for their survey.


I sometimes use thought experiments. I don't necessarily believe in every idea I post for discussion on this forum
Joined: Oct 2020
S
enthusiast
Offline
enthusiast
S
Joined: Oct 2020
Originally Posted by ArmouredHedgehog
Google forms does it. It is what THQ used for their survey.


I’m just concerned about the number of questions because some of them limit it, giving a shallow result.

Joined: Oct 2020
A
enthusiast
Offline
enthusiast
A
Joined: Oct 2020
Google forms does not limit the number of questions or respones. Some other platforms like survey monkey do, but their monthly plans with less limits are pretty much free when compared to the budget of the game.
Larian could just set up a virtual server with a database and do it on their own, it's not all that much effort.

Last edited by ArmouredHedgehog; 21/10/20 09:00 PM.

I sometimes use thought experiments. I don't necessarily believe in every idea I post for discussion on this forum
Page 2 of 4 1 2 3 4

Link Copied to Clipboard
Powered by UBB.threads™ PHP Forum Software 7.7.5