That poll has 500 respondents.
Over 1,000,000 folks have participated in EA.
It represents the thoughts of less than 0.05 % of the community.
Thank you for saying it ! At last someone has a clear mind here !
That alone probably doesn't mean the results can be dismissed, in fact it is very much the contrary (and I'm speaking as someone who doesn't know what the poll is about).
When doing a poll to get an estimate of the percentage of the population who thinks A (and the percentage that think not-A), you want to sample, uniformly from the population and without replacement, a number of people that is large enough, and you generally want this number to be a small fraction of the population. This is because, if you want to do your poll right and deliver confidence intervals (i.e. do the poll right), the computations are way easier this way. It's not super intuitive, but it has to do with whether the % you are measuring changes as you sample people. So you can certainly criticise the poll :
- For not having a large enough sample. 1 000, 5 000, or even 10 000 participants would be better.
- For having a non-uniform sample. Here, people are self-sampled, among a crowd that is probably different from the all-players crowd, which introduces a bias in the data.
But dismissing a poll, because the sampled population is a small fraction of the general population, is wrong. If you want to measure the % of the population of a country that has long hair, you want to sample a number which is simultaneously large, and small compared to the country's population (which means you want a large population).
