How to tell if a Pollster was Dishonest
Post #1107
Public opinion polls can be off of the mark for a variety of reasons. There will likely be some sampling error that cannot be controlled for, for instance. Another way to say this is that the sample will not likely be perfectly representative of the population, though such errors creeping into such samples will be random, with a mean of zero.
Another way that polling results could be off of the mark, even far off the mark, would involve dishonesty on the part of the pollster. A pollster could secretly take a biased sample if savvy enough to report on it in a nuanced way which hides that selection bias and makes the sample appear to have been taken randomly.
A third way that polling results could be off the mark, even far off the mark, is if a pollster was getting “set up” for a fall. The easiest way to set up a pollster for an embarrassing and possibly career-ending failure is by way of a computer hack, planting fake survey data there which does not actually reflect reality.
When public opinion polls are taken, they are sized large enough to reduce the margin of error to something meaningful. For example, if only 10 persons are polled and 6 of them support a candidate, then the 95% confidence interval around the proportion who support the candidate runs from 30% up to 90% — a +/- 30% margin of error.
But it is not helpful, or even meaningful, to have a range of estimates that spans from 90% all of the way down to 30% — it does not make you any more knowledgeable about whether that candidate has the support of the public. In the last Iowa poll for the 2024 election, the Des Moines Register published a poll with 44% favoring Trump.
But the election results showed that 56% favored Trump, and given how Trump filed suit against the pollster, it could prove useful to understand how likely it would be to “accidentally” find a sample of people of whom only 44% favor Trump, when sampling from a population of people where 56% do.
The poll sampled 808 persons, and the highest count favoring Trump would be x = 359. If 360 had favored Trump, then it would be untrue to report 44% in favor, like was done, because 360/808 = 44.55% — which rounds up to 45%. Here is the probability of “accidentally” getting 808 people where only 359 (44%) of them favor Trump:
The middle line at bottom shows the chance to get 359 or less in favor of Trump and after the decimal, there are seven zeroes, and then a 1 — i.e., less than 1 chance in 100 million. This probability is so tiny that it is extreme enough to settle paternity-dispute court cases:
The evidence does suggest that some kind of fraud did occur in that final Iowa poll, but as said above, it might be due to a computer hack from an unknown party, instead of being deliberate fraud by the polling company hired by the Des Moines Register. While the numbers do not add up, only a court investigation can ascribe actual guilt.
Reference
[Trump filed a lawsuit] — https://eu.desmoinesregister.com/story/news/politics/2024/12/17/donald-trump-lawsuit-gannett-des-moines-register-iowa-election-poll/77051510007/