After beating Republican challenger Sharron Angle by 5% (50% to 45%) in Nevada, the winner and still Senate Majority Leader Harry Reid complained that the polls leading up to the election were “false and misleading”. He may be right.
In media stories Mr. Reid was described as coming from behind to retain his seat. This claim was based on earlier polls that showed that the two were in a dead heat, and occasionally with Ms. Angle slightly ahead. One such poll was conducted by Rasmussen for Fox News. It showed Ms. Angle with 48% and Mr. Reid with 45% voter support. The survey had a statistical error of ± 3% and so the result was called a dead heat.
This begs the question: How can Ms. Angle be ahead of Mr. Reid +3% on Saturday and three days later on election day be behind -5%? What happened? Did a large number of Tea Party voters suddenly come to their senses and switch their vote to Mr. Reid? Somehow I doubt that. I believe there is a much simpler explanation, one that has nothing to do with the world of politics but a great deal to do with the world of polling.
In this hotly contested election no one in Nevada could escape the propaganda machines from both sides. This was particularly so among Ms. Angle supporters many of whom were highly motivated Tea Party members. In my many years of conducting surveys I have found again and again that highly motivated groups within society with strategic interests are much more likely to participate in a survey that, as they see it, helps achieve their interests compared to the general population. This would explain why Ms. Angle’s support was elevated in the pre-election polls and why on election day it was suddenly AWOL. Nevertheless, Ms. Angle’s final numbers may have been significantly pumped up by these inaccurate polls since they added gravitas to her candidacy..
In survey textbooks this phenomenon is called sample selection bias. Often, bias is not easy to identify and designing remedies for it may be difficult and costly. But remedies to correct bias problems are certainly possible.
Since most journalists regard survey results as objective facts, it is not surprising that there was very little skepticism shown for these Nevada poll results. And anyways, these guys are journalists not methodologists. Perhaps more to the point, the results fed a great story. It was a nose to nose horserace with the possibility of the most powerful Democratic senator being ousted by a Tea Party novice. It was the most compelling Senate race in the country.
Unfortunately it was all based on compromised polling data.
With accurate polls, the numbers on voting day may have been significantly more in Reid’s favor.
Also, with accurate results a very different scenario may have unfolded for the candidates. Ms. Angle may have adopted a more centrist position if she knew with certainty that she was trailing Mr. Reid. Instead she maintained her extremist rhetoric to the end.
The net effect from these competing dynamics may have produced a very different outcome in Nevada on November 2. That we will never know.
Therefore, when Mr. Reid complains that “We’ve got to do something about these misleading polls that are all over the country. They are so unfair, and you [the press] just gobble them up no matter where they come from. You just run with them like they are the finest pastry in the world.” he has a point. What the media doesn’t realize is that relying on polls to create stories is like walking into a minefield. You may know where some of the mines are. But it’s the ones you know nothing about that blow you up.
My advice to journalists is that if you are trying to get through that minefield, get someone who can help you navigate safely through it. Don’t try to do something for which you were not trained.