Reassurances from pollsters on the accuracy of results are suspect due to an obvious conflict of interest. They’re marketing their product. The press also has a conflict of interest. Media organizations often commission these polls. Can you remember the last time a media organization has questioned the results of a poll it paid for? Even if a news organization has no financial stake in a poll, it usually doesn’t have the technical expertise to independently assess the poll’s accuracy.
Contradictory polls published during the current Ontario election campaign are just the latest example of the problem. With pollsters criticizing each other’s methodology, the press seems helpless in deciding who’s right and who’s wrong.
Need for verification
Contradictory election polls create an obvious trust issue for the public. While some polls may be right, others are definitely wrong. But trust issues also arise when published polls are not contradictory but where the published opinion is at odds with the real opinion held by the public. The public senses the dissonance; the press, confident in the science of polling, has no such doubts. The second situation is much harder to spot and remedy.
Obviously there is a need to somehow verify poll opinions. That’s easier said than done.
Traditionally, the press makes a big deal about independently verifying the accuracy of information in its investigative stories. The same standard of verification is curiously absent in its poll stories.
Why is that? A poll, simply put, is an aggregated conversation on a common topic involving hundreds of individuals. Like any conversation, it’s easy to misinterpret its meaning.
So how does a media outlet verify (as much as it can) that the conversation was accurately reported? Usually it’s done by checking against results posted by other media outlets. (Of course if there is no consensus, as in the results from Ontario, this method is useless.)
If results are similar, all is well. In this game, nobody wants to be the outlier. The problem with this approach is its reliance on herd mentality. The poll questions are not too different, the analyses are not too different, and press conclusions are not too different.
This can hardly be held as a reliable standard of independent confirmation of polling accuracy.
Checking for internal consistency
Perhaps more success can be had by stealing a page from what good detectives have been doing from time immemorial to crack their cases – checking for internal consistency.
Polls usually ask a bunch of questions on a topic of interest. So, for example, in America those opposed to raising the debt ceiling may see the national debt as a bigger problem than an economic downturn due to default. But an economic downturn implies a loss of jobs. If in fact the consistency check shows that lack of jobs is perceived as the bigger problem by most, then finding that many are opposed to raising the debt ceiling suggests the question has been improperly worded or improperly understood by respondents. Any definitive conclusion as to the nature of public opinion on the subject of increasing the debt ceiling needs to be hedged against this uncertainty.
When one applies this criterion of internal consistency to the multitude of stories carried by the press about the debt ceiling and the Obama health care reform debates, one comes to the unavoidable conclusion that the press disseminated an inaccurate description of public opinion as measured by the polls.
Example #1 – Raising the debt ceiling
According to the press, polls showed that America was deeply divided on raising the debt ceiling. Half were opposed and were willing to entertain financial default, while half were in favor in order to prevent a financial calamity. Considering what was reported by other polls, this interpretation simply didn’t make any sense. Whether Republican or Democrat, polls have consistently shown that the top priority for most Americans (much higher than the debt issue) was to increase the availability of jobs. Debt default would have tanked the economy, contributing to substantial job losses.
A closer examination of the questions asked by the polls showed that the key question was badly framed, forcing respondents to be seemingly in conflict with one another. More importantly, the response to other questions asked by the polls revealed that this division was illusory. They showed that a majority of the public wanted the politicians to compromise.
If the press had focused on this message from the start instead of embracing it only at the end of the debate, Republicans would not have been emboldened to hold America hostage to extremist views. An agreement to raise the ceiling may have been accomplished in an orderly fashion well ahead of the August 2 deadline. As it was, the political debacle that ensued influenced S&P in downgrading the US credit rating.
Its confidence shaken, the US stock market lost over 1 trillion dollars in the first week of August. The damage was not restricted to America. Worldwide, market losses were estimated between $2.5 and $4 trillion.
Example #2 – Obama healthcare reforms
In the case of the healthcare reform debate, the press repeatedly cited polls showing that America was deeply divided in their support of the reforms. This conclusion was also highly misleading. A closer examination of other questions asked by the polls showed that by a large majority Americans were in favour of most of President Obama’s reforms. However, the public did express legitimate doubts about the economic viability of the reforms. The Congress and the President were certainly not helpful in assuaging these concerns.
The press, in typically simplistic fashion, focused on a question that showed half the country was opposed to President Obama’s handling of health care reform and half were in in favour of it. In fact, public opposition was not due to the proposed reforms but to President Obama. It was a terrible question. Anything Mr. Obama proposed would have been opposed by alarmed Republican voters, even a health care system based on a Republican blueprint from current Republican presidential candidate Mitt Romney.
Had the press focused on the common concerns of the majority of Americans rather than politically inspired divisions, there’s a good chance politicians would have been pressured by public opinion to behave more constructively. But that didn’t happen. In fact, ideological differences were so inflamed by the bitter debate that Republicans wanted to repeal the legislation after their midterm victory.
Why the press focus on the wrong questions
So why did the press choose to focus on the wrong question in their stories?
For the most part it was self-interest. Stories showing the public is in conflict, and divided into warring camps, produce an exciting narrative. It attracts readers. Showing the public has opinions on which there is broad unanimity that transcend partisan differences is generally not perceived as exciting. When everyone agrees on a subject, where is the conflict? Without conflict, the narrative is one of unanimity. For the press, looking for a narrative that attracts readers, it’s nolo contendere. Conflict trumps unanimity every time.
Consequences of wrong choices
The problem with making the wrong choice, however, is that there are consequences. The public is deceived by the press as to what opinion it holds on important national issues. Hence, instead of being united by opinion showing common purpose, the public is left with a sense of anger and frustration by reports of their illusory divisions. Instead of being united against political ineptitude, the public turns on itself.
For a fractured, dysfunctional Congress that revels in division, there could not be a more convenient outcome. They saw their partisan divisions as a mirror of what was happening across the country. It justified their irresponsible behavior.
Responsibility of the press
For all the damage it has caused, the press seems to have no sense of responsibility for its role in this fraudulent exercise. It continues to see itself as simply a conduit of public opinion as discovered by the polls. If there’s any problem, the failure must lie with the polls. The editorial choices the press makes in how it reports on polls seem immune to criticism even though it seems clear that in its reporting of public opinion the press is actively influencing its essence in a nontrivial manner. Why does this notion not seem to arouse any journalistic curiosity or suspicion.
Accurate reporting of public opinion by the press is a critical responsibility in a democracy.
The public relies on the press to cut through the partisan propaganda and deliver the facts as they are. Between elections, the pressure of public opinion is an important lever in getting elected politicians to behave responsibly. That’s why the press should strive to achieve an accurate reflection of this opinion in their stories. Focusing on ill-conceived poll questions whose primary justification is serving a narrative that sells papers is not the way to go. Had the press shouldered its responsibility for accurately reporting public opinion, the politicians may have been shamed by good example of public sentiment into behaving like adults and not spoiled brats.
As for these wildly different Ontario election polls, we’ll know soon enough where the finger of shame points.
This story was originally posted on the iPolitics site on September 30, 2011
