The US Election and Reliable Polling

by Angus Reid | November 17, 2020 1:03 pm

By Angus Reid, Chairman

In the wake of the 2020 US election, and the inability of some pollsters to correctly forecast races in key states, some observers are yet again predicting the end[1] of the polling industry. Defeated president Donald Trump is further fanning the flames, claiming pollsters conspired to deflate GOP estimates in order to suppress voter turnout[2] for his party.

Some higher profile misses include popular polling aggregator 538 projected a Biden win in Florida[3] by three points but Trump ended up winning by three points.[4] Pennsylvania, Wisconsin and Michigan were all supposedly safe wins[5] for the Democrats, yet the average margins were less than one percent when these three are combined. Several respected pollsters forecast a close race in Ohio[6], but when the votes were counted Trump trounced by Biden by nearly half a million votes.[7]

But before major pollsters assume a defensive crouch, some of the overblown hype about widespread problems in polling must be corrected. Let’s be clear: there really is no such thing as a regulated polling industry. Anyone can call themselves a pollster and release survey results which are often picked up by news organizations or aggregators such as 538 or RealClearPolitics.

Pick a race in a battleground state this cycle and you’ll find massive discrepancies among polling organizations. In Michigan, which Biden won by about two points, Ipsos had Biden ahead by ten percent[8] while Trafalgar Group had Trump winning by close to three percent. [9]

Faced with this cacophony of often competing forecasts the media and others have turned to organizations such 538 to sift through this mess and publish a poll of polls. Sadly, these aggregators are part of the problem since they mix results from less proven pollsters in with otherwise respected polling companies. They create a mirage of scientific precision and unity when, in truth, aggregators aren’t the ones doing any of the tough work that has become polling today.

Back in the 1980s, most polling was done by telephone[10] using live interviewers.  Accurate random sampling was fairly easy since landline telephone numbers were geolocated to specific communities and neighborhoods. Response rates to telephone surveys were above 70%[11] in a world that telemarketers had yet to invade. And the cost of entry for the aspiring pollster was relatively high (conducting a national poll of say, 1000 respondents was around $20,000).

Fast forward four decades – sampling is no longer easy given the penetration of cell phones. Telephone survey response rates, according to the Washington, D.C.-based Pew Research Center, hover around six percent[12]. At the same time a myriad of new technologies and approaches including Interactive Voice Response (being polled by computer), online surveys of widely varying quality and the use of offshore “live” interviewers means that the same national poll can be conducted for as little as $2000.[13] Anyone can claim to be a pollster for the price of a good set of golf clubs.

The need for reliable, accurate polling is greater than ever. Yet the science of polling is increasingly more complicated and riddled with potential errors. Predicting how people will vote is a greater challenge than ever. Even more problematic is predicting if they will vote. Wild swings in turnout are difficult to predict and can change election outcomes far more dramatically than voters changing who they plan to vote for. Towards the end of the 2020 presidential race the frantic travels and outdoor rallies of Donald Trump likely played a significant role in energizing his base to show up to vote.

What is the future of polling? Four trends warrant discussion and debate:

First, the days of accurate random telephone interviewing are rapidly coming to a close. Part of the problem with US polling is the enduring obsession of US media with live telephone[14] interviews.

Second, online polling involves massive differences in quality and accuracy. Building quality online panels of potential respondents can be a very expensive proposition compared with the relatively low costs associated with buying access to polls of respondents from budget providers.

Third, polling aggregators may create a false illusion of heightened accuracy by mixing quality survey results and poor data in the same blender. Their averages remind me of the lesson learned in introductory statistics – you can walk across a lake on average three feet deep, but you’ll probably drown.

The final point is offered through the perspective of a wide global lens. US Political Scientists Will Jennings and Christopher Wyezien published an exhaustive review[15] of polling accuracy worldwide which examined over 30,000 national polls from 1942 to 2017. Their conclusion bears some attention from those who wish to declare the end of polls:

Although claims about the demise of pre-election polling have become common in recent times, we find little basis to support them. In fact, some of our findings point to just the reverse. Relying on vote intention polls from more than 200 elections in 32 countries over a period of more than 70 years, there is no evidence that poll errors have increased over time, and the performance of polls in very recent elections is no exception.”

Despite these reassuring global, historic trends, a new threat looms in the wake of the 2020 election – the politicization of polling itself. The post-election antics of Donald Trump, who has called into question the honesty of vote counting in America, also includes numerous tweets and statements about inherent bias[16] in all polling.

Many decades ago, George Gallup, one of the founders of modern polling, abstained from voting in national elections[17] so that his polling results could never be perceived as biased by his personal beliefs.

Yet, if Trump’s allegations suggesting polling was a ploy by Democrats to help “steal elections” attract widespread attention from his base, there is a risk that the reputation of unbiased, fact-based polling may be under attack by the very voters whose thoughts and intentions it seek to measure. Talk about a self-fulfilling prophecy.

Polling ultimately depends on the cooperation of strangers. There is a key requirement of an ongoing relationship between pollster and respondent, and that is trust. Respondents must trust that their views when aggregated with those of others will represent a fair reading of public opinion.

I’ve spent more than 40 years working to build and maintain this trust. In the last 20 years, my colleagues and I have developed online communities where members have come to know our work and are nominally compensated for their time.  Our research panels in North America now have almost 200,000 members drawn from every region and demographic category.

Over the last two decades we’ve covered over 100 elections and only missed our predictions in two outings. Over the last 60 days we have covered three elections, using our wholly owned online panels. The results below speak for themselves. We can’t speak for the polling industry, but, overall, we’re very pleased with our results.

 

Endnotes:
  1. predicting the end: https://www.npr.org/2020/11/07/932422888/the-polls-for-this-election-were-off-again-is-this-the-end-of-the-industry
  2. suppress voter turnout: https://nypost.com/2020/11/05/trump-pollster-accuses-media-of-voter-suppression-through-bad-polling/
  3. Biden win in Florida: https://projects.fivethirtyeight.com/2020-election-forecast/florida/
  4. winning by three points.: https://www.politico.com/2020-election/results/florida/
  5. supposedly safe wins: https://projects.economist.com/us-2020-forecast/president
  6. close race in Ohio: https://projects.economist.com/us-2020-forecast/president/ohio
  7. half a million votes.: https://www.washingtonpost.com/elections/election-results/ohio-2020/
  8. by ten percent: https://www.reuters.com/article/us-usa-election-battleground-poll/days-before-u-s-election-bidens-lead-widens-in-rust-belt-reuters-ipsos-poll-idUSKBN27H1PO
  9. three percent. : https://www.thetrafalgargroup.org/news/mi-pres-1120/
  10. done by telephone: https://www.pewresearch.org/fact-tank/2019/02/27/what-our-transition-to-online-polling-means-for-decades-of-phone-survey-trends/
  11. were above 70%: https://academic.oup.com/poq/article/83/S1/280/5520778
  12. around six percent: https://www.pewresearch.org/fact-tank/2019/02/27/response-rates-in-telephone-surveys-have-resumed-their-decline/
  13. little as $2000.: https://www.pewresearch.org/fact-tank/2020/08/05/key-things-to-know-about-election-polling-in-the-united-states/
  14. live telephone: https://www.politico.com/story/2019/02/27/phone-polling-crisis-1191637
  15. an exhaustive review: https://www.nature.com/articles/s41562-018-0315-6
  16. inherent bias: https://www.usnews.com/news/elections/articles/2020-10-28/trump-attacks-suppression-polls-that-show-biden-ahead
  17. national elections: https://www.pbs.org/fmc/interviews/ggallup.htm

Source URL: https://angusreid.org/polling-us-election-results/