Analysis: Gold-standard polls got it right in 2022. Others did not

Over the New Year’s holiday weekend, the New York Times published a 4,000-word article noting how some political polls, as well as the polling aggregators, overstated Republicans’ strength in key midterm races, feeding the narrative of a coming “Red Wave” that never came to be.

But there’s a far simpler polling story from the 2022 midterms to tell: Gold-standard polls — those that conduct their surveys via cell phone interviews, have longstanding reputations and remain politically neutral — had a strong cycle.

It was the other polls — with more opaque methodologies and more partisan leanings — that didn’t fare as well. (It also didn’t help things when polling aggregators like RealClearPolitics averaged the gold-standard polls with the non-gold-standard ones.) 

Take the competitive Senate race in Wisconsin between Republican incumbent Ron Johnson and Democratic challenger Mandela Barnes, which Johnson won by 1 percentage point, 50.4% to 49.4%. 

The state’s recognized gold-standard poll, the Marquette Law Poll, showed Johnson up 2 points among likely voters the week before the election. Other neutral polling outfits like CNN, Siena College and Fox News had Johnson ahead between 1 and 2 points. 

Compare that to other surveys — including the Democratic-leaning poll Data for Progress and the GOP-leaning poll Trafalgar Group — all showing Johnson with larger leads. 

Or take Pennsylvania’s Senate contest, which Democrat John Fetterman won by nearly 5 percentage points. 

The final polls from Fox, Siena and Marist College, which all use gold-standard cell phone interviews, had Fetterman up between 4-6 points among likely voters. 

By contrast, other polls had Republican Mehmet Oz ahead. 

And the same story played out in Nevada’s super-close Senate race, which Democratic incumbent Catherine Cortez Masto won. 

Siena showed a tied race, while Suffolk had Cortez Masto ahead by 1 point. It was the other polls that had Republican challenger Adam Laxalt ahead. 

Yes, the New York Times’ article is correct that non-gold-standard polls often outnumbered gold-standard polls, especially in the midterm’s final days, due to cost cutting from news organizations, as well as polling misses in 2016, 2018 and 2020. 

For Subscribers

FOR SUBSCRIBERSAbdul El-Sayed tries to woo MAHA voters as Democrats seek inroads with RFK’s supporters

FOR SUBSCRIBERSKratom-like substances are popping up in a typically benign supplement

FOR SUBSCRIBERSGOP super PAC increases its spending for Sen. Jon Husted in tough Ohio race

FOR SUBSCRIBERSRecords reveal tumultuous marriage between slain New York Times executive and wife

FOR SUBSCRIBERSFrom clean energy warrior to ‘wall of shame,’ James Talarico reverses on a fossil fuel money pledge

FOR SUBSCRIBERSCongressional staff planned official trip with luxury stops in Italy and France weeks before election

FOR SUBSCRIBERSA harrowing ordeal aboard an Israel-bound plane

FOR SUBSCRIBERSOpenAI CEO Sam Altman to skip congressional hearing on rogue AI agents

FOR SUBSCRIBERSKey DOJ lawyers departed Trump ‘grand conspiracy’ probe amid pressure to produce, source says

FOR SUBSCRIBERSTrump’s new AI chatbot often links to a site launched under Clinton 26 years ago

FOR SUBSCRIBERSAt MAHA summit, industry CEOs the movement once vilified take the stage

FOR SUBSCRIBERSThis Paralympian has two goals: Win medals and give away $500,000 to her teammates

FOR SUBSCRIBERSCornell rape allegation spotlights stringent New York state laws that have long drawn criticism

FOR SUBSCRIBERSThe debate over AI’s future rages on both coasts

FOR SUBSCRIBERSDecades-old march dispute reignites in Northern Ireland, threatening political crisis

NBC News, for example, released national polls during the 2022 cycle, but declined to sponsor polls in battleground states. 

For Subscribers

FOR SUBSCRIBERSAbdul El-Sayed tries to woo MAHA voters as Democrats seek inroads with RFK’s supporters

FOR SUBSCRIBERSKratom-like substances are popping up in a typically benign supplement

FOR SUBSCRIBERSGOP super PAC increases its spending for Sen. Jon Husted in tough Ohio race

FOR SUBSCRIBERSRecords reveal tumultuous marriage between slain New York Times executive and wife

FOR SUBSCRIBERSFrom clean energy warrior to ‘wall of shame,’ James Talarico reverses on a fossil fuel money pledge

FOR SUBSCRIBERSCongressional staff planned official trip with luxury stops in Italy and France weeks before election

FOR SUBSCRIBERSA harrowing ordeal aboard an Israel-bound plane

FOR SUBSCRIBERSOpenAI CEO Sam Altman to skip congressional hearing on rogue AI agents

FOR SUBSCRIBERSKey DOJ lawyers departed Trump ‘grand conspiracy’ probe amid pressure to produce, source says

FOR SUBSCRIBERSTrump’s new AI chatbot often links to a site launched under Clinton 26 years ago

FOR SUBSCRIBERSAt MAHA summit, industry CEOs the movement once vilified take the stage

FOR SUBSCRIBERSThis Paralympian has two goals: Win medals and give away $500,000 to her teammates

FOR SUBSCRIBERSCornell rape allegation spotlights stringent New York state laws that have long drawn criticism

FOR SUBSCRIBERSThe debate over AI’s future rages on both coasts

FOR SUBSCRIBERSDecades-old march dispute reignites in Northern Ireland, threatening political crisis

But the gold-standard polls were still available for all to see, including at NBC News, where the Political Unit discourages reporting on surveys like Trafalgar and Data for Progress. 

Yet while the gold-standard polls had a strong 2022, there’s an even more important polling lesson from the midterms: Expecting horserace precision from surveys — whether they’re gold standard or not — has become a fool’s errand, given margins of error, lower response rates and America’s intense political polarization. 

Polls can tell us whether political figures are popular or not, whether races are close or not, and whether policies are popular or not — all legitimate reasons for news organizations to conduct and sponsor polls. 

But polls can’t tell you who’s going to win a race where one candidate is ahead by 1-2 points in one poll, but down 2-3 points in another. 

All that tells us is that the race is close.