On Wednesday morning, The New York Times/Siena College will release its first national survey of the 2020 cycle, followed on Thursday by polls of the six states likeliest to decide the presidency: Pennsylvania, Wisconsin, Michigan, Florida, Arizona and North Carolina.
Today, we release one element of the data: the methodology and composition of the sample. This offers a detailed look at how the poll was conducted.
In exchange, we ask one small favor. If you have a problem with how we did the poll, say so now; we will have little sympathy if you criticize the poll only after you find out you don’t like the results.
Don’t read this to mean anything about our results tomorrow. We’ve had the idea for a while of publishing our methodology before releasing the results, and indicated our intention to do so a few weeks ago. The results may be exactly what you expect. Or not.
We’ll also take it as an opportunity to discuss how the poll works and our methodological choices, and why you should trust it despite all that happened four years ago.
What is the Times/Siena poll?
It’s a telephone survey of registered voters, which starts with a data set known as the voter file: a list of everyone registered to vote in a state. This contains rich data on every voter, like age and party registration. We get our voter files from L2, a nonpartisan voter data vendor (L2 has worked with the Trump campaign as well as with progressive organizations like Latino Decisions and HaystaqDNA).
These files also include a telephone number for most, though not all, registered voters. Many voters provide their number on their voter registration forms. In other cases, the telephone numbers have been matched from commercial records, based on a person’s name and address. We draw a sample of these numbers, and our friends at Siena College and at a variety of call centers across the country dial away.
And yes, we call cellphone numbers. Over all, about two-thirds of our interviews are usually completed on cellphones, including nearly all of our interviews among respondents under age 40.
Why should we care about your polls after 2016?
This may be hard to believe, but we felt pretty good about our polling in 2016.
The final Times/Siena polls, in Florida and North Carolina, had the president ahead or tied over the last 10 days of the race. Only one other live-interview pollster (the highly regarded Ann Selzer poll) found Mr. Trump leading in top-tier battleground states over the final stretch. Other surveys showing President Trump ahead appeared to do so because they did not contact voters via cellphone, and as a consequence would have tended to lean toward the G.O.P.
Other high-quality polls also performed solidly in 2016. Most national polls, for instance, fared very well, including the final New York Times/CBS News survey that had Hillary Clinton ahead by three points nationwide, less than a percentage point from her final popular vote margin.
When we analyzed our data after the election, we assessed that we were right for the right reasons, like showing Mr. Trump with a wide lead among white working-class voters, and that there were opportunities to refine our approach. We reached a different conclusion than you might expect: If we could go back, we would have wanted more of our own polling in 2016, not less.
In 2018, we conducted a greater number of political surveys. We had an average error of around three points over nearly 50 polls of House races over the final three weeks, with virtually no bias toward either party. Over the final 10 days, the average error was just over two points. Out of more than 400 pollsters, the Times/Siena poll is one of six to earn an A-plus rating from FiveThirtyEight.
Why do you think these polls are good?
There are three major advantages that we think help explain our track record and offer cause for confidence:
-
Partisanship. Perhaps our most important advantage over even other high-quality pollsters is that we can adjust for the partisan makeup of the electorate, using data available on voter registration files.
For instance, we can make sure we have the right number of registered Republicans or Democrats, the right number of people who voted in the 2020 Democratic primary, the right number of precincts that voted heavily for Mr. Trump. This works best in states with party registration, like Pennsylvania or Florida, and is harder in the states without it, like Wisconsin or Michigan, but even in the worst cases it’s a lot better than nothing.
This is possible only because we start our poll with a voter registration file (not all pollsters do). It doesn’t ensure our results are perfect (and a poll can still be top-notch without this step), but it gives us extra confidence that we’re not fundamentally missing Democrats or Republicans, who may at times become more or less likely to respond to surveys.
And we go even further: We complete not just the right number of interviews with Democrats and Republicans, but also the right number by race and region. So, for instance, we have the right number of Hispanic registered Republicans in Miami-Dade County. To our knowledge, that’s an assurance that no other public pollster can make.
-
Education. Our samples are adjusted to properly represent voters without a college degree, based on census data. Many state pollsters still don’t adjust their samples by education, and this is considered one of the major reasons that state polls overestimated Mrs. Clinton’s standing in 2016.
-
Contacting hard-to-reach groups. We spend a lot of money to complete interviews with groups that all pollsters struggle to reach: low-turnout voters like younger people or Hispanics. These groups are always important, but properly representing low-turnout voters — especially low-turnout registered Democrats, who are often surprisingly favorable to Mr. Trump — was an essential part of why our polls were closer to the mark than most other polls in our postelection analysis of 2016 surveys.
Does anyone pick up the phone anymore?
Some people do. Just not many. We usually complete interviews with about 1 or 2 percent of the voters we try to reach. Low response rates undoubtedly pose a serious challenge to survey research, and there are some known response biases. For instance, people who take telephone surveys are likelier to volunteer in their community than demographically similar individuals who do not take telephone surveys. But for now, it does not appear that the people who take telephone surveys are vastly different politically from those who do not, after accounting for their demographic characteristics.
Why not online?
Online polling is almost certainly the future of polling and, in many ways, it’s also the present. There are some online polls that are fairly or even favorably comparable to many telephone surveys. But as a whole, online polls continue to have a weaker track record in election polling than telephone polls, and real challenges remain.
There’s a simple reason: There’s no way to conduct a random sample of everyone who uses the internet in the same way that you can conduct a random sample of everyone who has a telephone (or, in our case, a sample of the 60-plus-percent of people with telephone records on a voter file). This doesn’t preclude high-quality survey research. It just makes it a lot harder, and there’s no consensus on the best approach.
What about turnout?
The Times/Siena poll is a poll of registered voters at this stage of the cycle — not of likely voters. It’s hard to predict turnout the day before the election, let alone months ahead or in the midst of a pandemic with uncertain effects on access to voting. So for now estimated turnout won’t affect the survey result: We’ll report the result for all registered voters.
That said, turnout is undoubtedly an important question, and we will report findings among likely voters — as if the election were held tomorrow — even if it won’t be the lead result and even if there’s plenty of reason to wonder whether patterns evident today will hold in November.
Where can I learn more?
You can find a detailed methodology and a description of the composition of the electorate here (PDF). You can find our first presidential national poll results tomorrow at 5 a.m. Eastern.
"first" - Google News
June 24, 2020 at 06:33AM
https://ift.tt/2Vczwif
Second-Guessing Our Poll? First, Take a Look at How It Works - The New York Times
"first" - Google News
https://ift.tt/2QqCv4E
https://ift.tt/3bWWEYd
Bagikan Berita Ini
0 Response to "Second-Guessing Our Poll? First, Take a Look at How It Works - The New York Times"
Post a Comment