A dismaying proportion of the news coming at us from the mainstream media firehose is based on polls. It’s dismaying to me because in my past lives in business and politics, I have been a consumer of the polls themselves, not just the news based on them, and thus have first hand knowledge of how good they are. Which is not very. And they are getting worse fast.
Some days it seems as if every other story is about a poll: what politician is ahead, and who is behind; what foreign or fiscal or immigration or trade policy does the public like, or not; who supports a cause, or opposes it. And it’s usually presented as just a poll, as in “A new poll out today says that fewer than half of West Virginians believe the 2020 presidential election was legitimate.” Few broadcast stories and not all printed stories tell us anything about how the poll was conducted or by whom.
It’s as if someone said, “I have a car for sale for $10,000. You want it?” And then refused to answer any questions about the car’s make, age or condition. Of course no one would buy the car in that situation, and no one should buy any poll until they know the answers to several questions about it. Just a couple of answers pretty much invalidate the West Virginia poll, and we’ll get to that in a minute.
Back in the day, meaning more than 20 years ago, polls were pretty good at doing one thing — taking a somewhat hazy but recognizable snapshot of what a specific group of people in a specific place thought about a specific thing at a specific time. They were useful as long as everybody remembered that they were not predictive. They began to rot the instant they were announced, and changing conditions could easily invalidate the whole thing in a matter of days.
They were pretty good back then because they relied on the telephone, which back then meant a hard-wired landline. The pollster had a list of numbers and knew when a number was dialed, who owned it, where they lived, and that they were in their residence when they answered. None of that applies today because fewer than half of US households even have a landline any more (and those households are older and poorer than the general population), there are few directories of cell phone numbers, and when you call a cell phone you don’t know where it is when answered.
Another relatively new difficulty for today’s pollster is the refusal rate. Back in the day most people responded cheerfully to a pollster’s request for an interview. Today most people are annoyed by it. According to Pew Research, on average only nine of a hundred people contacted agree to participate in a poll. That makes it very difficult — time-consuming and thus expensive — to put together a significant number of respondents.
The first question to be asked about any poll whose results are being touted concerns the sample size. And any poll using a sample size of fewer than a thousand people should be disregarded. The West Virginia poll, for example, had a sample size of 400. And it was conducted by a firm whose business, according to its website, is market research, not polling. They are not the same thing.
In addition to the size of the sample, we need to know how it was constructed. For valid results, participants must be both carefully screened to make sure they belong to the group being researched — and then randomly selected to avoid bias. We need to know that they were actually interviewed, because various kinds of opt-in and automatic polls conducted for example by websites or via robocall, have been totally invalidated. One recent poll I saw that claimed to have measured an unusually high level of support for Donald Trump for president in 2024 turned out to have been based on an algorithm that counted references made in tweets. Useless.
The news media have become uncritical purveyors of whatever nonsense pollsters provide them with. It’s lazy journalism that leads to fake news and we cannot afford to accept it. And we need to think through the whole concept of recording the opinions of underinformed, uninterested people on complex matters, then using those opinions as a basis for policy.
Try flying an airliner by polling the passengers on what the pilot should do next and I guarantee you are going to crash.
We are among the dwindling numbers of U.S. households who still have a landline (technically, it’s not a landline anymore – we call it our house phone, because we had the number transferred to our mobile provider). Over 95 percent of the calls we get on our house phone are robocalls, telemarketing calls, or scams. Recently, a caller stating that she worked for a polling firm called – she could/would not tell me who hired the firm she worked for – said the interview would last for 15 minutes – and proceeded to ask for my opinion about local and state candidates from the 2020 elections – most of whom I had no recollection of. I hung up, because I didn’t have the time, and am automatically suspicious of any caller with an agenda. The never-ending avalanche of spam calls has poisoned the environment for polls – and one never knows if the information will be used against you.
I miss your voice. As a kid I had a wonderful 2nd grade teacher that read stories to the class. It was the highlight of my week. Her voice would bring the stories to life.
Your voice adds deeper meaning , it adds context and richness to the ideas you are conveying.
I miss hearing it.
Thanks for the wonderful compliment. Computer’s on the fritz, haven’t decided what to do about it. The voice should be back soon.
Another facet is that, Internet culture being what it is, polls conducted online tend to be the subject of group attack, ranging from kitschy fun (Boaty McBoatface is the prime example) to entirely invalidating (voters lying about their support of various candidates or issues to avoid recrimination). Election results (polls of a sort) are also not trustworthy, considering how many shenanigans are pulled prior to any votes even being cast. All evidence of an ongoing epistemological collapse that is deranging everything one might consider normal or sane.
Yes, except I’m not sure what you mean by “shenanigans.” Not buying into the Big Lie, are we?
No big lie, just routine gerrymandering, voter suppression, and refusal of the two parties to allow any meaningful third party to develop.
[“And we need to think through the whole concept of recording the opinions of underinformed, uninterested people on complex matters, then using those opinions as a basis for policy.”]
The West’s very polished Potemkin democracy in most iterations today makes use of voting as a mere, [very] crude check, in a supposedly “reasoned” and “moral” logical circuit series of checks&balances.
Enduring legacy of the Old Money-Bourgeois-Industrialists alliance.
The Machine has had increasingly special requirements though, as time went by.
I still believe the only way to get better deliberations up top is by having a certain set of fundamental truths emerge out of the torrents of information this late stage of civilization offers.
It may come; reality will inevitably break the remaining shards clean off that Overton window frame…but relief will be dampened by a strange feeling: a mix of raw, post-orgasm sedation and diffuse anxious febrility. The sensation of having been carried away by impulses seemingly due to a ‘hack’ of our ‘free will’ of some sort; combined with a classic bitter flavor of “Too soon old — Too late smart” that ‘ll probably leave a bad aftertaste…
Recent polling shows that 57% of Republican voters are in favor of Medicare-for-All!!!
Is this figure accurate? I say, hell no. As a person who spent 40 years working alongside Republican voters, I can tell you the true figure would be much, much higher. In fact, never once did I get f*cked up with a Republican voter who didn’t admit on the down low that it would be a goddamn Godsend for them and their family if they only had an affordable healthcare plane, no matter where it came from.
Because they never did. Like literally, NEVER.
Obviously, it was a different story if they where sober and in a group. Then the voices of the likes of Rush and Jerry and Tucker and Sean would fill the air thru their suddenly robotic conduits and it would become impossible to talk about any political issue if it didn’t involve hatred of the other.
But on this issue you don’t have to take my word for it. Just think back on the ulra-reactionary right wing Tea Party. At every rally they ever held, signs and placards could be seen everywhere in favor of socialized medicine. Remember? Big huge signs that read: Hey Big Government, Get Your Hands Off My Medicare!
Forget the paradox. As far as accurate polling is concerned it would be a simple framing issue. Frame it properly, and I think the only Americans that would not be in favor of Medicare-for-All at this point, would be the usual suspects; our elected representatives, corporate execs and their drones, some very well-to-do wine moms, the boys with the billion dollar yachts, and so on. You know the type. Roughly, the 2%.
Yup, framing is the key to polling. Long term trend polling that is. Polling events that are happening on the fly, like who is going win the mayoral race in my hometown of Buffalo for instance,* ain’t worth what they used to be, for all the reasons outlined in Tom’s fine piece.
*Polls here are indicating that America will not elect it’s first Socialist mayor since the early 70s. Here’s to hoping their wrong.