On the reliability of polls

We all know that the way you frame a question can influence the answer.

There are lots of different ways to do this, e.g., by making a question positive or negative, by emphasizing different aspects of the question, by making it complex or simple, by using charged / loaded language in the question, by the order in which things are presented, etc. And it’s not amateur hour. People study how to do this.

Knowing that pollsters can influence an answer, which is more likely with reference to political polls: (1) that pollsters are seeking an honest read on the situation, or (2) that pollsters are trying to get to a predetermined conclusion?

I think it would be a mistake to assume that it’s always 1 or always 2.

It’s well-established that political campaigns use “push polls” to influence public opinion, and it’s well established that media outlets have political agendas.

At the same time, …

  • news organizations get some level of acclaim when their polls accurately reflect results, and
  • some races (we are told) are particularly difficult to predict with polling.

Taken all together, I’m leaning toward the idea that early polls are more likely to be an attempt to influence the election (e.g., push a preferred candidate), while later polls are more likely to be relatively honest.

When I say “preferred candidate,” I don’t necessarily mean “a candidate the pollster likes.” I mean a candidate the pollster would rather have in the race, for whatever reason.

What do you think?

One thought on “On the reliability of polls”

  1. Those are interesting thoughts. I do think there are some orgs that try to get it right at every stage (not that they do) like fivethirtyeight.com

    OTOH, I don’t even know what the value of polls will be in the future with deep fakes becoming a thing. It is going to be so easy to spread a false story that even a true story will be hard to believe.

Comments are closed.