Research / Tait Martin

Always Look at the MOE

Folks, we have to be smarter about political polling… both in how we conduct them and how we read them.

Here’s a short stats lesson to help you better understand polls – ALWAYS LOOK AT THE “MARGIN OF ERROR.” It’s typically written as “MOE +/-“at the bottom of the reported results.

I’m going on a tirade today because I just saw a poll on TV with a +/-8% MOE…this means if the survey was given to a random sample, the answers would all be either plus or minus eight percentage points from the current answer. For example, if 50% of the sample supports a candidate, if the pollsters administered the same survey to another equally sized random sample in the same population, they should expect support between 42% and 58%.

Oh, and I guess I should also mention that a +/-8% MOE means they only surveyed 150 people. This was a major news source reporting the likely results of a Republican Primary outcome as if Moses himself walked down from the mountain with the information chiseled on stone tablets.

Here’s the problem with the whole thing. First, true random samples are tough to get. That’s what the whole “MOE” measure is built on. The standard definition of a random sample is, “Each individual is chosen entirely by chance and each member of the population has an equal chance of being included in the sample.” Most of the polls are not truly random. Just think of how many times you’ve been called to give your opinion, and you chose to hang up. Congrats, you screwed up a random sample. Are you happy with yourself now?

Second, you need to understand the numbers behind the margins of error. If the polls are reporting MOE, here are the sample sizes they used, which are basically the number of people the pollsters spoke to:

  • +/-10% – 96 person random sample
  • +/-9% – 119 person random sample
  • +/-8% – 150 person random sample
  • +/-7% – 196 person random sample
  • +/-6% – 267 person random sample
  • +/-5% – 384 person random sample
  • +/-4% – 600 person random sample
  • +/-3% – 1067 person random sample
  • +/-2% – 2401 person random sample
  • +/-1% – 9604 person random sample
  • 0% MOE – you have to interview the whole population

Big red flag: If they don’t report a margin of error, you should be questioning the validity of the poll.

Most political polls presented in the media fall in the margins of 3% to 5%. Yep, you read that right. All of the hoopla is being based on the opinions of approximately 400 to 1,100 people. And just think. You hung up on them when they were trying to get you to be part of this tiny group.

Come on people. We need to be smart about our use of research, especially when we’re trying to decide who’s possibly going to lead our country.

Educate yourself beyond polling. As a researcher, I think if polls are done well, they can be a great tool. Ultimately, though, nothing should act as a substitute for your own education on a candidate or on an issue.