Putting it all together

by Tom Ehrbar

Part 3 of a series on the survey process

If we want to know what’s on the mind of C-level manufacturing executives in US companies with annual revenues of $1 billion (for instance, and not an unusual request), we’d need to find out how large that population is. By consulting some standard public and private sources (mainly via a proprietary data feed from Hoover’s, which collects global figures on over 85 million companies in more than 900 industries), we can come up with that number. It turns out, the answer is about 1,300:

US companies, by annual revenue and number of employees

image

A lot of clumping among companies with $1 billion–$25 billion in revenue and up to 50,000 employees, so hard to distinguish individual firms there, but that’s Exxon Mobil with 2015 revenue of around $270 billion, and HP with about 287,000 global employees.

Since we now know our population size, we can use our calculating table to determine the number of survey interviews we’d need to do to reach a 95% confidence level within a 3- or 5-percentage-point margin. At a 3-point MoE, we’d need to survey 587 companies, shown in red here against our original population:

95% confidence, with MoE of ±3 points

requires sample size of 587 from a population of 1,300

image

If we’re happy to accept an MoE of 5 percentage points, we’d only need to get responses from 297 companies in this group, as shown below:

95% confidence, with MoE of ±5 points

requires sample size of 297 from a population of 1,300

image

As we can see from these examples, the relationship between margin of error and sample size is straightforward: as the size of the sample increases, the margin of error decreases, which is just another way of saying the more information you have the more accurate your results are going to be. But this relationship tails off rather quickly. After a certain point, adding another few survey respondents (or even 100 or more for a survey that is already reaching 1,500) has a very small effect on the MOE. And as we demonstrated in an earlier post, it’s possible to gauge the opinion of the entire US population within a margin of error of ±3-percentage points by sampling just over 1,000 people. The key, of course, is making sure your sample is both random and representative—a rule that holds for political polling as well as surveys of business leaders.

Tom Ehrbar is Senior Editor with the Thought Leadership group and manages much of its survey research.