Selecting and implementing a sampling method is a crucial stage of any online research project. Our guide illustrates the basics of sampling, explains common sources of bias, compares different methodologies, and teaches you how to put these principles into practice when building your own sampling strategy.
Every ten years, the U.S. government conducts a census—a count of every person living in the country—as required by the constitution. It’s a massive undertaking.
The Census Bureau sends a letter or a worker to every U.S. household and tries to gather data that will allow each person to be counted. After the data are gathered, they have to be processed, tabulated and reported. The entire operation takes years of planning and billions of dollars, which begs the question: Is there a better way?
Among public pollsters, the year 1936 lives in infamy, because that year, the magazine Literary Digest conducted what remains one of the worst public opinion polls in history.
After correctly predicting the previous five presidential contests, the Digest decided to mail questionnaires to more than 10 million Americans asking who they planned to vote for in the 1936 presidential race. Based on more than 2 million responses, the Digest confidently predicted that Alf Landon would win the presidency with 62% of the vote. Yet on election day, it was Franklin Roosevelt who won in a landslide, leaving the Digest to wonder how it got the outcome so wrong.
Most businesses can’t survive without conducting some research. What is our market share? Are our customers happy? Who is likely to buy this product? Questions like these are what lead businesses around the world to spend tens of billions of dollars per year on market research.
Regardless of whether you have a significant market research budget or one with very limited resources, it is of paramount importance for your business that your funds are spent efficiently and effectively. How do you do that? The first step might be recognizing when you do and do not need to gather your own data.
Conversations about sampling methods and sampling bias often take place at 60,000 feet. That is, researchers like to talk about the theoretical implications of sampling bias and to point out the potential ways that bias can undermine a study’s conclusions. Although these conversations are important, it is good to occasionally talk about what sampling looks like on the ground. At a practical level, what methods do researchers use to sample people and what are the pros and cons of each?