Probably Not. So why should we expect members of the general population to submit to similar requests when it comes to conducting our own market research?
The science of market research has evolved a great deal over the years, thanks to companies who offer complicated wiz bang computations and others who provide sample sizes larger than the populations of some small countries. Does this mean we’ve reached the pinnacle of market research?
The fact remains that even the most brilliant questionnaire can produce skewed and ambiguous results if the key fundamentals of market research are overlooked. Questionnaire length has a significant impact on the success of any research project and needs to be considered very carefully during the methodology design. All too often, researchers will attempt to seek answers to as many of the client’s issues as possible, consequently producing a survey that lacks focus, clarity and efficiency. The net result is a questionnaire that reduces the study response rate and fatigues participants into providing careless answers.
Chart 1 provides an approximation of the percentage change in response rate relative to a five-minute survey. For example: A 20-minute survey is likely to produce a response rate that is 59% lower than that of a five-minute survey, all else being equal.
You might be thinking, “So what if the response rate goes down? That won’t really impact my project.” Any decline in response rate will negatively impact the accuracy and representation of the population we are sampling.
TAKE THE FOLLOWING EXAMPLE: A classroom of 20 students and their grade distribution. Now consider a random sample pull of ten students.
Avg.=74.5 figure 1
Avg.=74.0 figure 2
The project requires us to study five students and their class average. We could conduct the sampling using two methodologies:
-
Interviewing five of the students from the random pull of 10, hence a 50% response rate.
-
Interviewing five of the students from the total classroom of 20 students, hence a 25% response rate.
We can quickly recognize that the probability of getting a representative group of five students from the full classroom is far less likely than from the random pull. In other words, the response-rate decline significantly reduces the probability of a representative group of students. One might argue to simply bump the sample size from five to 10 for the second methodology, thereby increasing the response rate to 50%. Although that would work for this tiny example, that solution becomes unrealistic when dealing with populations in the millions and sample sizes in the thousands.
This theory can also be applied to questionnaires that inhibit participation. When considering individuals within the population who might be hard to reach or simply don’t like surveys, asking for 20 minutes of their time could significantly reduce the likelihood of compliance. The problem is that these individuals could have very distinct attitudinal differences in your study and excluding them could result in skewed data.
This is not to say that all lengthy surveys will produce inaccurate results. Rather, it emphasizes the need to analyse all parameters of a research project to ensure a balanced, efficient, and intelligent design, aimed at providing the most accurate data, using real-world budgets.