BUSI 331 Week 5 Discussion Board Forum 2 Example

Discussion Board Forum 2: Sampling Design and the Accuracy of Survey Results

BUSI 331 Marketing Research, Discussion Board Forum 2 (Thread and Replies)

Thread by [Student Name]

The prompt and the classmates are composites; the textbook, research and Scripture cited are real.

Thread

A beautifully written questionnaire can still mislead if the wrong people fill it out. Malhotra (2015) explains that probability sampling gives every member of the population a known chance of selection, which allows the researcher to estimate sampling error, while nonprobability methods such as convenience and quota sampling rely on the researcher's judgment or on who is easy to reach.

The difference shows up in accuracy. Yeager et al. (2011) compared surveys fielded the same way except for how respondents were recruited and checked their results against benchmarks known from government records. Probability samples, whether contacted by telephone or over the internet, produced more accurate estimates than nonprobability internet panels, and weighting the nonprobability samples did not consistently close the gap. A large sample of the wrong people is still the wrong people.

Convenience samples raise a related problem. Peterson and Merunka (2014) compared findings from convenience samples of college students with findings from other samples and found that effects often differed in size and sometimes in direction, limiting how far student results could be generalized.

Proverbs gives honor to rulers who investigate matters thoroughly (English Standard Version Bible, 2001/2016, Proverbs 25:2). Careful sampling is part of searching things out honestly rather than settling for whoever is easiest to ask. For a marketer, it also protects the client: a decision built on an unrepresentative sample can cost far more than the research saved by cutting corners.

My credit union survey uses a random sample from the list of app users, a probability design, but the response rate will likely be modest, so the respondents may still differ from nonrespondents. I will compare respondents' age and branch distance with the full list to check, and I will report the limitation in Part 4. If older members are underrepresented, as I expect, the survey may understate the barriers they face, which matters because they are the members least likely to use the deposit feature. A second, shorter mailing to older members could reduce that gap, and a mailed paper version might reach those least comfortable with email or online forms. That would add cost and time, but the group most likely to be missed is also the group the credit union most needs to understand, so the extra effort seems justified.

Reply to Tyler

Tyler, your plan to survey the coffee brand's Instagram followers would be quick and inexpensive. The problem is the sampling frame. Followers are people already engaged enough with the brand to follow it, so their views may differ sharply from those of customers who buy the coffee but never follow it, and even more from people who stopped buying.

Hulland et al. (2018), reviewing survey practice in marketing, stress matching the sampling frame to the population the research question is about and reporting how respondents were obtained. In your case the question is about customers, but the frame is followers, and the two lists may overlap only partly. Groves (2006) adds that bias depends on whether the reasons people respond are related to what is being measured; enthusiastic followers are more likely to answer a survey about the brand.

Proverbs pairs false weights with dishonest measures as things the Lord detests (English Standard Version Bible, 2001/2016, Proverbs 20:10). Reporting followers' opinions as if they were all customers' opinions would tip the scale.

A further thought: could you also survey customers at the point of sale, perhaps with a code printed on receipts, to compare the two groups? Even a small receipt-based sample would show whether followers' opinions match those of ordinary buyers, and the comparison itself would be a useful finding for the brand.

Reply to Leah

Leah, using an online panel to reach 300 respondents for your snack brand study is practical, and you noted the cost advantage well. I would add attention to data quality. Goodman et al. (2013) compared participants from an online labor platform with other samples and found them generally usable but more likely to pay less attention to materials, recommending attention checks. Chandler et al. (2019) found that online panels varied in demographic diversity and data quality, and that researchers could improve quality by screening and by choosing panels carefully. For a snack brand, a panel that underrepresents older shoppers or rural households could miss important buyers, so it is worth checking the panel's demographic profile against the brand's actual customer base before buying responses.

Proverbs contrasts the naive, who accept every claim, with the prudent, who weigh each step (English Standard Version Bible, 2001/2016, Proverbs 14:15). A prudent researcher checks whether respondents were paying attention before trusting their answers.

Would you add one or two attention-check items, and how would you handle respondents who fail them? Deciding the rule before collecting data, rather than after seeing the results, keeps the cleaning honest and easier to defend in your report.

References

Chandler, J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019). Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behavior Research Methods, 51(5), 2022-2038. https://doi.org/10.3758/s13428-019-01273-7

English Standard Version Bible. (2016). Crossway. (Original work published 2001)

Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26(3), 213-224. https://doi.org/10.1002/bdm.1753

Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675. https://doi.org/10.1093/poq/nfl033

Hulland, J., Baumgartner, H., & Smith, K. M. (2018). Marketing survey research best practices: Evidence and recommendations from a review of JAMS articles. Journal of the Academy of Marketing Science, 46(1), 92-108. https://doi.org/10.1007/s11747-017-0532-y

Malhotra, N. K. (2015). Essentials of marketing research: A hands-on orientation. Pearson.

Peterson, R. A., & Merunka, D. R. (2014). Convenience samples of college students and research reproducibility. Journal of Business Research, 67(5), 1035-1041. https://doi.org/10.1016/j.jbusres.2013.08.010

Yeager, D. S., Krosnick, J. A., Chang, L., Javitz, H. S., Levendusky, M. S., Simpser, A., & Wang, R. (2011). Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Public Opinion Quarterly, 75(4), 709-747. https://doi.org/10.1093/poq/nfr020