BUSI 453 Week 3 Discussion Board Forum 2 Example

Discussion Board Forum 2: Qualitative Research and Causal Research Design

BUSI 453 Applied Marketing Analysis, Discussion Board Forum 2 Thread and Replies

Thread by [Student Name]

The forum prompt and the classmates are composites; the textbook, the research and the Scripture cited are real.

Thread

Qualitative research and experiments are often presented as opposites, one soft and exploratory, the other rigorous and causal. I think they are better understood as two stages of the same project. Malhotra (2015) describes qualitative research as unstructured and exploratory, based on small samples, and designed to provide insight and understanding, while causal research uses experiments to determine cause-and-effect relationships. The first tells a firm what to test; the second tells it what works.

Consider a snack company planning a line of savory flavors for adults. It does not yet know what adult snackers want, so it begins with qualitative work. Griffin and Hauser (1993) compared ways of eliciting customer needs: an hour with one customer surfaced about as many needs as a two-hour group session, and somewhere between 20 and 30 interviews captured the large majority, around nine in ten, of the needs a much bigger study would turn up. Interviews, then, are an efficient first step: the company could talk with two dozen adult snackers, hear what they want from a snack, such as less salt, more protein or portion control, and develop flavor concepts from what they say.

But interviews cannot say which concept will sell, because people are poor predictors of their own behavior. That is the job of an experiment. Anderson and Simester (2003) ran field experiments in a women's clothing catalog, mailing versions that differed only in price endings. Items priced at $39 sold more than the same items priced at $34, a result no survey respondent would have predicted. Only a controlled comparison, with random assignment to versions, could reveal it.

Experiments carry their own risks. Malhotra (2015) explains that field experiments gain external validity, because they happen in real settings, but give up some control, so outside events such as a competitor's promotion can threaten internal validity. The snack company could test two flavors and two package sizes in matched stores, with other stores as a control group, and track sales for eight weeks. Random assignment of stores and a control group protect against history effects, the textbook's term for outside events that occur during the test.

The sequence works because each method covers the other's weakness. Interviews generate ideas worth testing; experiments test which ideas actually change behavior. Skipping the first step risks testing the wrong ideas, carefully; skipping the second risks launching ideas that customers praised in conversation but will not buy. Malhotra (2015) also notes that qualitative findings should not be treated as conclusive or projected to the population, because the samples are small and not representative. A snack company that hears enthusiasm for a smoky flavor from twelve interviewees has a hypothesis, not a forecast, and the store test is where that hypothesis meets real shoppers.

Paul urged believers to be transformed by the renewal of their minds, so that by testing they might discern what is good and acceptable (English Standard Version Bible, 2001/2016, Romans 12:2). Discernment in Scripture involves testing, not merely accepting impressions. Experiments bring the same humility to marketing: an idea that sounds good in an interview still has to be tested before a company commits its resources.

Reply to Brianna

Brianna, your plan to measure the effect of a new display ad campaign with an online A/B test is a strong use of causal design, since random assignment of users to see or not see the ad isolates its effect. I would add one caution about sample size.

Lewis and Rao (2015) analyzed 25 large field experiments on online display advertising and showed that an ad's true effect on any one person's spending is tiny next to the ordinary ups and downs of what people buy, so a test has to be enormous before it can see the effect clearly. Many campaigns, even with millions of users, could not measure their returns precisely. Gordon et al. (2019) compared large randomized experiments at Facebook with the observational methods advertisers often use instead and found that the observational methods frequently misestimated the ad effects, sometimes by a wide margin. Methods that compared exposed and unexposed users after the fact, even with careful statistical matching, could not reliably recover what the randomized tests showed, because users who are shown an ad are systematically unlike users who are not, in ways no data set fully records.

Together the studies support your choice of an experiment over before-and-after comparisons, but suggest planning the sample size in advance. Luke 14:28 asks who would build a tower without first counting the cost (English Standard Version Bible, 2001/2016). Counting the cost here means calculating how many users the test needs before launching it. Do you know roughly how many users your campaign will reach? If the reach is modest, it may be wiser to measure an earlier outcome, such as visits to the product page, which changes more often than purchases and needs a smaller sample to detect. You could then link page visits to sales in a separate analysis.

Reply to Tyler

Tyler, you argued that focus groups are always better than individual interviews because group discussion sparks ideas people would not reach alone. That synergy is one of the advantages Malhotra (2015) lists, and Calder (1977) showed that focus groups can serve different purposes, exploratory, clinical and phenomenological, each with its own value.

The evidence on idea generation is more mixed, though. Fern (1982) compared focus groups with the same number of people interviewed individually and found that individual interviews produced more ideas than focus groups, and ideas of similar quality. Group settings can lead people to defer to dominant members or hold back ideas that differ from the group. Fern also found that the presence of a moderator and whether members knew each other made less difference than one might expect, which suggests the group format itself, not how it is run, limits idea output. Focus groups still serve well when a firm wants to observe how customers talk to each other about a product, but for listing needs or ideas, individual interviews may be more efficient.

Proverbs 18:17 observes that a first speaker's story sounds convincing only until someone else questions it (English Standard Version Bible, 2001/2016). In a focus group, the first confident speaker can shape what follows. A good moderator invites the quiet members to examine that first view. Which of your project's questions would benefit most from group discussion? I would guess the questions about how people talk about the product with friends would suit a group, while the questions about personal needs and frustrations might come out more fully in one-on-one interviews, where no one feels pressure to agree.

References

Anderson, E. T., & Simester, D. I. (2003). Effects of $9 price endings on retail sales: Evidence from field experiments. Quantitative Marketing and Economics, 1(1), 93-110. https://doi.org/10.1023/A:1023581927405

Calder, B. J. (1977). Focus groups and the nature of qualitative marketing research. Journal of Marketing Research, 14(3), 353-364. https://doi.org/10.1177/002224377701400311

English Standard Version Bible. (2016). Crossway. (Original work published 2001)

Fern, E. F. (1982). The use of focus groups for idea generation: The effects of group size, acquaintanceship, and moderator on response quantity and quality. Journal of Marketing Research, 19(1), 1-13. https://doi.org/10.1177/002224378201900101

Gordon, B. R., Zettelmeyer, F., Bhargava, N., & Chapsky, D. (2019). A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook. Marketing Science, 38(2), 193-225. https://doi.org/10.1287/mksc.2018.1135

Griffin, A., & Hauser, J. R. (1993). The voice of the customer. Marketing Science, 12(1), 1-27. https://doi.org/10.1287/mksc.12.1.1

Lewis, R. A., & Rao, J. M. (2015). The unfavorable economics of measuring the returns to advertising. The Quarterly Journal of Economics, 130(4), 1941-1973. https://doi.org/10.1093/qje/qjv023

Malhotra, N. K. (2015). Essentials of marketing research: A hands-on orientation. Pearson.