MRKT 600 Week 8 Quantitative Research Prospectus Example

Stars, Comments and Age: A Quantitative Research Prospectus for a National Survey Experiment on How Adults Choose a Primary Care Physician

Abstract

Prospective patients now see a physician's star average and a set of written comments side by side, and the two often point in different directions. This prospectus proposes a quantitative study of how U.S. adults who have lately found a new primary care doctor weigh those two kinds of information. A national online survey experiment with 450 respondents, recruited by age quota through a research panel, will ask each respondent to pick between a 4.8-star physician whose recent comments describe the same problem and a 4.4-star physician whose comments raise no complaints, randomly varying whether the complaints concern office service or clinical care. Paired comparisons, chi-square tests and logistic regression will test whether comments are trusted more than averages, whether choices differ by age and whether clinical complaints weigh more than service complaints. A course pilot with 62 respondents supported the first two expectations. The results should help health systems decide how to present and respond to reviews.

Introduction

Background and Problem

Online reviews of physicians began as a niche feature of a few independent websites and now appear on booking platforms, search results and the directories health systems publish for their own doctors. A patient looking for a new primary care physician will usually meet two kinds of review information at once: a numeric average, shown as stars, and a list of comments in which earlier patients describe their visits. Most of the time the two agree. Sometimes they do not, as when a physician with an excellent average has several recent comments describing the same problem.

How patients resolve that disagreement is a marketing question with practical stakes. If patients rely mainly on averages, practices will compete to raise them, through requests for reviews from satisfied patients. If patients rely mainly on comments, the content of the most recent and most specific comments will matter more than the average, and practices will need to fix what the comments describe. The literature shows that both kinds of information influence choice, but it does not show how patients weigh them against each other in primary care or whether the weighting differs across groups of patients.

Purpose of the Study

This quantitative study sets out to measure how U.S. adults who began care with a new primary care physician during the previous two years weigh star averages against written comments, whether that weighting differs by age, and whether the kind of complaint in the comments, about office service or about clinical care, changes the choice.

Research Questions and Hypotheses

Research question 1: Do adults trust written comments more than star averages as a guide to the care a physician provides? H1: Mean trust in written comments will exceed mean trust in star averages.

Research question 2: If the better-rated of two physicians carries specific complaints in its comments and the other carries none, does the choice differ by age group? H2: The proportion choosing the lower-rated physician without complaints will be smaller among adults 55 and older than among younger adults.

Research question 3: Does the type of complaint affect that choice? H3: The proportion choosing the lower-rated physician will be greater when the complaints concern clinical care than when they concern office service.

Significance

For health systems, the study will show whether review management should focus on volume and averages or on answering and fixing specific complaints, and whether the answer differs for older patients. For researchers, it addresses a gap the literature review identified, the direct comparison of averages and comments in primary care among U.S. adults, with an experimental manipulation that observational data cannot provide.

Literature Review

This section condenses the Module/Week 2 review around the question the proposed study will answer.

What Moves Consumers in Online Reviews

Research on electronic word of mouth treats reviews as a form of advice from strangers whose influence depends on who writes, what is written and who reads it (Cheung & Thadani, 2012). People who post reviews do so for reasons that include concern for other consumers and a wish for social connection, so reviewers are a self-selected group (Hennig-Thurau et al., 2004). In retail settings, a study of book sales on two websites showed that changes in reviews moved sales and that negative reviews had the larger effect (Chevalier & Mayzlin, 2006). Review depth also made reviews more helpful, and for experience goods moderate ratings were judged more helpful than extreme ones (Mudambi & Schuff, 2010). Physician care is an experience good at best, so these retail findings give reason to expect that patients read comments closely and attend to negative detail.

Physician Reviews and Their Readers

Ratings of U.S. physicians multiplied during the late 2000s and were mostly favorable (Gao et al., 2012), yet many physicians had only a handful of reviews, a pattern found in Boston early on (Lagu et al., 2010) and on commercial sites several years later (Lagu et al., 2017). A national U.S. survey found that many adults considered ratings important and that a share of them had picked or ruled out a particular doctor on the strength of ratings (Hanauer et al., 2014). German surveys found that users of rating sites were more often younger, female, more educated and living with a chronic condition (Terlutter et al., 2014), and they identified characteristics associated with use in a large online panel (Emmert et al., 2013). Comments themselves describe manner, competence, time and practice organization (Emmert et al., 2014). Age therefore appears both as a predictor of using reviews and as a plausible influence on how they are read, which motivates H2.

Evidence on Choice

On a large U.S. booking platform, topics extracted from review text, such as waiting time and bedside manner, improved models of which physician patients chose (Xu et al., 2021). Experiments with hypothetical physicians reach similar conclusions. When a high overall rating was held constant, negative comments lowered the intention to choose a physician, factual negative claims did more damage than evaluative ones and a physician's reply only partly repaired it (Han et al., 2024). Other experiments varied whether reviews were factual or emotional and how many there were (Grabner-Kräuter & Waiguny, 2015) and examined how valence interacted with the riskiness of the condition and trust in the platform (Han et al., 2019). The last study is the closest precedent for H3, since clinical complaints imply more risk than service complaints.

What Reviews Measure

Whether reviews tell patients anything about clinical quality is doubtful at the level of the individual doctor. Ratings of specialists on five platforms showed no significant link to performance measures or peer review (Daskivich et al., 2018), and the connection between reviews and outcomes in chronic disease care was weaker and more conditional than readers might expect (Saifee et al., 2020). Hospital reviews, which are more numerous, track the federal patient experience survey more closely (Bardach et al., 2013) and raise topics the survey omits (Ranard et al., 2016). Physicians are more skeptical of rating sites than patients are (Holliday et al., 2017). These findings matter for interpreting the proposed study: if patients weigh comments heavily, they may be reading about service more than about medicine, which has ethical implications for how practices present reviews.

The Gap

The studies show that averages and comments both influence choice, but none of them measures directly how U.S. adults choosing primary care trade one against the other, whether age changes that trade, or whether clinical complaints carry more weight than service complaints. The proposed study addresses all three.

Procedures and Methods

Research Design

The study uses a cross-sectional survey with an embedded between-subjects experiment. Creswell (2014) describes survey designs as suited to describing trends and relationships in a population and experiments as suited to testing whether a manipulated condition affects an outcome; the combined design lets the study do both. Every respondent answers the two trust items, which allows the within-person comparison for H1, and every respondent makes one choice between two physician profiles. Respondents are randomly assigned to one of two versions of the choice: in the service version, the three negative comments describe referrals that were never sent; in the clinical version, they describe a physician who dismissed symptoms that later proved serious. All other features of the profiles are identical. Moskowitz and Gofman (2007) argue that varying features systematically and letting choices reveal their weight is more informative than asking customers what matters, and the manipulation applies that logic to the one feature the pilot could not test.

Population and Sample

The target population is U.S. adults 18 and older who switched to or started with a new primary care physician within the last two years. Because no sampling frame lists this population, the study will recruit through an established online research panel, using quota sampling by age group, with 150 completed responses in each of three groups: 18 to 34, 35 to 54 and 55 and older. Alreck and Settle (2004) explain that quota samples control the makeup of the sample on selected characteristics but remain nonprobability samples, so the study will report results as estimates for panel members who meet the criteria and will compare the sample's education and region with census figures to show how far it departs from the population. Equal age quotas correct the pilot's main weakness, a small older group, and give each group enough respondents for comparison.

Sample Size and Power

The sample size was set by power analysis using G*Power (Faul et al., 2007), setting power at .80 and the two-tailed alpha at .05. For H2, a chi-square test with two degrees of freedom requires 241 respondents to detect a small-to-medium effect of w = 0.20. For H3, the two-by-two comparison of choice by complaint version, at one degree of freedom, needs 349 respondents to detect w = 0.15. For H1, a paired t test requires 199 respondents to detect a small effect of dz = 0.20, by the conventions of Cohen (1988). The planned 450 completed responses exceed each requirement and allow for removing up to about 20 percent of responses for failed attention checks while still meeting the largest requirement. Appendix B summarizes the calculations.

Instrument

The instrument, in Appendix A, revises the fourteen-item pilot survey. It opens with consent and two screening items, the second confirming that the respondent looked at any online information while choosing. The choice task follows, then a confidence item, the two trust items on a seven-point scale, three items on how the respondent actually chose and demographics. Two changes come from the pilot. First, the choice stays ahead of the trust items, as in the pilot, so that abstract trust questions cannot color it, and each trust item gains a second wording so that a two-item index can be formed. Second, one attention check and one manipulation check have been added: the manipulation check asks respondents what the negative comments were about. Alreck and Settle (2004) recommend that sensitive items come late and offer a way to decline, so the chronic condition item appears near the end with a "prefer not to say" option.

Variables

The dependent variables are trust in star averages and trust in written comments, each a seven-point rating, and choice, coded 1 for the lower-rated physician without complaints and 0 for the higher-rated physician. The independent variables are age group, a three-level category, and complaint type, a two-level manipulated factor. Control variables are gender, education, chronic condition, whether a personal recommendation was used and the number of comments read.

Data Collection

The panel provider will send invitations to members who match the screening criteria until each age quota is filled. The survey will run on a standard online survey platform, which will randomize the complaint version and record completion time. Responses completed in less than one third of the median time, or failing the attention check, will be removed before analysis, a rule stated in advance so that exclusions cannot be chosen after seeing the results.

Analysis Plan

H1 will be tested with a paired t test comparing the two trust items, with Cohen's dz as the effect size. For H2, a chi-square test will compare choice across the three age groups, reporting Cramér's V, followed by pairwise comparisons with a Bonferroni correction. For H3, a second chi-square test will compare choice between the service and clinical versions. Because age and complaint type may interact, a logistic regression will then predict choice from age group, complaint type, their interaction and the control variables, reporting odds ratios with 95 percent confidence intervals. Creswell (2014) advises reporting the results for each hypothesis in turn and stating effect sizes and confidence intervals alongside significance tests, and the analysis will follow that order.

Validity and Reliability

The trust items are single items, which limits reliability, so the larger study adds a second wording of each, allowing a two-item index whose correlation will be reported. The random assignment of complaint type protects the internal validity of H3, while H1 and H2 rest on observed associations and cannot show cause. External validity is limited by the panel sample and the hypothetical choice, limits the paper reports rather than hides.

Anticipated Ethical Issues

No data will be gathered until the protocol clears Liberty University's Institutional Review Board, and the researcher has completed the CITI human subjects training required in Module/Week 2. The guide excludes special populations and invasive or stressful procedures, and the design avoids both: participants are adults, the task is a brief hypothetical choice and no medical information beyond one optional item is requested.

Four issues require specific safeguards. The first is informed consent. The opening screen will explain the purpose, the expected time of about six minutes, the panel's compensation, the voluntary nature of participation and the right to stop at any point; continuing will indicate consent. The second is privacy. The survey will not collect names, contact details or the names of physicians, and the panel provider will supply only anonymous identifiers, so responses cannot be linked to individuals. The third is the sensitive health item. Asking about a long-term condition touches health information, so the item is optional, worded generally and placed near the end. The fourth is the hypothetical physicians. The profiles are fictional, and the clinical complaint describes a common type of complaint without naming any real physician or practice; the closing screen will state that the physicians were invented for the study.

A further issue concerns the use of the findings. If the study shows that patients weigh comments about service heavily, a practice might be tempted to bury unfavorable comments rather than address them. Proverbs teaches that hiding wrongdoing does not lead to success, while admitting it and turning from it brings mercy (English Standard Version Bible, 2001/2016, Proverbs 28:13). The report of the findings will recommend responding to complaints and correcting their causes, and it will not advise suppressing reviews.

Preliminary Pilot Findings

The research program has already produced two sets of preliminary findings. In Module/Week 4, two adults were interviewed about choosing a primary care physician. Both used star averages as a first screen; one let the comments decide, and the other relied on a neighbor's recommendation and read reviews only to look for problems. Both gave weight to complaints that were specific and repeated, and remarks about the office, such as unanswered phones and lost referrals, counted for as much as remarks about the doctor.

In Module/Week 7, a fourteen-item survey built from those themes was completed by a convenience sample of 62 adults. Respondents trusted comments more than averages, with mean ratings of 5.35 and 4.45 on the seven-point scale, and the paired test gave t(61) = 3.91 and p < .001, a medium effect (dz = 0.50). In the choice task, 41 of the 62 respondents, or 66.1 percent, chose the lower-rated physician without complaints. Choice was related to age, χ²(2, N = 62) = 11.24, p = .004, V = 0.43: about three in four respondents under 55 chose the physician without complaints, compared with 3 of the 12 respondents 55 and older.

The pilot shaped the larger design in four ways. First, its older group fell short of its quota, and one expected cell count was below 5, so the larger study uses equal age quotas. Second, all pilot complaints concerned referrals, a service problem, which left open whether clinical complaints would weigh more; H3 and the manipulation address that question. Third, the pilot measured each kind of trust with a single item, which leaves its reliability unknown, so the larger study adds a second wording of each. Fourth, the pilot effect sizes, which were medium to large, suggest the power analysis's small-effect assumptions are conservative.

Expected Outcomes of the Larger Study

If the pilot pattern holds, the larger study should find that trust in comments exceeds trust in averages, though the difference may be smaller in a national panel than in a local convenience sample. The age difference in choice may also be smaller, because the pilot's older group was small and recruited largely through a church bulletin, which may have reached readers less familiar with booking platforms. The study expects the clinical version of the complaint to increase choice of the physician without complaints, since experimental work has examined how disease risk shapes the effect of review valence on patient choice (Han et al., 2019), and a complaint about missed symptoms implies far more risk than a complaint about a lost referral.

Three outcomes would each be informative. Support for all three hypotheses would suggest that comments drive choice for most patients, that clinical complaints are especially damaging and that older patients rely more on averages. An interaction in the logistic regression, for instance if older respondents responded to clinical complaints but not to service complaints, would suggest that age changes what kind of comment matters rather than whether comments matter. A failure to replicate the age difference would suggest that the pilot result reflected its sample, which is itself a useful lesson about convenience samples. In each case, results will describe panel members who meet the criteria and every estimate will carry its effect size and a confidence interval.

Recommendations

For Future Research

Three directions follow from the design's limits. First, a field study could test whether the choice pattern appears in actual bookings, for instance by working with a health system to vary the order in which comments appear in its directory and observing which physicians receive appointment requests. Second, because the study manipulates only complaint type, a fuller conjoint design could vary the average, the number of reviews, the age of the comments and the presence of a physician's reply together, as Moskowitz and Gofman (2007) recommend for learning how features combine. Third, a qualitative follow-up with older respondents who chose the higher-rated physician could explain why, whether through greater trust in numbers, less time spent reading or a different idea of what a comment reveals.

For Marketing Management

For health systems and practices, the expected findings point to several practices. Comments about service, such as referrals and phone access, should be routed to operations as well as to marketing, since fixing the underlying problem changes future comments. Clinical complaints call for a careful, privacy-respecting reply and an internal quality review. Directories should present comments prominently rather than averages alone, since patients appear to read them closely, and should explain what reviews do and do not measure, given the weak link between ratings and clinical quality (Daskivich et al., 2018). Outreach to older patients may need channels beyond reviews, such as referrals from trusted clinicians.

Conclusion

The prospectus turns a question raised by the literature, how patients weigh averages against comments, into three hypotheses that a national survey experiment can test. The design builds on interviews and a pilot survey, fixes the pilot's weaknesses with age quotas, randomization and a power analysis, and plans each test in advance. Its safeguards protect participants and the honesty of the findings. Whatever the results, they should help health care marketers present reviews truthfully and respond to the problems that reviews reveal.

Appendix A: Survey Instrument

• 1. Consent screen and agreement to continue
• 2. Have you chosen a new primary care physician within the last two years? (Yes / No)
• 3. Did you look at any online information about physicians while choosing? (Yes / No)
• 4. Choice task, randomly assigned version (service complaints or clinical complaints): Dr. A, 4.8 stars from 120 reviews with three recent specific negative comments, or Dr. B, 4.4 stars from 95 reviews with no negative comments
• 5. Confidence in the choice (1 to 7)
• 6. Manipulation check: what were the negative comments about? (Office service / Medical care / Not sure)
• 7. Trust in star averages, two wordings (1 to 7)
• 8. Trust in written comments, two wordings (1 to 7)
• 9. Attention check item
• 10. Number of comments read, use of a personal recommendation and the factor that mattered most
• 11. Age group, gender, education, region and chronic condition (optional)
• 12. Closing screen stating that the physicians were fictional

Appendix B: Power Analysis

Appendix C: Timeline

References

Alreck, P. L., & Settle, R. B. (2004). The survey research handbook (3rd ed.). McGraw-Hill/Irwin.

Bardach, N. S., Asteria-Peñaloza, R., Boscardin, W. J., & Dudley, R. A. (2013). The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Quality & Safety, 22(3), 194-202. https://doi.org/10.1136/bmjqs-2012-001360

Cheung, C. M. K., & Thadani, D. R. (2012). The impact of electronic word-of-mouth communication: A literature analysis and integrative model. Decision Support Systems, 54(1), 461-470. https://doi.org/10.1016/j.dss.2012.06.008

Chevalier, J. A., & Mayzlin, D. (2006). The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research, 43(3), 345-354. https://doi.org/10.1509/jmkr.43.3.345

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE Publications.

Daskivich, T. J., Houman, J., Fuller, G., Black, J. T., Kim, H. L., & Spiegel, B. (2018). Online physician ratings fail to predict actual performance on measures of quality, value, and peer review. Journal of the American Medical Informatics Association, 25(4), 401-407. https://doi.org/10.1093/jamia/ocx083

Emmert, M., Meier, F., Heider, A.-K., Dürr, C., & Sander, U. (2014). What do patients say about their physicians? An analysis of 3000 narrative comments posted on a German physician rating website. Health Policy, 118(1), 66-73. https://doi.org/10.1016/j.healthpol.2014.04.015

Emmert, M., Meier, F., Pisch, F., & Sander, U. (2013). Physician choice making and characteristics associated with using physician-rating websites: Cross-sectional study. Journal of Medical Internet Research, 15(8), Article e187. https://doi.org/10.2196/jmir.2702

English Standard Version Bible. (2016). Crossway. (Original work published 2001)

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175-191. https://doi.org/10.3758/BF03193146

Gao, G. G., McCullough, J. S., Agarwal, R., & Jha, A. K. (2012). A changing landscape of physician quality reporting: Analysis of patients' online ratings of their physicians over a 5-year period. Journal of Medical Internet Research, 14(1), Article e38. https://doi.org/10.2196/jmir.2003

Grabner-Kräuter, S., & Waiguny, M. K. J. (2015). Insights into the impact of online physician reviews on patients' decision making: Randomized experiment. Journal of Medical Internet Research, 17(4), Article e93. https://doi.org/10.2196/jmir.3991

Han, X., Lin, Y., Han, W., Liao, K., & Mei, K. (2024). Effect of negative online reviews and physician responses on health consumers' choice: Experimental study. Journal of Medical Internet Research, 26, Article e46713. https://doi.org/10.2196/46713

Han, X., Qu, J., & Zhang, T. (2019). Exploring the impact of review valence, disease risk, and trust on patient choice based on online physician reviews. Telematics and Informatics, 45, Article 101276. https://doi.org/10.1016/j.tele.2019.101276

Hanauer, D. A., Zheng, K., Singer, D. C., Gebremariam, A., & Davis, M. M. (2014). Public awareness, perception, and use of online physician rating sites. JAMA, 311(7), 734-735. https://doi.org/10.1001/jama.2013.283194

Hennig-Thurau, T., Gwinner, K. P., Walsh, G., & Gremler, D. D. (2004). Electronic word-of-mouth via consumer-opinion platforms: What motivates consumers to articulate themselves on the Internet? Journal of Interactive Marketing, 18(1), 38-52. https://doi.org/10.1002/dir.10073

Holliday, A. M., Kachalia, A., Meyer, G. S., & Sequist, T. D. (2017). Physician and patient views on public physician rating websites: A cross-sectional study. Journal of General Internal Medicine, 32(6), 626-631. https://doi.org/10.1007/s11606-017-3982-5

Lagu, T., Hannon, N. S., Rothberg, M. B., & Lindenauer, P. K. (2010). Patients' evaluations of health care providers in the era of social networking: An analysis of physician-rating websites. Journal of General Internal Medicine, 25(9), 942-946. https://doi.org/10.1007/s11606-010-1383-0

Lagu, T., Metayer, K., Moran, M., Ortiz, L., Priya, A., Goff, S. L., & Lindenauer, P. K. (2017). Website characteristics and physician reviews on commercial physician-rating websites. JAMA, 317(7), 766-768. https://doi.org/10.1001/jama.2016.18553

Moskowitz, H. R., & Gofman, A. (2007). Selling blue elephants: How to make great products that people want before they even know they want them. Wharton School Publishing.

Mudambi, S. M., & Schuff, D. (2010). What makes a helpful online review? A study of customer reviews on Amazon.com. MIS Quarterly, 34(1), 185-200. https://doi.org/10.2307/20721420

Ranard, B. L., Werner, R. M., Antanavicius, T., Schwartz, H. A., Smith, R. J., Meisel, Z. F., Asch, D. A., Ungar, L. H., & Merchant, R. M. (2016). Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care. Health Affairs, 35(4), 697-705. https://doi.org/10.1377/hlthaff.2015.1030

Saifee, D. H., Zheng, Z., Bardhan, I. R., & Lahiri, A. (2020). Are online reviews of physicians reliable indicators of clinical outcomes? A focus on chronic disease management. Information Systems Research, 31(4), 1282-1300. https://doi.org/10.1287/isre.2020.0945

Terlutter, R., Bidmon, S., & Röttl, J. (2014). Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. Journal of Medical Internet Research, 16(3), Article e97. https://doi.org/10.2196/jmir.3145

Xu, Y., Armony, M., & Ghose, A. (2021). The interplay between online reviews and physician demand: An empirical investigation. Management Science, 67(12), 7344-7361. https://doi.org/10.1287/mnsc.2020.3879