MRKT 600 Week 7 Survey Instrument Example

Survey Instrument and Analysis: How 62 Adults Weighed Star Ratings and Comments When Choosing a Primary Care Physician

Research Question and Hypotheses

The research question asks how adults who recently chose a primary care physician weigh star averages against written comments. The Week 4 interviews suggested two hypotheses. First, respondents will rate written comments as more trustworthy than star averages. Second, when a higher-rated physician has specific negative comments and a lower-rated physician has none, the choice will differ by age group, with younger respondents more likely to choose the physician without complaints.

The Instrument

The instrument has fourteen items in four sections. As Alreck and Settle (2004) advise, items use plain language, ask one thing at a time and give balanced response options, and the trust items share one seven-point scale so they can be compared directly. Each item traces to the analysis plan: items 5 and 6 feed the paired comparison, items 3 and 10 feed the test of choice by age, and the remaining items describe the sample and help explain the results. The choice task in item 3 was written from an interview in which a participant described skipping a highly rated doctor after reading about lost referrals, so the scenario reflects a real decision rather than an abstract trade-off. Item 9 gives personal recommendation its own option because the older Week 4 participant started from a neighbor's advice and opened reviews mainly to look for warning signs.

Section A: Screening.

• 1. Within the last 24 months, did you start seeing a new primary care physician? (Yes / No; No ends the survey)
• 2. Did you look at online reviews of any physician while choosing? (Yes / No)

Section B: Choice task.

• 3. Imagine two physicians who both accept your insurance and have open appointments next week. Dr. A has a 4.8-star average from 120 reviews, and three of the most recent comments describe referrals that were never sent. Dr. B has a 4.4-star average from 95 reviews and no comments about problems. Which would you choose? (Dr. A / Dr. B)
• 4. How confident are you in that choice? (1 = not at all confident, 7 = completely confident)

Section C: Trust and use.

• 5. How much do you trust a physician's star average to tell you what care will be like? (1 = not at all, 7 = completely)
• 6. How much do you trust written comments to tell you what care will be like? (same scale)
• 7. When choosing, about how many written comments did you read? (None / One to five / Six to twenty / Over twenty)
• 8. Did a friend, relative or another clinician recommend the physician you chose? (Yes / No)
• 9. Which mattered more in your final choice? (Reviews / Personal recommendation / Insurance and location / Something else)

Section D: About you.

• 10. Age group (18 to 34 / 35 to 54 / 55 or older)
• 11. Gender (Woman / Man / Prefer to self-describe / Prefer not to say)
• 12. Highest education (High school or less / Some college / Bachelor's degree / Graduate degree)
• 13. Do you have a long-term health condition that requires regular visits? (Yes / No / Prefer not to say)
• 14. Is there anything else you want to tell us about how you chose your physician? (Open text)

Administration

The survey was administered online over ten days to a convenience sample recruited through a church bulletin, two workplace newsletters and a community social media group. Consent language at the start explained the study's purpose, that participation was voluntary and anonymous, and that results would be used for a class assignment. Of the respondents who passed the screening questions, 62 completed the survey: 17 aged 18 to 34, 33 aged 35 to 54 and 12 aged 55 or older. The plan in Forum 3 had set a quota of fifteen respondents over 55, which was not met. Before launch, three acquaintances completed a pilot version. Two found the original wording of item 3, which named star averages as "ratings," ambiguous, so the item was rewritten to give the number of reviews and describe the comments plainly. The pilot also showed that the survey took about four minutes, which was stated in the invitation.

Results

Respondents rated written comments as more trustworthy than star averages. A paired t test on items 5 and 6 found a mean difference of 0.90 points, t(61) = 3.91, p < .001, with a standardized effect size (Cohen's dz) of 0.50, which Cohen (1988) would classify as medium in size. The first hypothesis was supported for this sample.

Overall, 66.1 percent chose Dr. B, the lower-rated physician without complaints. The choice differed by age group, χ²(2, N = 62) = 11.24, p = .004, Cramér's V = 0.43. Respondents 55 and older were much more likely to choose Dr. A, the higher-rated physician, and among them the mean trust in star averages, 4.92, slightly exceeded trust in comments, 4.67, the reverse of the younger groups. One expected cell count, 4.1, fell below 5, so the chi-square result should be treated with caution; an exact test would be preferable in a larger analysis.

Interpretation

The results are consistent with the Week 4 interviews and with research showing that review text shapes physician choice. In this sample, specific complaints about referrals outweighed a 0.4-point advantage in star average for most respondents under 55, while older respondents leaned on the average. Creswell (2014) advises that quantitative results be interpreted in relation to the research question and the hypotheses, and here they suggest that health systems publishing reviews should expect comments, not only averages, to drive choices among younger patients. For a practice manager, the practical lesson is that answering specific complaints, such as the referral problems in the scenario, may matter more than raising the average by a few tenths. The older group's pattern deserves its own study, since a group of twelve cannot show whether the difference reflects age, health needs or habits of reading reviews. These questions carry into the Week 8 prospectus.

Limitations

The sample is a small convenience sample recruited through channels that likely overrepresent people connected to churches, employers and local online groups, and the 55-and-older group is small. The results describe these respondents and cannot be generalized to all adults choosing physicians. The choice task is hypothetical, and stated choices may differ from real ones. A probability sample of new patients, drawn from a health system's scheduling records, would be needed to test whether the age pattern holds in the population. Alreck and Settle (2004) note that well-designed surveys still depend on who answers them, and this report's conclusions are limited accordingly.

References

Alreck, P. L., & Settle, R. B. (2004). The survey research handbook (3rd ed.). McGraw-Hill/Irwin.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates.

Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). SAGE Publications.