HLTH 511 Week 2 Discussion Board Forum 2 Example

Discussion Board Forum 2: The Research Process and Trustworthy Evidence

HLTH 511 Research Methods in Public Health, Discussion Board Forum 2

Thread by [Student Name]

The classmates are composites; the textbook, Scripture and the peer-reviewed sources cited are real.

Public health relies on published studies to decide which programs to fund and which advice to give. If many published findings do not hold up when repeated, those decisions rest on weak ground. The chapters for this week help explain why this happens and what researchers can do about it, starting long before any data are collected.

Jacobsen (2017) describes research as a process that begins with a focused, answerable question, continues with a careful review of what is already known and requires a clear plan for data collection and analysis before the study begins. Each step protects against error. A vague question invites researchers to search the data for anything significant, and a plan made after seeing the data allows choices that favor a desired result.

The Open Science Collaboration (2015) repeated 100 studies published in three psychology journals, using larger samples and the original materials where possible. Although 97% of the original studies reported statistically significant results, only 36% of the replications did, and the replication effects averaged about half the size of the original ones. Ioannidis (2005) had argued years earlier that research findings are less likely to be true when studies are small, when effects are small, when many relationships are tested, when designs and analyses are flexible and when there are financial or other interests in a particular result. Most failed findings are not frauds; they are the predictable product of small studies, flexible analyses and a system that rewards surprising results. Public health research shares many of these conditions, especially in small community studies with many outcomes. A pilot of a school nutrition program, for example, might measure a dozen outcomes in two schools and report the one that reached significance, a pattern that makes a chance finding look like a discovery.

The research process offers remedies. Registering the question, hypotheses and analysis plan before collecting data limits flexible choices. Calculating sample size in advance reduces underpowered studies. Reporting all outcomes, not just significant ones, gives readers an honest picture. Sharing data and analysis code, where privacy allows, lets others check the work, and independent replication of important findings before they shape policy guards against acting on a fluke. None of these steps is expensive; they mainly require discipline and a willingness to accept results that are less exciting than hoped.

Ephesians 4:25 tells believers to lay aside lying and be truthful with each other, since they belong to one body (English Standard Version Bible, 2001/2016). For a Christian researcher, honesty includes not only avoiding fabrication but also resisting the subtle temptation to present a study as stronger than it is. Careful methods are a way of loving the neighbors who will live with the programs our research recommends.

Reply to Grace

Grace, you noted that journals are more likely to publish studies with significant results, and I think this publication bias helps explain the replication findings. When studies with null results remain unpublished, the published literature overstates how often interventions work and how large their effects are. Jacobsen (2017) emphasizes that a literature review should be comprehensive, but a review can only include what has been published, so even a careful researcher may build on an incomplete picture. The Open Science Collaboration (2015) found that replication effects were about half the size of the originals, which is what we would expect if journals favored studies that happened to find unusually large effects. In public health, this bias could lead a health department to adopt a program that looked effective in a few published pilot studies but would show little benefit in a larger trial. Trial registries and journals that publish well-designed studies regardless of results are two practical responses, and some journals now review a study's methods before results are known, accepting it on the strength of the design alone. As future MPH professionals, we can also look for systematic reviews that search registries and unpublished reports, since they give a more balanced picture of the evidence than a handful of exciting studies. When I read a systematic review now, I check whether the authors searched registries or tested for publication bias, because a review built only on published papers may repeat the very distortion you described. Asking that one question can change how much weight a finding deserves.

Reply to Omar

Omar, your concern that replication failures could damage public trust in science is important, especially after the pandemic, when guidance changed as evidence grew. Ioannidis (2005) argued that many published findings may be false because of small studies, flexible analyses and pressure to find significant results, but I think the answer to that problem is greater transparency rather than less communication. When public health agencies explain how strong the evidence is, admit uncertainty and update advice openly when new studies appear, they show the public that science corrects itself. Jacobsen (2017) describes research as a cumulative process in which each study adds to a body of knowledge, and communicating that process honestly may protect trust better than presenting each new finding as final. In our own work, we can avoid overstating results from a single study and describe what remains uncertain, even when a press office or funder would prefer a stronger headline, because credibility spent on one exaggerated claim is hard to earn back. People are more likely to trust experts who are candid about what they do not know than experts whose confident claims later change without explanation. Health departments could, for example, label guidance with the strength of the evidence behind it and explain in advance that recommendations may be revised as more studies are completed. That approach prepares the public for change instead of surprising them with it, and it treats community members as partners able to understand uncertainty rather than as audiences who must be shielded from it.

References

English Standard Version Bible. (2016). Crossway. (Original work published 2001)

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

Jacobsen, K. H. (2017). Introduction to health research methods: A practical guide (2nd ed.). Jones & Bartlett Learning.

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716