NURS 600 Week 7 Written Assignment 5 on Program Evaluation Example

Written Assignment 5: A Four-Level Evaluation Plan for a Simulation-Based Sepsis Recognition Module in a Nurse Residency

The Module and the Decision

The module is a four-hour session in the first month of a twelve-month nurse residency at a 400-bed hospital. Residents complete a 30-minute online lesson on sepsis screening and then rotate through two high-fidelity simulations of a patient developing sepsis on a medical unit, each followed by structured debriefing. About 120 residents complete the module each year, and it costs roughly 300 hours of educator time and simulation center use annually. The residency director must decide next year whether to keep the module as it is, shorten it, or extend it to experienced nurses transferring into the hospital. Oermann and Gaberson (2017) describe program evaluation as the systematic collection of information to judge the worth of a program and guide decisions, and the evaluation plan below is built around that decision.

The Framework

Kirkpatrick's framework looks at training from four vantage points: how learners felt about it, what they took away, what they now do differently at work, and what changed for the organization or its patients as a result. Each level answers a different question, and the higher levels are harder and more costly to measure but more important for the director's decision. A recent pilot shows how the first three levels can be measured together. Pham et al. (2026) evaluated a four-session conflict resolution program for 24 newly promoted nurse clinicians using a pre-post knowledge test, confidence ratings, focus groups and a three-month follow-up survey. Knowledge rose from a mean of 12.4 to 17.8 on a 20-item test and confidence from 2.8 to 4.2 on a five-point scale, both largely maintained at three months, and 92% reported applying the strategies at work.

The Evaluation Plan

Levels 1 and 2: Reaction and Learning

At the reaction level, residents rate the module's relevance, the realism of the scenarios and the usefulness of debriefing, and they add one comment on what to change. Reaction surveys cost little and help refine delivery, but they cannot show that learning occurred. A national evaluation of nine asynchronous cancer care e-learning modules illustrates both strengths and limits: Armoogum et al. (2026) surveyed 123 participants using a Kirkpatrick-based questionnaire and found that 88.6% rated the program good or very good and most reported improved knowledge, but the data were self-reported.

At the learning level, residents take a 15-item sepsis test before the online lesson and after the simulations, and educators rate each resident's performance in the second simulation with the Lasater Clinical Judgment Rubric. The test measures knowledge, while the rubric measures noticing, interpreting, responding and reflecting in a realistic situation.

Level 3: Behavior

The behavior level asks whether residents screen for sepsis and escalate appropriately on the units. Two to four months after the module, a trained auditor reviews 60 charts of patients cared for by residents who had a positive sepsis screen, checking that the screen was documented, that the provider was notified within the hospital's standard of 30 minutes, and that the notification included key data. Behavior in practice is the level the director most needs, because a resident who scores well in simulation but never escalates on the unit has not been helped by the module.

Level 4: Results

At the results level, the quality department reports the median time from a positive sepsis screen to provider notification on units where residents work, compared with the same quarter the previous year. Oermann and Gaberson (2017) caution that outcomes at this level are influenced by many factors besides education, such as staffing and changes in the screening tool, so the evaluation will describe results as associations, not proof that the module caused them.

A Design Question the Evaluation Can Answer

The evaluation can also inform a practical design question. Because of simulation center limits, half the residents in each session observe one scenario rather than participating actively. Amechghal et al. (2026) reviewed 15 studies with 1,799 participants comparing active participants and observers in simulation-based education and found that most outcomes at the reaction level, and knowledge, skills and most attitudes at the learning level, did not differ significantly between roles, although active participants reported more emotional arousal. Comparing learning scores between residents who participated actively and those who observed in our module would show whether the same pattern holds here, which could allow larger groups and a shorter module.

Using the Results

Results will be summarized in a two-page report to the residency director and the education council each spring, with a recommendation to keep, revise or extend the module. Strengths, weaknesses and the data behind each will be stated plainly. If the behavior audit shows that residents screen but do not escalate within the standard, the report will recommend revising the debriefing to focus on escalation and repeating the audit the following year, rather than declaring the module a success because residents enjoyed it and scored well on the test. If residents who observed learned as much as those who participated actively, the report will propose larger groups, which would free educator time for a refresher session at six months. Educators spend time, and residents spend hours away from patient care, on every education program a hospital offers, and Proverbs 21:5 observes that "the plans of the diligent lead surely to abundance" (English Standard Version Bible, 2001/2016), a reminder that careful planning, including planning how to judge our own work, is part of faithful service. Evaluating whether those hours produce better care is part of faithful stewardship.

References

Amechghal, A., Naciri, A., Takhdat, K., Loubbairi, S., Mellouki, A., Lahlou, L., & Nassik, H. (2026). The impact of the learner role in healthcare simulation-based education on learning outcomes: A systematic review. Journal of Educational Evaluation for Health Professions, 23, Article 24. https://doi.org/10.3352/jeehp.2026.23.24

Armoogum, J., Davis, J., Gale, J., & Taylor, V. (2026). Impact of an asynchronous e-learning intervention on cancer knowledge and clinical practice for pre-registration nursing and allied health professions students and cancer support workers. Journal of Cancer Education. Advance online publication. https://doi.org/10.1007/s13187-026-02970-6

English Standard Version Bible. (2016). Crossway. (Original work published 2001)

Oermann, M. H., & Gaberson, K. B. (2017). Evaluation and testing in nursing education (5th ed.). Springer Publishing Company.

Pham, H. Y., Lim, S. H., Lee, M., Aloweni, F., Jaafar, K. B., & Ng, G. N. (2026). A mixed-methods pilot evaluation of a principled negotiation conflict resolution training program for newly promoted nurse clinicians. Journal of Continuing Education in Nursing. Advance online publication. https://doi.org/10.3928/00220124-20260706-01