Healthcare Quarterly

Healthcare Quarterly 17(4) December 2014 : 7-9.doi:10.12927/hcq.2015.24124

Abstract

If you were to have an operation tomorrow, would you want your surgical team members to feel comfortable speaking up, to defy hierarchy, to interact with each other just as well as they perform technical aspects of the procedure? Would you want to feel like part of the team? Your answers to these admittedly leading questions are based on the culture of the surgical team and the interdependence of team members and are at the heart of a current debate around the surgical checklist's effectiveness.

In British Columbia (BC), many individuals responded to the paper by Urbach et al. (2014) that described the minimal impact on patient mortality after implementation of the surgical safety checklist in Ontario. They wrote to the Surgical Quality Action Network (SQAN) to express their perspectives, and interestingly, some refuted and others supported the conclusions. Given the strong reaction this study created in the surgical community, a number of key stakeholders have prepared a response in order to provide another perspective to the article and emphasize the checklist's value for improving the culture of surgical teams.

A Different Perspective

Urbach et al. (2014) describe the association of short-term patient outcomes prior to and after implementation of the checklist. Their conclusion identified no statistical evidence of checklist use decreasing selected morbidity and mortality in the surgical population that they studied. However, the research had a number of important design limitations that put into question these conclusions. Surgical community members questioned why it was retrospective and non-randomized, why a sample size calculation was not performed – particularly in light of an adjusted mortality rate that, compared to previous "positive" studies, was low (0.71%) at baseline – and why effectiveness of the checklist was assessed so early (three to six months) after implementation. More importantly, many questioned how strong conclusions about the checklist's apparent ineffectiveness could be based on a result that only narrowly missed statistical significance (a P value of 0.07% – i.e., just missing the "magic" 0.05 value) for the observed difference in adjusted mortality (post checklist introduction of 0.65%).

Unfortunately, in addition to the weaknesses identified above, the article also did not address human factor improvements such as teamwork and communication, efficiency or reductions in near misses. A breakdown in teamwork and communication is usually at the centre of a poor outcome (Catchpole et al. 2006; Flin et al. 2003; Lingard et al. 2008; Mishra et al. 2009; Yule et al. 2006). Technical initiatives such as the checklist predictably fail if attention is not paid to the contextual culture and human factors in the area where they are implemented. Surgical environments with a challenged patient safety culture usually struggle to adopt any tool, including the checklist.

Improvements in Surgery

This study has stimulated a renewed conversation about the checklist, its role and how it can be used. It has served as an excellent reminder as to why we conduct quality improvement work in the first place and prompts us to consider:

  1. the limitations to generalizations of statistical significance related to patient safety,
  2. the fine line that exists between the definitions of implementation and team adoption as it relates to culture,
  3. the drivers of change, the bias we all have in perceiving we are doing the right thing all the time and the time required for humans to change and
  4. the need for evaluations, assessments and audits to address our biases and to keep all team members accountable.

Quality improvement initiatives are commonly challenged and critiqued based on the statistical significance of controlled experiments and empirical evidence. A disconnect can occur between the "intuitive benefits" to patient care versus "statistical significance." For example, limited empirical evidence demonstrates improved outcomes with pulse oximetry and end-tidal carbon dioxide monitoring in the anaesthesia domain. Yet few individuals would abandon these tools to rely solely on the vigilant practitioner to monitor surgical patients (Smith and Pell 2003). To this point, tools that aid the team in providing excellent care in complex systems are useful, although they may not be the sole determinant of outcomes. When considering a strong patient safety culture, the intuitive benefits of quality patient care go above and beyond mortality rates and the statistical significance of such findings.

A Safety Culture

Culture refers to shared attitudes, beliefs and patterns of behaviour that determine how a team functions. While many in healthcare hope to "flick the switch and watch a transformation unfold" when introducing improvements such as the surgical checklist, a strong patient safety culture is imperative for successful implementation of new tools and guidelines. Although measuring the culture of a team can be difficult, measurement tools and surveys show links between good culture and better outcomes for patients and clinical satisfaction (Manser 2009). For example, the Joint Commission identified communication breakdown as the most common factor implicated in adverse events (Joint Commission 2008), and culture-related interventions have shown improvements to clinical outcomes and reductions in adverse events (Catchpole et al. 2006; Lingard et al. 2008; Mishra et al. 2009; Neily 2010). The checklist has challenged the performance of teams and the culture of many organizations in BC. Effective implementation of a checklist is a cultural process, not a technical one (Bosk et al. 2009).

The Checklist's Possibilities

The checklist is about starting and framing a conversation with the entire team – patient included. It is more than a prescriptive "tick box" activity but rather confirming the completion of the actions it calls for by focusing on communication, teamwork and leadership (Hayes et al. 2009; Leape 2014). The success is tied to changing operating room culture so that all team members have greater situational awareness and are more likely to speak up if they detect a problem. All members have a common understanding of the patient, his or her needs, the operative plan and the goals to be achieved.

As Pronovost et al. (2006) wrote "The answer to the question on what a simple checklist can achieve is; on its own, not much." There is a drastic difference between "doing it" and "doing it well!" When properly applied, the checklist does, of course, remind the team of those things that should be done for every patient. But the value of such an activity is lost if we simply go through the motions, prescriptively answer questions without critical decision-making and forget the patient.

Compliance with the Checklist in BC

Across BC, self-reported compliance scores of checklist use are approximately 87%, with a health authority range between 70% and 98%. However, measuring compliance can induce a false sense of security in how well teams are actually performing and even how often it is actually implemented (Hilligoss and Moffatt-Bruce 2014; Rydenfält et al. 2014). Consistent with the findings found by Makary et al. (2006), two health authorities in BC conducted an observational evaluation that team members perceive proper completion of the checklist to be higher than actually completed. With this in mind, how can we move away from the "tick box mentality" to changing the "surgical team culture" and supporting teamwork and communication in the surgical suites?

How Can We Help Ensure the "Conversation" Happens?

The letters to SQAN provided first-hand, vivid stories of how the checklist is more than a simple tool.

"I would feel most uncomfortable now not conducting a surgical checklist/pause and verifying with the entire surgical team, that we are all in agreement as to what we are doing, and how we are going to do it" said a surgeon. "The prescriptive checklist notion was never the intent (and indeed the WHO encouraged local modifications of the 'standard' checklist). Rather the purpose was to change the culture of how we function as a surgical team."

And a Quality and Patient Safety Leader described their personal experience as a patient:

"I asked my surgeon if the surgical safety checklist would be used for my surgery. The response was 'yes, of course' and proceeded to include me in the introductions, explain the procedure, and participate in a comprehensive discussion with the team around the safety critical elements relating to my surgery. This made me feel like I was part of the conversation, a member of my surgical team."

In BC, we are making it a priority to improve teamwork and communication in the surgical suites by empowering team members with the skills to communicate, be situational leaders and be better prepared to perform as a team so everyone feels respected and accountable. In the absence of direct monitoring or observation, understanding the safety culture is truly unknown (Leape 2014). Adoption of these skills will require resources and expertise to provide teams with coaching and training. It will take time for teams to adopt these skills so they can foster a safety culture where comprehensive communication and teamwork is the norm. We encourage all surgical team members to observe the environment in which they work. Find an opportunity where it is your sole role to watch a case or procedure. Inevitably, you will see technical components but, more importantly, you will see how the team performs.

The checklist can help create an environment in which everyone's voice is heard and valued. It includes anaesthesiologists, housekeepers, nurses, patients, porters, students, surgeons and technologists. This is likely the surgical environment you would want to be treated in.

About the Author

Allison M. Muniak, M.A.Sc, BSc, is the Human Factors Specialist with Vancouver Coastal Health and the B.C. Patient Safety and Quality Council.

D. Douglas Cochrane, MD, FRCSC, is the Chair of the B.C. Patient Safety and Quality Council.

Marlies van Dijk, RN, MSc, Provincial Implementation Lead: Innovation, Quality and Healthcare Improvement, Alberta Health Services.

Andy Hamilton, FRCPC Anesthesia, CCPE, is the Medical Director, Surgical Services, Interior Health Authority of B.C., and Co-Chair of the Provincial Surgical Executive Committee.

Stephan K.W. Schwarz, MD, PhD, FRCPC, is an Associate Professor in the Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia.

J. Patrick O'Connor, MD, FRCP(C) Anesthesia, is the Vice-President of Medicine and Quality & Safety with Vancouver Coastal Health.

Ramesh L. Sahjpaul, MD, FRCSC, is the Chief, Department of Surgery and Medical Director for the Surgery Program at Lions Gate Hospital, Vancouver Coastal Health.

Acknowledgment

The authors wish to acknowledge the participation of the Surgical Quality Action Network of British Columbia members in the content of the article and the assistance of the British Columbia Patient Safety and Quality Council (BCPSQC) staff members Andrew Siu, Kevin Smith and Corrina Hayden.

References

Bosk C.L., M. Dixon-Woods, C.A. Goeschel and P.J. Pronovost. 2009. "The Art of Medicine: Reality Check for Checklists." The Lancet 374(9688): 444–45.

Catchpole, K.R., A.E.B. Giddings, M.R. deLeval, G.J. Peek, P.J. Godden, M. Utley et al. 2006. "Identification of Systems Failures in Successful Paediatric Cardiac Surgery." Ergonomics 49(5/6): 567–88.

Disease-Specific Care Certification – National Patient Safety Goals. 2008. The Joint Commission. Oak Brook Terrace, IL. <http//www.jointcommision.org>.

Flin, R., G. Fletcher, P. McGeorge, A. Sutherland and R. Patey. 2003. "Anaesthetists' Attitudes to Teamwork and Safety." Anaesthesia 58(3): 233–42.

Hayes, A.B., T.G. Weiser, W.R. Berry, S.R. Lipsitz, A.H. Breizat, E.P. Dellinger et al. 2009. "A Surgical Safety Checklist to Reduce Morbidity and Mortality in a Global Population." The New England Journal of Medicine 360(5): 491–99.

Hilligoss, B. and S.D. Moffatt-Bruce. 2014. "The Limits of Checklists: Handoff and Narrative Thinking." BMJ Quality Safety 23(7):528–33. doi: 10.1136/bmjqs-2013-002705.

Leape, L.L. 2014. "The Checklist Conundrum." The New England Journal of Medicine 370(11): 1063–64.

Lingard, L., G. Regehr, B. Orser, R. Reznick, G.R. Baker, D. Doran et al. 2008. "Evaluation of a Preoperative Checklist and Team Briefing among Surgeons, Nurses, and Anaesthesiologists to Reduce Failures in Communication." Archives of Surgery 143(1): 12–17.

Makary, M.A., J.B. Sexton, J.A. Freischlag, C.G. Holzmueller, E.A. Millman, L. Rowen et al. 2006. "Operating Room Teamwork among Physicians and Nurses: Teamwork in the Eye of the Beholder."Journal of the American College of Surgeons 202(5): 746–52.

Manser, T. 2009. "Teamwork and Patient Safety in Dynamic Domains of Healthcare: A Review of the Literature." ActaAnaesthesiolScand 53(2): 143–51.

Mishra, A., K. Catchpole and P. McCulloch. 2009. "The Oxford NOTECHS System: Reliability and Validity of a Toll for Measuring Teamwork Behaviour in the Operating Theatre." Quality & Safety in Health Care 18(2): 104–08.

Neily, J. 2010. "Association between Implementation of a Medical Team Training Program and Surgical Mortality." Journal of the American Medical Association 304(15): 1693–700.

Pronovost, P., D. Needham, S. Berenholtz, D. Sinopoli, H. Chu, S. Cosgrove et al. 2006. "An Intervention to Decrease Catheter-Related Bloodstream Infections in the ICU." The New England Journal of Medicine 355(26): 2725–32.

Rydenfält, C., Å. Ek and P.A. Larsson. 2014. "Safety Checklist Compliance and a False Sense of Safety: New Directions for Research." BMJ Quality & Safety 23(3): 183–86.

Smith, G.C.S. and J.P. Pell. 2003. "Parachute Use to Prevent Death and Major Trauma Related to Gravitational Challenge: Systematic Review of Randomised Controlled Trials." British Medical Journal 327(7429): 20–27.

Urbach, D.R., A. Govindarajan, R. Saskin, A.S. Wilton and N.N. Baxter. 2014. "Introduction of Surgical Safety Checklists in Ontario, Canada." The New England Journal of Medicine 370(11): 1029–38.

Yule, S., R. Flin, S. Paterson-Brown and N. Maran. 2006. "Non-Technical Skills for Surgeons: A Review of the Literature." Surgery 139(2): 140–49.

Comments

Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed