Healthcare Quarterly

Abstract

The Current State and Prevailing Wisdom

Medication errors leading to adverse events continue to be a challenge in the acute care setting. The latest data suggest that more than 1.5 million patients are harmed each year in the United States by medication errors, incurring US$3.5 billion in extra costs to hospitals alone (Aspden et al. 2007). These results are despite the higher level of attention this issue has received in healthcare over the past decade and the new developments and investments in technology to address medication error. This investment has been in part due to the strong advocacy for the use of technology to address patient safety, such as computerized physician order entry (CPOE) , bar-coding and smart pump infusion systems (Bates 2000, 2007; Poon et al. 2006; Smetzer 2002; Vanderveen 2007; Young 2006).

Medication ordering and administration are the two areas of the delivery process with the highest incidence of error (Bates et al. 1995). We look to the two main technologies designed to address these problem areas, CPOE and smart pump systems, respectively.

Medication Ordering: The State of CPOE

Beyond the advocacy, there has been some evidence to suggest that CPOE systems are effective. A 2006 study of the use of CPOE in an intensive care unit showed a dramatic difference in the incidence of medication prescription error compared with a paper-based system, 3.4% vs. 27.0% (Colpaert et al. 2006). Another study in a pediatric critical care unit showed the complete elimination of medication prescription errors (Potts et al. 2004).

Systematic reviews on CPOE have also shown some positive outcomes with mostly in-house developed systems (Kaushal et al. 2003), but the most recent reviews of commercial systems have found little high-quality evidence to support their use, with all reviews concluding that more study is needed (Ammenwerth et al. 2008; Poissant et al. 2005; Wolfstadt et al. 2008).

Even more concerning may be the iatrogenic aspects of CPOE. There have been a number of notable studies since 2005 that have indicated potential harm as a result of CPOE, including findings that showed that the systems facilitated medication errors (Koppel et al. 2005), adverse events (Ash et al. 2007) and even an increase in pediatric mortality rates (Han et al. 2005).

Given this lack of clarity in the evidence, it should not be surprising that CPOE adoption rates are low, with only 5% of US hospitals using the technology in 2005 (Cutler et al. 2005), climbing to only 8% in 2008 (Anderson 2009). The more likely reason for the lack of adoption is the significant cost of not only the commercial product and its technical implementation, but also the associated process redesign, training and change management initiatives. Even those who have adopted CPOE continue to struggle with issues of compliance, the defining of common order sets and alert fatigue caused by evidence-based decision support (Anderson 2009). No other CPOE implementation has received more attention than that at the Cedar-Sinai Medical Center in Los Angeles in 2002, where the medical staff protested by refusing to use the system after only three months, forcing the administration to shelve the US$34 million investment. The deployment was so difficult that the hospital has not resurrected its use, setting back the initiative for many years (Connolly 2005, March 21).

Medical Administration: The State of Smart Pump Systems

The other area of high incidence of error is medication administration. Smart pump systems use hospital-specific drug libraries that define the upper and lower allowable dosing limits for the drug being administered. The device alerts the user if the entered parameters result in a dosage that exceeds the allowable soft limits. The devices can also be configured to prevent the pump from running altogether if the programming exceeds predefined hard limits. However, we found that most institutions report that clinicians often ignore and override soft limit alerts. This is consistent with results from other studies that have shown the same behaviour (Institute for Safe Medication Practices 2007; Murdoch et al. 2008; Pratt 2004), reducing the benefits of having a drug library.

Despite reported adoption levels of smart pump technology as high as 20-25% of all US hospitals (Moyer 2005), our own analysis of smart pump deployments in Ontario shows implementation in only approximately 10% of hospitals, with another 7% planning its use in the near future. Since infusion pump manufacturers have all but abandoned the development of traditional pumps in favour of some form of smart pump technology, most hospitals have converted to smart pumps primarily to replace their aging population of traditional pumps, not necessarily because they were driven by the purported safety rationale of the technology.

As a consequence, many institutions have treated the implementation as they would a traditional infusion pump deployment. We have found that for smart pumps' benefit to be fully realized, a comprehensive drug library must be developed, deployed, maintained and updated via wireless communication. This requires a human resource and technical infrastructure for which many hospitals are not prepared. Some sites involved pharmacy in the initial development of the drug library but did not plan or budget for their involvement in ongoing maintenance. Consequently, many institutions have incomplete drug libraries; forcing clinicians to bypass the pumps' safeguards, thereby circumventing the benefits of the system by reverting to purely manual programming - which is prone to error.

The Opportunity Cost

The current state of these technologies has caused some to argue that the healthcare sector has been too quick to adopt them without sufficient evidence and study. The opportunity cost as a result of the significant investment in these technologies could set back the patient safety agenda by diverting much-needed funding from strategies that are more effective (Auerbach et al. 2007). A more moderate perspective is the pragmatic over the dogmatic, as Ammenwerth et al. suggest with the often-contradictory findings of CPOE studies: "Learn from failures, but also avoid both uncritical scepticism that may arise from drawing overly general conclusions from one negative trial, as much as uncritical optimism from limited successful ones" (Ammenwerth et al. 2006 : 586).

We argue that, given that these technologies are still in their infancy, a user-centred approach to their design and implementation will eventually result in a strong positive impact on patient safety.

The Unrealized Potential

It is notable that we describe, study and deploy these technologies in isolation, rather than as components of an integrated medication delivery process. This fragmented approach has likely hobbled these technologies and prevented them from reaching their full potential. CPOE without the use of clinical decision support has been shown to initially increase the rate of adverse drug events in a highly computerized hospital (Nebeker et al. 2005). A singular, overdependence on CPOE technology has also been attributed to unintended adverse outcomes in Oregon (Campbell et al. 2007).

A highly publicized case of a chemotherapy overdose at the Cross Cancer Institute in Edmonton, Alberta, is an example of a hospital with a high-level of computerization, including CPOE, that still experienced a tragic adverse outcome. This technological sophistication of the medication delivery process did not extend to the point of administration, where nurses were required to perform a complex manual calculation of the ambulatory pump's rate of infusion (U et al. 2007).

Rather than having the continuity that is needed in realizing a complete medication delivery system, industry produces and healthcare organizations deploy a number of technologies that are often not interoperable and that do not facilitate a single overarching purpose. The result is a tapestry of interventions that were not necessarily designed to work together or for which the designers never foresaw the manner in which they would be implemented. Ultimately, the benefit remains not fully realized due to design flaws but also due to unforeseen organizational behaviour. New approaches to address these challenges are already well established in other industries and have only recently been applied to healthcare.

Human Factors and the User-Centred Perspective

Using knowledge of the strengths and limitations of human performance, a human factors, user-centred perspective can inform system design to ensure that individual technologies achieve their intended purpose and benefits. This user-centred perspective has been described as the human-tech approach (Vicente 2004). This approach focuses on understanding the needs of all users of a system based on their goals, required tasks, environmental constraints, skill and knowledge. Starting with an understanding of user needs, a user-centred approach to designing, evaluating and implementing new systems helps identify the extent to which users' needs are met and how the system is likely to fail in anticipated circumstances. Based on these data, decisions can be made about how to refine the design, purchase a different system or develop additional support mechanisms prior to implementation. The approach should extend to seemingly non-technological aspects of implementation, such as user work schedules, staffing levels and the physical environment, all of which have an impact on human performance and the success of the implementation.

What Can Be Done: Usability Testing to Inform Procurement

Usability testing identifies safety problems through the observation of actual users performing representative tasks using the technology under evaluation in a controlled environment. Applying this approach for the purposes of procurement can allow an organization to validate the product's safety claims and understand first-hand the challenges of its implementation and clinician adoption. Simple experimental usability testing can be achieved with as few as six to eight users with appropriately crafted scenarios.

A review of the literature has shown widespread usability problems with CPOE products (Khajouei and Jaspers 2008). Our own work in this area on behalf of University Health Network and other healthcare organizations has provided valuable data for decision-makers to consider for their high-risk technology implementations. This includes the comparative usability evaluation of CPOE systems for chemotherapy we conducted, which identified that one product had 13 issues of critical severity. Users successfully completed scenario tasks in only 62% of the cases. The other product under testing had an 85% success rate with only one issue of critical severity. A traditional approach to procurement would not have been likely to reveal these outcomes. Only during implementation would many of these issues be discovered. Upon choosing a product, the organization would have a much greater chance of a securing design changes from the vendor with these findings, as well as mitigating the implementation risk through targeted education and change management initiatives.

Our usability testing also explained many of the problematic implementations of smart pump systems. The testing highlighted the manner in which clinicians use the device and respond to dosing alerts when the smart pumps are programmed erroneously. Furthermore, usability testing identified workarounds (e.g., not using the drug library) that were commonly reported in our interviews with other Ontario hospitals as well as in the literature (McAlearney et al. 2007).

Human Factors in System Implementation

The human-tech approach also addresses factors that impact organizational behaviour such as leadership and safety culture. These macro-level factors are important determinants for successful change. Implementing new, large-scale technologies such as CPOE and smart pump systems requires leadership and a strong safety culture to overcome the challenges associated with change (Abrams and Carr 2005). Human-tech methods such as usability testing and field study observations are inclusive of all stakeholders and create opportunities for a wide range of input. This approach requires leadership from those managing implementation projects and can promote user acceptance by allowing for an open dialogue across levels of the system (Saathoff 2005).

To this end, another standard human factors technique that facilitates technology planning and implementation is a user needs assessment (UNA). UNA is a process of identifying key stakeholders and discovering and assessing their needs. A UNA would help ensure that institutions assemble a multidisciplinary team that enables (1) communication with all stakeholders in the medication administration process, (2) the infrastructure for safe technology implementation and (3) effective maintenance after implementation.

This tactic was used implicitly in one of the few major successful CPOE deployments in Canada (Abrams and Carr 2005; Rossos et al. 2006). Other institutions, such as Virginia Mason Medical Center, have implicitly used this approach through the deliberate design of standardized order sets on paper, before moving to the electronic system (Anderson 2009).

In our own UNA analysis of smart pump implementations, we found a lack of key stakeholders needed to maximize the safe and effective use of the technology. In particular, there was little or no involvement from pharmacy, information technology and risk management. The result was a lack of resources and skills to develop and maintain components of the system, including the drug libraries, continuous quality improvement (CQI) logs and wireless infrastructure. Thus, the majority of institutions are uninformed as to (1) how well smart pump features are being used and (2) whether smart pump technology actually increases the safety of their intravenous medication administration practice.

Holistic Approach from Industry and Providers

The use of these techniques transcends all levels of a patient safety implementation, incorporating the micro (user interfaces, ergonomics), the meso (inter-system communication and integration) and the macro (organizational design) perspectives. To realize the full potential of these systems, a holistic approach is needed with a higher standard of technical and process integrations, from the direct transfer of orders through to the automated programming of infusion pumps. Crane and Crane (2006) envision complete system designs from the electronic health records through CPOE and clinical decision support, bar-coding and, finally, automated dispensing machines. We would extend this design through to the point of administration with complete smart pump system integration and positive patient identification. Although idealistic, this level of holistic design is the norm in other industries, which recognize it as a key element of success.

Unprecedented healthcare industry partnership is needed for the realization of such systems. No single company has the ability to deliver such a design with this level of integration. Those companies that embrace open standards and collaborate on a complete user-centred, fully-interoperable design will position themselves for market leadership. Companies that participate in the Integrating the Health Environment (IHE)initiative are examples of what is required to move in this direction, which has already been so successful in the transformation of medical imaging (Vegoda 2002).

It will be the responsibility of the healthcare organizations to demand user-centred systems from industry. Hospitals should consider enforcing these high standards, instead of continually compromising their implementations by developing workarounds and relying on costly change management initiatives, ultimately leaving the user and the patient to deal with the consequences of an incomplete, fragmented design.

Conclusion

Ultimately, much of what is required from industry and healthcare providers is to refocus their efforts to develop and implement patient safety systems using simple, intuitive human factors methods as outlined above. The user-centred perspective transcends the organizational boundaries of medicine, pharmacy and nursing to produce a system that ensures an integrated approach that addresses the entire medication delivery process.

About the Author

Joseph A. Cafazzo, PhD PEng, is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, University Health Network (UHN); the Department of Health Policy, Management and Evaluation, Faculty of Medicine, University of Toronto; and the Institute of Biomaterials and Biomedical Engineering, Faculty of Medicine, University of Toronto, Toronto, Ontario.

Patricia L. Trbovich, PhD, is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, UHN; and the Institute of Biomaterials and Biomedical Engineering, Faculty of Medicine, University of Toronto.

Andrea Cassano-Piche, MASc PEng, is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, UHN.

Anjum Chagpar, MHSc PEng, is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, UHN.

Peter G. Rossos, MD, MBA, FRCP(C), FACP is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, UHN; and the Department of Medicine, UHN, Faculty of Medicine, University of Toronto.

Kim J. Vicente, PhD PEng, is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, UHN; the Cognitive Engineering Laboratory, University of Toronto; and the Department of Mechanical and Industrial Engineering, Faculty of Applied Science and Engineering, University of Toronto.

Anthony C. Easty, PhD PEng CCE, is a member of Healthcare Human Factors, Centre for Global eHealth Innovation, UHN; and the Institute of Biomaterials and Biomedical Engineering, Faculty of Medicine, University of Toronto.

References

Abrams, H. and D. Carr. 2005. "The Human Factor: Unexpected Benefits of a CPOE and Electronic Medication Management Implementation at the University Health Network." Healthcare Quarterly 8 (Special No.): 94-98.

Ammenwerth, E., J. Talmon, J.S. Ash, D.W. Bates, M.C. Beuscart-Zephir, A. Duhamel, P.L. Elkin, R.M. Gardner and A. Geissbuhler. 2006. "Impact of CPOE on Mortality Rates - Contradictory Findings, Important Messages." Methods of Information in Medicine 45(6): 586-93.

Ammenwerth, E., P. Schnell-Inderst, C. Machan and U. Siebert. 2008. "The Effect of Electronic Prescribing on Medication Errors and Adverse Drug Events: A Systematic Review." Journal of the American Medical Informatics Association 15(5): 585-600.

Anderson, H.J. 2009. "CPOE: It Don't Come Easy." Health Data Management 17: 18.

Ash, J.S., D.F. Sittig, E.G. Poon, K. Guappone, E. Campbell and R.H. Dykstra. 2007. "The Extent and Importance of Unintended Consequences Related to Computerized Provider Order Entry." Journal of the American Medical Informatics Association 14(4): 415-23.

Aspden, P., J. Wolcott, L. Bootman and L.R. Cronewett. 2007. Preventing Medication Errors: Quality Chasm Series. Washington, DC: National Academies Press.

Auerbach, A.D., C.S. Landefeld and K.G. Shojania. 2007. "The Tension between Needing to Improve Care and Knowing How to Do It." New England Journal of Medicine 357(6): 608-13.

Bates, D.W. 2000. "Using Information Technology to Reduce Rates of Medication Errors in Hospitals." BMJ 320(7237): 788-91.

Bates, D.W. 2007. "Preventing Medication Errors: A Summary. American Journal of Health-System Pharmacy 64(14 Suppl. 9): S3-9.

Bates, D.W., D.J. Cullen and N. Laird. 1995. "Incidence of Adverse Drug Events and Potential Adverse Drug Events: Implications for Prevention." Journal of the American Medical Association 274: 29-34.

Campbell, E.M., D.F. Sittig, K.P. Guappone, R.H. Dykstra and J.S. Ash. 2007. "Overdependence on Technology: An Unintended Adverse Consequence of Computerized Provider Order Entry." AMIA ... Annual Symposium Proceedings 11: 94-98.

Colpaert, K., B. Claus, A. Somers, K. Vandewoude, H. Robays and J. Decruyenaere. 2006. "Impact of Computerized Physician Order Entry on Medication Prescription Errors in the Intensive Care Unit: A Controlled Cross-sectional Trial." Critical Care 10(1): R21.

Connolly, C. 2005, March 21. "Cedars-Sinai Doctors Cling to Pen and Paper." The Washington Post A01.

Crane, J. and F.G. Crane. 2006. "Preventing Medication Errors in Hospitals through a Systems Approach and Technological Innovation: A Prescription for 2010." Hospital Topics 84(4): 3-8.

Cutler, D.M., N.E. Feldman and J.R. Horwitz. 2005. "U.S. Adoption of Computerized Physician Order Entry Systems." Health Affairs 24(6): 1654-63.

Han, Y.Y., J.A. Carcillo, S.T. Venkataraman, R.S.B. Clark, R.S. Watson, T.C. Nguyen, H. Bayir and R.A. Orr. 2005. "Unexpected Increased Mortality after Implementation of a Commercially Sold Computerized Physician Order Entry System." Pediatrics 116(6): 1506-12.

Institute for Safe Medication Practices. 2007. "Smart Pumps Are Not Smart on Their Own." Medication Safety Alert! April.

Kaushal, R., K.G. Shojania and D.W. Bates. 2003. "Effects of Computerized Physician Order Entry and Clinical Decision Support Systems on Medication Safety: A Systematic Review." Archives of Internal Medicine 163(12): 1409-16.

Khajouei, R. and M.W. Jaspers. 2008. "CPOE System Design Aspects and Their Qualitative Effect on Usability." Studies in Health Technology and Informatics 136: 309-14.

Koppel, R., J.P. Metlay, A. Cohen, B. Abaluck, A.R. Localio, S.E. Kimmel and B.L. Strom. 2005. "Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors." Journal of the American Medical Association 293(10): 1197-203.

McAlearney, A.S., J. Vrontos Jr., P.J. Schneider, C.R. Curran, B.S. Czerwinski and C.A. Pedersen. 2007. "Strategic Work-Arounds to Accommodate New Technology: The Case of Smart Pumps in Hospital Care." Journal of Patient Safety 3(2): 75-81.

Moyer, P. 2005. "New 'Smart Pumps' Reducing IV Medication Errors." Drug Topics 149(16): 10s.

Murdoch, L.J. and V.L. Cameron. 2008. "Smart Infusion Technology: A Minimum Safety Standard for Intensive Care?" British Journal of Nursing 17(10): 630-36.

Nebeker, J.R., J.M. Hoffman, C.R. Weir, C.L. Bennett and J.F. Hurdle. 2005. "High Rates of Adverse Drug Events in a Highly Computerized Hospital." Archives of Internal Medicine 165(10): 1111-16.

Poissant, L., J. Pereira, R. Tamblyn and Y. Kawasumi. 2005. "The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review." Journal of the American Medical Informatics Association 12(5): 505-16.

Poon, E.G., J.L. Cina, W. Churchill, N. Patel, E. Featherstone, J.M. Rothschild, C.A. Keohane, A.D. Whittemore, D.W. Bates and T.K. Gandhi. 2006. "Medication Dispensing Errors and Potential Adverse Drug Events before and after Implementing Bar Code Technology in the Pharmacy." Annals of Internal Medicine 145(6): 426-34.

Potts, A.L., F.E. Barr, D.F. Gregory, L. Wright and N.R. Patel. 2004. "Computerized Physician Order Entry and Medication Errors in a Pediatric Critical Care Unit." Pediatrics 113(1 Pt. 1): 59-63.

Pratt, N. 2004. "Intravenous Medication Safety Wireless Data System." HealthLeaders 7(12 Suppl.): 39-44.

Rossos, P.G., H. Abrams, R. Wu and P. Bray. 2006. "Active Physician Participation Key to Smooth MOE/MAR Rollout." Healthcare Quarterly 10(Special No.): 56, 58-64.

Saathoff, A. 2005. "Human Factors Considerations Relevant to CPOE Implementations." Journal of Healthcare Information Management 19(3): 71-78.

Smetzer, J. 2002. "'Smart' Infusion Pumps Join CPOE, Bar Coding as Key Medication Error Prevention Measures." AHA News 38(5): 8.

U, D., S. Hyland, J. Greenall, J. Gosbee and S. Lane. 2007. Fluorouracil Incident Root Cause Analysis. Edmonton, AB: Cross Cancer Institute.

Vanderveen, T. 2007. "Smart Pumps: Advanced Capabilities and Continuous Quality Improvement." Patient Safety and Quality Healthcare Jan/Feb.

Vegoda, P. 2002. "Introducing the IHE (Integrating the Healthcare Enterprise) Concept." Journal of Healthcare Information Management 16(1): 22-24.

Vicente, K.J. 2004. The Human Factor (1st ed.). New York: Routledge.

Wolfstadt, J.I., J.H. Gurwitz, T.S. Field, M. Lee, S. Kalkar, W. Wu and P.A. Rochon. 2008. "The Effect of Computerized Physician Order Entry with Clinical Decision Support on the Rates of Adverse Drug Events: A Systematic Review." Journal of General Internal Medicine 23(4): 451-58.

Young, D. 2006. "IOM Advises CPOE, Other Technology for Preventing Medication Errors." American Journal of Health-System Pharmacy 63(17): 1578, 1580.