Adverse events involving medication use represent a significant patient safety issue in Canada. This was most recently identified through the findings of the Canadian Adverse Events Study, released in May 2004 (Baker et al.) One strategy for addressing this issue is to utilize a systems approach to patient safety rather than focusing on individual performance. Practitioners, however, need tools to assist them in identifying system weaknesses as well as guidance and direction for improvement. This paper describes the Canadian experience with such a tool; namely, the acute care hospital Medication Safety Self-Assessment™ (MSSA), which was designed to assist hospitals to identify areas of risk in their medication use systems.

The MSSA, originally developed by the Institute for Safe Medication Practices (ISMP) in the United States, was adapted for use in Canada in 2002 by ISMP Canada (with support from the Ontario Ministry of Health and Long-Term Care). The MSSA is a comprehensive survey tool for use by a multidisciplinary hospital team. The tool consists of 195 evaluative characteristics that serve to assess the safety of medication practices within the hospital and identify opportunities for improvement. Most of the characteristics represent system improvements ISMP and ISMP Canada have recommended in response to analysis of medication errors or problems identified during on-site consultations.

Survey Format and Methodology

The MSSA is divided into 10 key elements of safe medication use and then subdivided into 20 core characteristics (see Appendix 1). Each core characteristic section is made up of representative individual characteristics. Hospitals are asked to rate their compliance with each individual characteristic using the following scale:

A: No activity to implement this characteristic
B: Discussed, but not implemented
C: Partially implemented in some or all areas
D: Fully implemented in some areas
E: Fully implemented throughout

Each response is assigned a weighted score. The scores were developed by ISMP through an assessment of the impact on patient safety and the ability of the characteristic to ensure sustained improvement (Smetzer 2003.) The higher weighted score indicates a greater impact on the safety of the medication use system as a whole. Completion of the self-assessment requires a three- to five-hour commitment by a team of physicians, pharmacists, nurses and senior administrative staff. Once the completed survey has been submitted via the ISMP Canada website, individual users can compare their results to those of other respondents, on both a national aggregate and provincial/regional aggregate basis.

Figure 1 shows a sample comparison of one hospital to the national aggregate. The individual hospital's results are displayed as a bar graph with the national aggregate and standard deviation superimposed. Similar graphs can be obtained for comparison to provincial/regional data.

If more than one self-assessment is conducted and the data entered, hospitals are able to track their quality improvement efforts over time. Figure 2 shows a sample comparison of an individual hospital's results after two surveys.

Results and Interpretation

The provinces of British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Nova Scotia and Newfoundland and Labrador support the participation of their hospitals in the MSSA, while in other provinces participation is by individual hospital. At the time of writing, approximately one-third of Canadian hospitals (a total of 195) had completed the MSSA.

The MSSA data have provided insight into the status of medication use systems in Canadian hospitals. The average aggregate score for participating hospitals is 672.2 (or 55% of the achievable score of 1224). There is a substantial variation in scores, which range from 347 to 1039. Analysis of the responses generated three levels of results, broken down by key elements, core characteristics and individual characteristics. Only a portion of the key issues (items receiving the highest and lowest scores) will be highlighted in this paper.

Key Elements

Hospitals demonstrated the highest scores in areas related to the management of medication delivery devices, environmental factors and drug standardization, storage and distribution (Key Elements V, VI and VII). A nationwide MSSA survey completed in the US by ISMP in 2000 and published in 2003 found the same three key elements received the highest scores (Smetzer et al. 2003). Canadian scores were lowest in the key elements related to: patient information, communication of drug orders and other drug information, staff competency and education, and patient education (Key Elements I, III, VIII and IX), where the aggregate responses were between 40 and 50% of the achievable score. Comparison with the US survey results identified the same areas of low scores with the exception of staff competency and education (VIII). The aggregate scores by key element are shown in Figure 3.

Core Characteristics

Assessment of Canadian aggregate scores for the core characteristics, shown in Figure 4, indicates that only 9 of 20 core characteristics had an average aggregate result greater than 60% of the achievable score. Furthermore, wide ranges in responses indicate significant variability in the level of implementation of various medication safety strategies across the country.

The highest core characteristic score was related to sequestering of hazardous chemicals from patients and drug preparation areas. The average aggregate response indicated 85% of the achievable score. The next highest scoring was in the use of proven infection control practices in storage, preparation and administration of medications, with an average aggregate score of 80%.

The lowest core characteristic response was related to the availability of essential patient information, having an average aggregate score of 40% of that achievable score. Communication of drug orders in a standardized way, strategies for look-alike/sound-alike drug products, provision of ongoing education about medication error prevention, encouragement of practitioner reporting and multidisciplinary analysis of errors all demonstrated aggregate scores of less than 50% of the achievable score.

Comparisons of aggregate scores for core characteristics, by hospital demographics such as bed size, type and specialty, revealed very similar patterns of response, suggesting that hospitals of all sizes and types face similar challenges regarding their medication use systems.

Specific Characteristics

A review of aggregate scores for individual characteristics provides additional information about the status of medication use systems across the country. Some example scores for individual characteristics are noted below and demonstrate how individual hospitals can use the information to target improvements. The finding that characteristics related to management of error in a nonpunitive way showed average aggregate responses of 80% or greater is an encouraging result that suggests that a safety culture is becoming more evident, at least in participating hospitals.

A 90% average aggregate response was obtained for limiting the number of patient controlled analgesia (PCA) pumps to two or fewer within an institution and an 80% response for the development and implementation of monitoring criteria for PCA. Safety issues with PCA use and strategies for reducing the risk associated with administration of opioids by this route were addressed in several safety bulletins published by ISMP and ISMP Canada in 2003 and 2004.

A great deal of attention has been focused on removing potassium chloride concentrate from patient care areas, in response to several highly publicized deaths. The average aggregate response for this characteristic was 80%. Challenges continue to exist with the management of potassium chloride concentrate in paediatric and dialysis care areas.

Automatic screening of medication orders for patient allergies received an average aggregate response of 80% of achievable score. However, less than 20% of achievable score was obtained for the step of making patient allergies a mandatory field which must be filled in before orders can be entered. Mandatory entering of patient weights and a direct interface between the pharmacy and laboratory computer systems to automatically alert practitioners to the need for potential drug therapy changes also received aggregate scores of less than 20% of achievable scores.

Other findings showed that Canadian hospitals were lacking in implementing high leverage safety strategies such as bar coding for medication administration, computerized physician order entry (CPOE) and creation of designated medication safety positions. The average aggregate scores of these characteristics were less than 35% of achievable scores. Anecdotal follow-up by ISMP Canada suggested that high cost and complexity posting barriers to implementation of these technologies. On the other hand, there has been good acceptance of the importance of clinical pharmacist functions, with an average aggregate response of 70% for inpatient services and 45% for outpatient services. A recent study by Forster et al. (2004) reinforces the value of clinical pharmacist involvement in identifying and preventing adverse drug events.

Provincial MSSA Initiatives

In addition to the national aggregate responses, it is worthwhile to share some findings from the Ontario and the British Columbia provincial MSSA initiatives since these two provinces had a very high level of participation. Regional surveys were also conducted in Winnipeg and Halifax, and other new provincial projects will be completed in Alberta and Newfoundland during the fall of 2005.


Thirty-one Ontario hospitals completed an initial survey in 2002 plus a repeat survey in 2003 as part of a larger study protocol. The average aggregate score for these hospitals was 657 (53.7% of achievable score) in 2002 and 743.6 (60.8%) in 2003, demonstrating a relative improvement of 13.2%. Gains were achieved in 18 of 20 core characteristics, as illustrated in Figure 5. The total number of Ontario hospitals participating has increased to 75, which represent 39% of the Canadian aggregate. A comprehensive medication safety collaborative with the Ontario Ministry of Health and Long-Term Care raised the profile of medication safety, and might explain the higher participation rate.

The review of MSSA results for Ontario helped to identify a number of issues requiring intervention. The first intervention undertaken in Ontario in November 2002 focused on removing potassium chloride concentrate from patient care areas. This provincial safety initiative resulted in a significant increase in compliance with safe practice. As a result, similar initiatives were undertaken by other provinces. A second intervention, designed to improve the management of narcotic (opioid) medications, was initiated in 2004 and is still underway.

British Columbia

The Patient Safety Task Force of British Columbia (BC) invited 54 hospitals in their six regions to complete the Medication Safety Self-Assessment™ in 2004. Ninety-three percent of invited hospitals participated in the survey. ISMP Canada provided data analysis comparing results within and amongst the six regions. The BC aggregate score was 673, or 55% of the total achievable score, which, coincidentally, is identical to the current national aggregate score.

The following priority areas for action were identified based on review of MSSA results for BC:

  • Manufacturer labelling/packaging and look-alike/sound-alike drug names
  • Provision of ongoing safe medication education for practitioners
  • Active analysis of errors for system redesign

A follow-up survey of the BC hospitals will be conducted in early 2006.


There are limitations to the interpretation of these Medication Safety Self-Assessment™ results. The sample size, although representing approximately one-third of Canadian hospitals, is still small and thus may not be generalizable. As no statistical analysis has been performed, the confidence interval and significance of data differences have not been determined. The goal of this paper was to provide an overview of some of the Canadian data and demonstrate the value of the tool for assessing risk issues and developing priorities for individual hospitals and for provinces and regions. The tool is not designed for individual hospitals to make success comparisons with their peers. Rather, it is intended to allow hospitals to assess their medication use system weaknesses and to contribute to an aggregate database to assist in determining the areas of the medication use process that require more effort for improvement.

It is understandable that some may challenge the scientific validity of the safe practices contained in the Medication Safety Self-Assessment™. But while the characteristics contained in this tool are not proven by formal research methodology, it has been argued that many medication safety practices are "common sense" and well supported by human factors literature in other industries (Leape et al. 2002). The tool has been well accepted by Canadian hospitals and has been referenced within the guidelines to the 2005 Canadian Council on Health Services Accreditation Standards. Internationally, 1,435 hospitals in the United States participated in a national survey in 2000 and over 1,600 in a repeat survey in 2004 (Smetzer et al. 2003; ISMP Alert 2005). The State of New South Wales in Australia has recently received approval and funding to adapt and implement an Australian version of the MSSA. ISMP (US) has also developed a community practice version, currently being modified for use in Ontario. A long-term care version is in the development phase in Canada.

The MSSA offers a comprehensive structured process for assessing the safety of a hospital's medication use system in a manner that is proactive, unbiased and encourages consensus building. It provides a mechanism to enhance the perspective of healthcare practitioners towards a system-based approach to preventing adverse events. The ISMP Canada web-based access feature allows for an overview of system issues from provincial and national perspectives, which can be used to develop provincial and national priorities for safe medication practices.

[To view the Appendix, please download the PDF.]


Baker, G.R., P.G. Norton, V. Flintoft, R. Blais, A. Brown, J. Cox, E. Etchells, W.A. Ghali , P. Hebert, S.R. Majumdar, M. O'Beirne , L. Palacios-Derflingher, R.J. Reid, S. Sheps and R. Tamblyn. 2004. "The Canadian Adverse Events Study: The Incidence of Adverse Events Among Hospital Patients in Canada." CMAJ 170(11): 1678-86.

Forster, A.J., R.B. Halil and M.G. Tierney. 2004. "Pharmacist Surveillance of Adverse Drug Events." Am. J. Health. Syst. Pharm. Jul 15 61(14): 1466-72.

"High Alert Drugs and Infusion Pumps: Extra Precautions Required." 2004. ISMP Canada Safety Bulletin 4(4).

"Latent Failures Identified with CADD® Pump Error Analysis." 2004. ISMP Canada Safety Bulletin 4(9).

Leape, L.L., D.M. Berwick and D.W. Bates. 2002. "What Practices Will Most Improve Safety? Evidence-based Medicine Meets Patient Safety." JAMA 288(4): 501-7.

"Looking Forward: Making 'Pro-change' Your New Year's Resolution." 2005. ISMP Medication Safety Alert! 10(1).

"Misprogram a PCA Pump? It's Easy!" 2004. ISMP Medication Safety Alert! 9(15).

"Safety Issues with Patient-controlled Analgesia, Part I." 2003. ISMP Medication Safety Alert! 8(14).

"Safety Issues with Patient Controlled Analgesia: Part II - How to Prevent Errors. " 2003. ISMP Medication Safety Alert! 8(15).

Smetzer, J.L., A.J. Vaida, M.R. Cohen, D. Tranum, M.A. Pittman and C.W. Armstrong. 2003. "Findings from the ISMP Medication Safety Self-Assessment for Hospitals." Jt. Comm. J. Qual. Saf. 29(11): 586-97.