The Manchester Patient Safety Culture Assessment Tool (MaPSCAT) was used to examine the levels of safety culture maturity in four programs across one large healthcare organization. The MaPSCAT is based on a theoretical framework that was developed in the United Kingdom through extensive literature reviews and expert input. It provides a view of safety culture on 10 dimensions (continuous improvement, priority given to safety, system errors and individual responsibility, recording incidents, evaluating incidents, learning and effecting change, communication, personnel management, staff education and teamwork) at five progressive levels of safety maturity. These levels are pathological ("Why waste our time on safety?"), reactive ("We do something when we have an incident"), bureaucratic ("We have systems in place to manage safety"), proactive ("We are always on alert for risks") and generative ("Risk management is an integral part of everything we do"). This article highlights the use of a new tool, the results of a study completed with this tool and how the results can be used to advance safety culture.

The measurement of patient safety culture has been a top priority for many healthcare organizations across Canada. This interest stems partly from the fact that it is a requirement of Accreditation Canada, but it also because leaders have understood the importance of examining the underlying values that drive staff behaviour in relation to patient safety. These behaviours include such things as reporting adverse events, working as a team and making decisions that consider and optimize patient safety at all points of care. The quest for measurement of patient safety culture has led to the development of numerous tools that differ in their theoretical underpinning, origins and applications. The current study seeks to contribute to this knowledge base by providing an overview of a new measurement tool titled the Manchester Patient Safety Culture Assessment Tool (MaPSCAT) and detailing how this tool has been used in an acute care setting to gain valuable insights into the patient safety culture.

The Manchester Patient Safety Culture Assessment Tool

In order to improve safety culture, it is essential to base changes on a framework of safety culture that takes into account the multi-dimensional nature of the concept (Hale 2000). In line with this idea, Parker et al. (2006a) looked to the theoretical typology of organization culture based on James Reason's (1997) adaption of the Westrum (1996) model. This typology distinguishes between cultures based on how information is handled, and identifies three different levels of organizational culture – pathological, bureaucratic and generative. In addition to detailing the style of information processing in a unit, the typology references the role of leaders who shape the unit's culture through their symbolic actions and provide rewards and punishments that communicate what they feel is important; these then influence the views of the workforce (Westrum 2004). Westrum (2004) suggests that good information flow and processing has important effects on patient safety (such as good teamwork), and that an open and generative culture means a better uptake of innovations and response to danger signals.

Parker et al. (2006a) first adapted this framework for an empirical study in the petroleum industry, extending the number of levels of safety culture to five and applying them to a range of dimensions. This resulted in a normative framework identifying "good" or "bad" safety cultures and illustrating how safety culture could be improved. The framework also facilitated the comparison of organizational cultures and subcultures (Lawrie et al. 2006). This work was then expanded to the healthcare field with the development of the Manchester Patient Safety Framework. This framework was developed through extensive reviews of the literature in healthcare and consultations with experts in the field. It was tested with healthcare professionals and formulated into a research tool, MaPSCAT. The MaPSCAT is the result of collaboration between researchers in the United Kingdom and Canada who were interested in developing a patient safety culture tool that is rooted in acute care and based on the Manchester Patient Safety Framework. The 10 different dimensions of safety culture used in this tool are outlined in Table 1. Within these 10 dimensions, statements were developed to reflect five increasingly mature levels of safety culture. The levels of safety maturity range from pathological through reactive, bureaucratic and proactive and, finally, to generative. At the lowest level of safety culture, pathological refers to "why do we need to waste our time on patient safety," next reactive refers to taking patient safety seriously once an event has occurred. The bureaucratic level of culture refers to having systems in place to deal with patient safety issues, and then proactive is when the organization is alert and thinking about patient safety issues that might occur. At the generative level of safety culture maturity patient safety is seen as an integral part of everything that the organization engages in (NPSA, 2006).

This tool was tested and validated in acute care settings in Canada and the United Kingdom before the implementation in the current study (Law et al. 2008, June). This validation process resulted in the modification, retesting and finalizing of 24 questions for the survey. Some were dropped due to a lack of agreement in the ranking. Therefore, there are one, two or three questions per dimension that are calculated together to create a result for that safety dimension.

The MaPSCAT advances the research in safety culture measurement as it (1) measures 10 dimensions of safety culture, (2) examines these dimensions on a safety maturity scale, (3) aggregates scores to create a safety culture profile and (4) provides guiding statements on how to improve the safety culture.

Patient Safety Culture Dimensions

In a number of reviews of culture assessment tools, it has been found that there is variation in the types and number of safety culture dimensions that are encompassed in the tools (Colla et al. 2005; Fleming and Hartnell 2007; Flin et al. 2006; Pronovost and Sexton 2005). What can be gleaned from these reviews are some common categories of communication and reporting and recording of events. Three of the reviews identified dimensions of leadership, safety systems, teamwork and values and beliefs about safety and teamwork. Learning and individual factors such as personnel resources and job satisfaction were also highlighted in two reviews. Added to these, the MaPSCAT includes the dimensions of incident evaluation and continuous improvement. Previous tools and their dimensions stemmed mostly from the United States and were rooted in high-reliability theory. Yet, the structures and operations of healthcare systems vary across countries. Thus, a tool designed in one system may not have the same relevance in other national systems (Waterson et al. 2010), so a tool developed and validated in a number of countries helps to increase confidence in using the tool. As well, it has been proposed that a patient safety culture measurement tool designed for use in acute care should be rooted in the customs and practices of acute care in order to be applied to this setting (Parker et al. 2006b).

Multi-dimensional Approach

This idea of a multi-dimensional approach to assessing safety culture is not new (Fleming and Wentzell 2008), but its implementation as an organizational survey is. Fleming and Wentzell (2008) provide details of the Patient Safety Culture Improvement Tool, which was envisioned to be used by hospital teams to identify and discuss specific cultural issues. However, reliability and validity are pending the further testing of this tool. The MaPSCAT was developed through an extensive study that involved healthcare professionals ranking the safety statements, re-working problematic statements and dropping certain statements, followed by retesting (Law et al. 2008). This study has aided in establishing the validity of the instrument, although the MaPSCAT will still require further psychometric testing such a factor analysis following the collection of additional data.

This multi-dimensional approach in the MaPSCAT has respondents read a series of statements about a dimension that reflect the various levels of safety maturity. Then respondents must choose the statement that best reflects their culture. Table 2 provides one series of statements posed to participants.

The tool requires the participants to read all of the statements before determining which one they will select. The statements are not set up in a logical progression of the safety culture maturity levels. Most safety culture assessment tools provide specific statements to participants and ask them to rate these on a scale of agree or disagree (Singer et al. 2003). Anecdotal evidence through the course of this work has pointed to some participants' preference for the MaPSCAT format in which "they had to really think about their answers" instead of simply putting a check mark beside a list of answers.

Patient Safety Culture Profile

Another unique and useful feature of this tool is that individual responses for each of the 24 questions can be aggregated indicating the unit level of safety culture for 10 dimensions. The perceived levels of safety culture for each of the components form a profile of safety culture that portrays how the organization is doing on that specific dimension of safety.

Utility for Creating Change

Finally, given that this tool is directly based on a theoretical model of safety culture maturity, organizations receiving their safety culture profiles can refer back to the model and understand what higher levels of culture in specific dimensions might look like. For example, if an organization received a rating at the bureaucratic level of learning and change, the members could refer to the framework and see that in order to move to the proactive and or generative level, they should focus their efforts on engaging staff and patients in the investigation and learning about safety events. Thus, this tool not only provides scores on the culture but also offers a basis from which to initiate efforts for cultural changes.

Overall, the MaPSCAT is a new and unique way in which to measure patient safety culture. In order to demonstrate the utility and implementation of this tool, the following study provides details of a pilot project conducted at Hamilton Health Sciences (HHS) using this measurement tool.

Research Methodology

The pilot study employed survey methods using the MaPSCAT quantitative tool. Program directors across HHS were contacted by the assistant vice-president of quality, patient safety and clinical resource management and asked if they would consider being involved in this research project; five directors initially agreed to participate. Due to program realignments, one director had to later withdraw the program.

Site contacts at each of the programs were provided with surveys in individual envelopes. The study participants were asked to read the consent, fill out the questionnaire and return it in the sealed envelope. All of the questionnaires were returned to the principal investigator via the site contacts. In order to enhance the response rate, two sets of communications were conducted in each of the programs. First, as the surveys went out, individuals were sent an e-mail outlining the study and asking them to fill out the questionnaire. At one week before the deadline for returning the surveys, staff were sent another communication to ask them to fill out the survey and return it to the site contact. The inclusion criteria for individuals filling out the surveys was as follows: (1) participants must have responsibilities within the unit that are associated with patient care (i.e., managers, physician, nurses, technicians, allied health professionals and other support staff); (2) participants must be able to read English (the questionnaires has not been translated into other languages); and (3) participants must have completed their probationary period in their current position since it was felt that such experience was needed to have an adequate understanding of safety culture issues in the unit.

Results

Study Participants: Response Rate

A total of 360 surveys were given out across the programs. The response rates ranged from 33 to 85% within the four programs, with an overall response rate of 45.3% (previous research reported response rates between 26 and 91% [Flin et al. 2006]). Therefore, the data are reflective of 163 HHS staff from four programs.

Nursing respondents had the greatest representation in the survey (63%); there was low representation from physicians, with only 2.5% of the questionnaires filled out by this group. The remaining respondents included allied health, technicians, educators, managers and support staff.

Survey Results

For each of the 10 components on the survey, 10 graphs were made to depict the ratings based on the percentage of responses at that level of safety maturity. The graph in Figure 1 demonstrates results for the dimension of teamwork.

Although the graphs such as the one presented in Figure 1 provide an excellent overview of the results for each dimension separately, one graph representing results in all dimensions aids in the comprehensive view of the results (Figure 2).

One of the main questions for researchers and decision-makers alike is deciding which level to discuss and highlight in the results and, for the applied aspect, where to focus strategies to enhance the patient safety culture. Fleming and Meakin (2004) propose one solution to this dilemma: they suggest that one should select the highest level, where 66% of participants select that level or a level above. For example, in Figure 2, the result for teamwork would be considered proactive as more than 66% of respondents indicated proactive or higher. It was outlined that this approach may provide confidence that this is the minimum level achieved. Therefore, the following summary reflects this framing of how to understand these results.

Summary by Level

Proactive and Generative Culture

A generative culture is seen as the highest level of safety, where the management of "safety is an integral part of everything we do," and a proactive culture is seen as one in which "we are always on alert and thinking about patient safety issues." Respondents rated priority given to safety (73.01%), evaluating incidents (68.42%) and teamwork (71.05%) at the proactive level or higher (see Figure 2). This suggests that staff believe that safety is taken very seriously by the organization and in their own daily work. As well, respondents believe that the organization has a strong teamwork environment that is also focused on evaluating incidents. It should be noted that the dimension of system error and individual responsibility was within 2.24% of reaching this level, and the recording incidents dimension was within 0.16% of this rating level.

Bureaucratic Culture or Higher

A bureaucratic culture is defined as one in which "we have systems in place to manage safety." The bureaucratic level or higher reflects the majority of the responses (see Figure 2), with six of the 10 dimensions being evaluated in this way: commitment to continuous improvement (82.4%), system error and individual responsibility (96.26%), recording incidents and best practices (85.84%), learning and effecting change (88.29%), communications about safety (72.56%) and staff training and education (82.34%). In relation to all of these dimensions, a broad statement can be made that emphasizes the fact that this organization has gone to great lengths to ensure a framework for safety is in place and that there are policies and procedures available for patient safety. However, there are further opportunities to enhance the implementation of these patient safety practices to improve the overall patient safety culture.

Reactive Culture or Higher

A reactive culture is defined as one in which "we do something when there is an event," and this was the rating in the dimension of personnel management for safety (84.41%; see Figure 2). This may reflect the fact that some individuals feel that staff support for patient safety is minimal, and this is reflected is staff behaviour.

Discussion

The results provided by this tool allow decision-makers to understand where they are doing well, and to celebrate these successes, as well as where there remain opportunities to enhance the safety culture. This information is conveyed through dimension summaries and graphic profiles that link to an overarching framework for safety maturity. The questionnaire provides a summary of safety culture dimensions, versus a copious amount of information from a large number of individual survey questions. With the MaPSCAT, decision-makers can examine their scores at these levels and refer back to the framework to see what types of statements and actions are aligned to higher levels of culture. It is important to provide results in a way that will ensure their uptake (Goering et al. 2003); this format may enhance decision-makers' ability to do so. This unique way of studying and presenting the results may make MaPSCAT more appealing to decision-makers than previous tools as MaPSCAT helps to provide ideas and direction for moving the culture forward.

Conclusions

It is evident that staff at HHS perceive there to be a high priority given to safety, an appropriate focus on evaluating incidents and a great teamwork environment in their organization. The results indicate that, in six other dimensions, the organization has taken steps to move the culture forward and is committed to the overall safety agenda through the development of a framework with policies and programs to enhance safety. However, more work is needed to further embed the values and behaviours that they would like to achieve to move toward higher levels of safety culture.

Implications

This is the first Canadian study using the MaPSCAT, and it was met with great interest and positive response at the leadership level. Leaders outlined that the summaries on the specific dimensions were helpful in allowing them to identify target areas of improvement. For example, they felt they would be able to reflect and then conceptualize future directions in the area of learning and change at a broad level as compared with simply focusing in on specific survey questions. Further research is needed to determine the extent to which this format will aid in the initiation of safety culture change efforts.

It is also important to recognize that safety culture responses vary if subcultures exist (Schien 1996), and to factor that in the design of culture research (Fleming and Wentzell 2008). Although the results were combined, there could be considerable variation by program or by health professional. Further analysis may reveal additional information from which to gauge program specific improvements. This level of segmentation in the initiation stages of the data analysis would help to address results within the local context and to assist in targeting change. The MaPSCAT appears to be a promising alternative for measuring patient safety culture in acute care, although further research and application are needed.

Comments

Be the first to comment on this!