In 2004, the Ministry of Health in British Columbia established the BC Patient Safety Task Force to improve patient safety throughout the province. One undertaking of the task force was the creation of the BC Patient Safety and Learning System (PSLS), a web-based adverse event reporting system that facilitates system-level learning and improvement. PSLS began as a grassroots initiative of the provincial risk management community and continues as a collaborative effort of British Columbia's health authorities and their insurer, with leadership now provided by the newly formed BC Patient Safety and Quality Council.

The ultimate goal of PSLS is to make healthcare safer. In the short term, desired outcomes include changes in the awareness, attitudes, skills, opinions and motivations of staff and leaders regarding patient safety. Medium-term outcomes include observable improvements in reporting and learning practices. In the long term, PSLS aims for better information sharing, reduced costs associated with adverse events, an increased use of evidence to guide decision-making and a culture of safety.

Prior to a large-scale rollout of PSLS, we undertook a pilot study to provide proof of concept, configure software, develop a training approach and demonstrate that technology implementations can support and enable cultural shift. Our pilot site was the neonatal intensive care unit (NICU) at BC Women's Hospital in Vancouver, part of the Provincial Health Services Authority (PHSA). The NICU is a level IV tertiary nursery for extremely ill and premature infants. The intensive, often invasive care required by these babies, combined with their profound fragility, creates a context where adverse events are relatively likely. The NICU was staffed by nearly 300 people: nurses, physicians, medical trainees, respiratory therapists and other clinicians and specialists. The hospital did not use a clinical information system, but staff used computers in their daily work.

Planning

Safety reporting systems cannot be effective if staff do not use them. Typically, as few as 5% of adverse events are formally reported, and near-miss events are seldom brought to the attention of leaders (Leape 1994). Reporting rates may be improved by removing practical barriers through computerization; but if no other changes are made, few gains will be seen in learning and benefit to patients. The goal of our pilot study was to identify strategies and conditions that would promote the adoption of PSLS and encourage reporting and learning behaviours in personnel. We aimed to foster a culture that would inspire individuals to report safety events and near misses and to use data to learn and improve (McCarthy and Blumenthal 2006). Our target audiences were front-line workers who report events, managers who follow up on reports, leaders who investigate serious incidents and executives who receive summarized data.

Our literature review included World Health Organization (WHO) guidelines for organizations considering implementing reporting systems (World Alliance for Patient Safety 2005), which we used to validate our concept. We also considered models, tools and implementation methods used by healthcare adverse event reporting systems in other countries, including Australia, the United States and the United Kingdom, and we consulted with the leaders of several systems to learn from their experiences. Although PSLS is provincial in scope, we found many aspects of the United Kingdom's National Reporting and Learning System to be applicable to our context. This national system's reports, taxonomy and learnings, which were readily available through the National Patient Safety Agency, were used to inform our implementation planning and system configuration. However, there were some key differences between PSLS and most other large systems. Most notable was our aim to use one shared tool and database across British Columbia, offering the system directly to health authority reporters and leaders to support event reporting, follow-up and local improvement efforts, and concurrently make comprehensive data available at the provincial level for greater analysis and learning.

Focus Groups

To assess perceptions of NICU staff regarding safety and reporting, we conducted three multidisciplinary focus group interviews in March 2006. Participants identified fear of retribution, confusion about what to report and why, a lack of feedback or evidence of action taken following reporting (the "black hole" phenomenon) and the inconvenience of paper forms as barriers to reporting. Similar observations have been made by others (Bourn 2005; Evans et al. 2006; Harper and Helmreich 2005; Kingston et al. 2004; Lawton and Parker 2002; National Patient Safety Agency 2005).

Survey

In April 2006, we administered the Hospital Survey on Patient Safety Culture (Westat 2004) to NICU personnel, with responses from 198 of 272 (73%) potential participants. Strengths were in the dimensions "teamwork within the unit" and "manager actions promoting safety." Areas for attention and improvement included "overall perception of patient safety" and "staffing." Indicators regarding reporting, such as lack of feedback and concerns about punishment, were consistent with barriers identified by the focus groups.

Operational Indicators

To gauge baseline reporting practices, we measured report volume, completeness, data quality and reporter characteristics. In February 2006, we began a three-month audit of all 616 incident reports submitted from across Children's and Women's Health Centre of British Columbia. Most (91%) were incomplete or contained errors. On average, reports took 26 days to arrive at the Quality, Safety and Risk Management Department (QSRM). The average length of follow-up investigation was 51 days. All 402 follow-up reports received during the audit period were rated for quality on a scale of zero (lowest) to four (highest) by assigning one point for evidence of each of the following: systems thinking, root-cause analysis, identification of contributing factors and changes made or suggested to prevent recurrence. Most (67%) received a score of zero or one.

The local evidence confirmed that there was room for improvement in reporting practices. Staff needed better reporting tools and safety education. Leaders needed new processes and skills to follow up on and manage issues.

Reporting and Learning Tool

PSLS offers a web-based event reporting, analysis and management tool that supports continuous system improvement (Figure 1). Events can be reported (anonymously, if desired) from any network computer, and data are stored in a secure central database. Details include severity, location and people affected. PSLS allows follow-up responsibilities for individual events to be assigned to specific people via automatic e-mail notification, and it facilitates communication and monitoring of follow-up activities. Reporting spans a wide scope of events, classified into three types: (1) safety hazards - unsafe circumstances that could potentially cause harm; (2) near misses - events that could have caused harm to a specific patient but were caught in time; and (3) adverse events - unexpected and undesired outcomes associated with the care of a patient.

[Figure 1]

Our Approach

Our pilot team included executive leaders to ensure high-level, visible support and alignment with organizational priorities and directives, such as a new non-punitive reporting policy enacted by the PHSA Board of Directors. Three project co-directors collaboratively provided project management, business/clinical and technical perspectives and leadership. QSRM staff offered expertise in quality improvement and event management, and seconded pilot site nurse leaders acted as "cultural translators" to help the pilot team appreciate the realities of work in the NICU. Information technology (IT) specialists who focused on system and software configuration were co-located with the pilot team. Our approach to project management incorporated slack to accommodate shifting pace and emerging findings plus both top-down and bottom-up interventions, as has been recommended for complex projects (Ivory and Alderman 2005). We based our project framework on the Comprehensive Unit-Based Safety Program (CUSP) developed at Johns Hopkins Hospital (Johns Hopkins Hospital 2003; Pronovost et al. 2005). CUSP offers a staged approach to engaging staff and leaders in safety by assessing safety culture, educating them about safety, identifying and handling safety issues, making improvements and sharing stories.

We combined CUSP with a robust change management plan, which began by identifying and assessing stakeholders according to interest, impact and influence. We then developed detailed stakeholder engagement and communication plans, which employed many channels - posters, newsletters, e-mails and a website - and used existing meetings to deliver tailored and targeted safety messages to executive leaders, managers, physicians and front-line staff.

We assessed the readiness of the pilot site staff for the changes that would be brought about through the implementation of PSLS by examining roles, existing safety practices and computer use. We developed a training plan based on our findings and the results of the planning stage. We also created a comprehensive evaluation plan and tools so that we could measure and clearly demonstrate results.

Training

Training efforts involved the entire pilot team (Figure 2) and used the introduction of PSLS to catalyze discussions about patient safety, reporting and learning. Prior to Go Live (system initiation), we trained managers on event follow-up by holding four-hour, interactive, small-group sessions in a computer classroom using a training version of PSLS. Session leaders reinforced the importance of coaching and giving feedback to reporters, and provided a manual, quick reference card and access to a QSRM Help Desk for support, feedback and help. Afterward, managers accessed the production environment, which had been pre-loaded with data from paper incident report forms, to apply their learning.

[Figure 2]

Front-line staff training began with Go Live. Pilot team members gathered small groups around computers in the NICU for 20-minute sessions and engaged staff in discussions about patient safety, emphasizing reasons for reporting, what to report, how to report and what to expect after reporting; they left a quick reference card at each computer. Staff then used safety event scenarios to enter reports using the online training environment. Pilot team members reviewed the practice reports and rewarded reporters with small calculators. The focus of education was on patient safety, but teaching staff to use the new reporting form was also critical.

Training sessions were formally evaluated. Eleven managers evaluated the classroom sessions and indicated they were satisfied or very satisfied with the approach and their preparedness to use PSLS. Thirty-two front-line staff evaluated the effectiveness of the short learning sessions and practice report approach, with 100% answering "yes" to the question, "Now that you have completed your BC PSLS education session/practice scenarios, do you feel you could properly complete and submit a safety event form?"

Pilot Evaluation

Survey

In March/April 2007, PHSA conducted a health authority-wide survey using the Hospital Survey on Patient Safety Culture. The NICU response rate was 45% (123 respondents). Although we had not yet implemented PSLS in the pilot site, we compared the two sets of survey results to assess the impact of our engagement of NICU personnel in preparing for the launch of PSLS. The most significant areas of improvement were in "teamwork across units" and "staffing." There was a decline in the "supervisor/manager expectations and actions promoting patient safety" dimension, which may have reflected significant restructuring and changes to leadership roles prior to and during the survey. We also compared the NICU survey results to those of the rest of BC Women's Hospital and found that the NICU scores were higher in most domains.

Operational Indicators

At the conclusion of the three-month pilot, we compared reporting and follow-up indicators before and during the pilot to gauge the ability of PSLS and our implementation approach to enable a reporting culture (Table 1). There was significant improvement in all areas. The number of reports submitted was much greater during the pilot, and a larger proportion were near misses or hazards. The electronic system allowed reports to be submitted to QSRM more quickly and greatly reduced notification time. Mandatory fields in the electronic form ensured all reports submitted were complete. Finally, there was a significant improvement in the quality of follow-up.

Discussion

The results of the pilot study were promising. Indicators showed an increased willingness of staff to report on all types of events. Greater participation by non-nurses showed that the training approach and communications strategies were effective in engaging other disciplines and encouraging them to take a more active role in safety.

Improvement in the quality of follow-up may be the best indication that PSLS and our implementation approach can accelerate and enhance learning and quality improvement. PSLS enabled managers to respond to events quickly and supported concurrent participation in follow-up and communication between people across the organization. Managers were more likely to determine contributing factors and causes because they were able to engage staff in discussing events while details were easier to recall, invite them to participate in problem solving and track recommendations and changes.

Despite improvements in follow-up quality, the time taken for review showed only slight improvement due to two factors. First, the NICU was extremely busy with a consistently high census and acuity, so much of the managers' time was spent in clinical support work, which limited opportunities to perform non-urgent, non-clinical duties. Second, the change in practice required of managers was greater than anticipated. With the previous paper-based system, managers waited for several reports to accumulate and then used pre-scheduled "office days" to complete follow-up paperwork. PSLS allowed immediate notification of events but required periods of uninterrupted time for managers to complete review activities, which was at odds with the demands and hectic pace of the NICU.

Although the pilot period was short, PSLS did create some opportunities for quality improvement. One example involved the implementation of a computerized medication recording system. With the help of staff reporting, PSLS allowed the quick identification of a number of medication events related to transcription errors, provided evidence of the problem and thereby enabled a quick response from pharmacy and the NICU to implement change.

The pilot provided only limited data, impacting our ability to formulate and generate aggregate reports. Input from pilot site leaders about meaningful reports was, and continues to be, valuable as we evolve this component.

In general, staff in the NICU had adequate access to computers to facilitate their use of PSLS, but this is not the case in all sectors of the BC healthcare system. Other methods of reporting need to be explored if PSLS is to be accessible in all care settings.

Lessons Learned

The pilot study laid the foundation for the provincial implementation of PSLS. A successful pilot was critical to establishing the infrastructure for the provincial system and obtaining support for implementation. Here we outline some lessons learned that have informed that rollout and that may be of interest to those planning similar initiatives.

Change Management

CUSP provided an effective basis for sequencing the steps of our change management framework. Based on our experience, we have identified some additional refinements, facilitators and success factors. We believe the primary reason for the outstanding adoption of PSLS during our pilot was our use of the implementation as a vehicle to engage staff in discussions about patient safety. We used communication forums and training sessions to increase staff awareness and to encourage them to take an active role in safety.

PSLS reinforced reporting behaviours by providing managers and others with immediate notification of reports and the means to help them "close the loop" by providing feedback to reporters. The system helped cultivate a mutual trust that staff would be attentive to hazards and report safety concerns and managers would appreciate staff efforts and follow up as needed. When leaders demonstrated commitment to safety by responding and taking visible action, staff were encouraged to sustain reporting behaviours.

Integrating pilot site staff into the pilot team was essential. These "embedded" representatives were able to influence practice and effect change in the pilot site by using their existing relationships and local knowledge to engage pilot site staff (Reay et al. 2006). Pilot site staff were in the best position to identify both problems and solutions and were often keen to be part of the project. Giving them opportunities to participate in activities such as form design recognized their important contribution, ensured the project remained relevant to the clinical program and promoted sustainable change.

Project Management

We found that a rigid, linear project management approach did not fit with the iterative nature of this project. Focusing on project control and adhering to predetermined sets of tasks and timelines did not allow the flexibility required to respond easily to emerging issues. Quality improvement project methods and tools combined with thoughtful change management offered a more nimble framework.

Co-location of IT specialists with the pilot team allowed them to develop a thorough understanding of the business drivers for the project and its connection to the work of front-line care providers. The other members of the team also benefited, developing a better understanding of the possibilities and limitations of the software, the importance of configuration and system set-up to user adoption and the work involved in system maintenance. We have maintained this co-location approach for the provincial rollout to facilitate iteration and a high level of responsiveness to feedback; the administrative, business, technical and project management implications of issues can be addressed simultaneously and with the benefit of complementary perspectives.

Conclusion

The pilot study allowed us to develop and demonstrate a successful approach to the implementation of PSLS within our NICU pilot site, most importantly by using the system implementation as an opportunity to foster safety culture. Lessons learned have informed the direction we are now taking with the large-scale rollout of PSLS in British Columbia. The diverse, collaborative project team, flexible project management approach, robust change management plan, safety-focused training methods and commitment to evaluation established through the pilot have served us well with our implementations in four of the six BC health authorities to date. The pilot highlighted the significant changes that managers need to make during follow-up; this challenge continues to be a key focus during the rollout.

In addition to validating our implementation approach, the pilot provided proof of concept for PSLS. We will continue to build on the experiences of other systems, such as the UK National Reporting and Learning System, and to incorporate important work in progress, such as the WHO International Classification for Patient Safety, the Agency for Healthcare Research and Quality Common Formats and the Canadian Medication Incident Reporting and Prevention System.

The short duration of the pilot study did not allow us to develop comprehensive, robust aggregate reports; this work is now a priority. Providing regular, timely, meaningful and actionable information using data from the learning system is critical to promoting ongoing adoption and is necessary to show the value of PSLS and its ability to support improvements in patient safety and care.

Comments

Be the first to comment on this!