Abstract

In 2008, University Health Network was surveyed using Accreditation Canada's new Qmentum program. The following article describes UHN's experience rolling out the program to over 12,000 staff, physicians and volunteers. The article also outlines key challenges and lessons learned by the multi-site organization, with a focus on staff engagement, on-site survey preparation and sustainability moving forward. Staff feedback on the Qmentum program was extremely positive, and forecast results from Accreditation Canada were excellent.

Background

University Health Network (UHN) is one of Canada's largest academic health sciences centres, operating three major hospitals in downtown Toronto: Toronto General Hospital (TGH), Toronto Western Hospital (TWH) and Princess Margaret Hospital (PMH). UHN employs 10,000 full- and part-time staff, over 1,200 full- and part-time physicians and 1,900 volunteers. Each year, the hospital provides care for approximately one million patients. In addition to its role as one of the country's leading research institutions, UHN is a teaching hospital affiliated with the University of Toronto.

In its commitment to quality, Accreditation Canada routinely reviews and revises its programs to ensure components address the needs of the ever-changing healthcare environment. As a result, Accreditation Canada transitioned from its successful Achieving Improved Measurement (AIM) program to Qmentum, with the underlying concept of moving quality forward with momentum. Key program elements of AIM and Qmentum are outlined in Table 1. After conducting several successful pilots in 2007, Accreditation Canada slated organizations being surveyed in 2008 to implement the Qmentum program; UHN was one of these organizations. One prominent feature of Qmentum is the use of a tracer methodology to assess compliance to national standards of excellence. Tracers are interactive, involving four components: reviewing client files and documents; talking and listening to patients, families and staff; directly observing client care and tours; and recording what is read, heard or observed. The immediate benefit of this approach is that it engages and promotes participation by front-line staff, patients and families in the accreditation process.

 

Table 1. AIM and Qmentum program elements
AIM Program Elements Qmentum Program Elements
Standards New or revised standards
Self-assessment against national standards Questionnaires to assess compliance to national standards
  Instruments: Patient Safety Culture Survey Governance Functioning Tool Worklife Pulse Tool
  Required organizational practices
  Indicators
  Quality performance roadmaps
On-site documentation On-site documentation
On-site survey (main evaluation method = interviews) On-site survey (main evaluation method = tracers) Minimal interviews
Award decision and report Award decision and report (includes some ongoing elements throughout 36-month cycle)
AIM = Achieving Improved Measurement.

 

Methodology

Preparations for the Qmentum implementation at UHN began in early 2007 by outlining the new program to UHN senior management, including an initial discussion of compliance to the 25 required organizational practices (ROPs). The accreditation lead also developed a preliminary project plan and recruited a team in November 2007. The UHN Accreditation Resource Team (ART) was composed of a diverse group of individuals from a variety of healthcare backgrounds (medical imaging, nursing and epidemiology). The team included an accreditation lead, project manager, specialist and consultant (amounting to three full-time equivalent positions). The team came together in January 2008 and began rolling out Qmentum using a three-phase approach, outlined in Table 2.

 

Table 2. Accreditation phases in rollout of Qmentum at UHN
Accreditation Phases Key Elements
Phase 1: January-April 2008 Finalization of UHN accreditation teams and associated standards
Population of Accreditation Canada client organization portal (team information)
Creation of communication plan with UHN public affairs
Qmentum kickoff presentations for all UHN staff
Presentations to board, senior management, programs, teams, departments, units
Population of accreditation intranet site
Establishment of UHN Operations Committee as the accreditation steering committee. Monthly progress reports and dialogue with the ART. Launch of monthly accreditation status report, shared with UHN Operations and Senior Management Committees
Assignment of UHN operations leads for 2005 accreditation recommendations and 2008 ROPs:
  • Assessment of compliance status, including evidence
  • Development of corporate plans to move each ROP to green (if applicable)
  • Assessment of team compliance to each ROP and 2005 recommendation, including evidence and plan to move status to green (if applicable)
Creation of accreditation team tool kits
Establishment of weekly accreditation working group meetings (middle management and program leads)
Launch of Patient Safety Culture Survey, including posters and buttons (March 2008 for four weeks)
Launch of Governance Functioning Tool (February board meeting)
Launch of team questionnaires (April 2008 for four weeks)
Creation of Excel templates to:
  • Analyze quality performance roadmap (QPR) results
  • Track team QPR follow-up actions, timelines and ownership
Phase 2: May-August 2008 Analysis of QPR results
Presentations to board, senior management, programs, teams, departments and units
Development of team action plans for QPR red and high-priority yellow flags
Biweekly Accreditation Team lead forums
Indicator submission
Development of tool kits to prepare for the tracers and survey week
Development of survey week schedule (nine versions)
Phase 3: September-November 2008 Kickoff presentation to all staff: priority processes, tracer methodology, survey week details
Presentations to board, senior management, programs, teams, departments and units
Participation in surveyor-candidate training exercise at TGH
Participation in the pre-survey teleconference
Conducting mock tracers (36 clinical and 12 administrative mocks)
Gathering required on-site documentation
Finalizing survey week preparations (posters, celebration plans, catering requirements, surveyor packages and surveyor workroom set up)
ART = Accreditation Resource Team; ROP = required organizational practice; TGH = Toronto General Hospital; UHN = University Health Network.

 

Phase 1: January to April 2008

The ART used a collaborative approach throughout the phases. The team met weekly to brainstorm ideas, assign accountability, discuss progress with action items and revise the project plan and timelines. One of the main challenges was the compressed time frame for program rollout; Qmentum is a 36-month cycle, and UHN had only 12 months to complete it.

In consultation with UHN vice-presidents, 15 standards were selected to evaluate UHN programs and services. Nineteen teams representing one or more of these standards were formed aligning as much as possible to the current clinical program structure. Each team was assigned at least one team lead, most commonly the program clinical director. To ensure that UHN could receive meaningful site- or department-specific results, the 19 teams were further subdivided into 54 sub-teams/roadmaps.

 

[Figure 1]


Early in 2008, the ART met with UHN public affairs to develop an overall communication plan for the year. This was important to establish consistent ongoing messages using existing communication channels. The team also devised a schedule for the rollout of education materials and instruments and questionnaires to staff. Education began in late January, with the ART conducting kickoff presentations to over 200 leadership staff. The presentation centred around new program features of Qmentum versus AIM, 2008 accreditation teams and key milestones for the year. In addition, the ART conducted over 90 education sessions to management and various teams, both clinical and administrative, at all three sites in a 15-week time period.

The UHN Operations Committee acted as the accreditation steering committee in order to link accreditation with operational responsibilities. To address compliance with the 25 ROPs and 2005 accreditation recommendations, a committee member was assigned ownership of one or more of these components. The ART developed a corporate ROP template in which each owner could indicate a compliance colour (red, yellow or green) and provide details on current initiatives and action plans to move each respective ROP or recommendation to green. The ART also distributed an ROP template so that leads could document team contributions to ROPs or recommendations, as applicable. To summarize and communicate UHN's overall compliance status, the ART created a monthly accreditation status report to distribute to the Operations Committee and Senior Management Team.

 

[Figure 2]


To prepare for the rollout of various accreditation instruments, the ART developed a team tool kit for the 19 teams. The tool kit consisted of an accreditation one-page summary, a copy of relevant standards and questionnaires, a suggested distribution list for the questionnaires and a request for an e-mail distribution list for clinical staff in each program, unit or department, covering the 54 roadmaps. In addition, a small Accreditation Working Group was struck to assist with brainstorming strategies for rolling out the tools and to help facilitate accreditation initiatives moving forward within each program.

In keeping with UHN's commitment to an open and transparent environment, a decision was made to involve as many staff and physicians as possible in the process. Rollout of the instruments began in late February with the UHN Board of Directors completing both the Governance Functioning Tool and Governance Questionnaire. The Patient Safety Culture Survey ran in March for four weeks. The ART, in collaboration with UHN graphics, developed a slogan as a marketing tool: "I Did It - Patient Safety Survey." This slogan was incorporated into e-mails, posters and buttons (Figure 3), which were handed out upon completion of the survey. Team questionnaires rolled out in April, with a smaller audience than the Patient Safety Culture Survey. Corresponding response rates were tracked and shared weekly with the Operations Committee and leads. Final response rates at UHN were as follows: 3,556 Patient Safety Culture Surveys (32% return rate), 18 Governance Functioning Tools (86% return rate) and 3,455 team questionnaires (68% return rate).

 

 

Several strategies were employed to increase response rates: sending e-mail reminders promoting completion online, hosting coffee and chocolate bar sessions at each site, posting daily response rates on the UHN intranet and handing out paper questionnaires at education sessions. One challenge with the questionnaires was that physicians were often asked to complete multiple questionnaires such as those pertaining to critical care and surgical services. In response, physicians were instructed to complete at least one but no more than three questionnaires, and were thanked for their contribution.

Instruments and questionnaires closed at the end of April. As UHN is an organization that emphasizes measurement and analysis of data, the quality performance roadmaps (QPRs) presented several challenges, the main one being the rigidity of the portable document format (PDF). QPRs could not be exported to Excel, thereby limiting the ability to summarize, theme and report the results across teams. To address this issue, the ART developed a complex methodology to copy the QPRs into Excel templates, sort and summarize the results and analyze the data. To effectively present results to the UHN audiences, each question was categorized into eight specific themes: culture, communication, medication use, work life and workforce, infection control, accessibility, assessment, and planning and performance.

Phase 2: May to August 2008

Accreditation Canada assigns a QPR flag colour (red, yellow or green) to each questionnaire response. UHN used 15 different standards (and associated questionnaires), for a total of 2,463 QPR flags, with the following results: red flags 3.2%, high-priority yellow flags 28.7%, low-priority yellow flags 16.9% and green flags 51.3%. UHN had 784 red or yellow high-priority flags requiring action. The majority fell within three themes: communication, medication use, and planning and performance. Again, results were presented to the UHN audiences.

Biweekly Accreditation Team lead forums were initiated so that the ART could provide updates and directions to leads and managers on Qmentum process next steps. Accreditation teams developed action plans for all red and yellow high-priority flags, as applicable. The ART distributed an Excel template for teams to track their action plans for submission into the client organization portal.

In collaboration with UHN Departments of Pharmacy and Infection Control, the ART submitted indicator data as required by Accreditation Canada. An Excel template was developed so that indicator values could be entered and trends tracked over several fiscal quarters, as applicable. This template streamlines the collection of indicator values for annual submissions.

The ART collaborated with Accreditation Canada to develop UHN's survey week schedule. The complexity of the organization led to numerous drafts as the ART needed to ensure that surveyors assessed each standard at each site, as applicable. The schedule also allotted time for meetings between surveyors and the board, senior management, community partners and clinical leadership.

In parallel, the ART designed a second team tool kit to prepare teams for the new tracer methodology and on-site survey components. This tool kit contained a one-page summary of the survey process, a poster of the tracer components, manager and front-line staff checklists, sample questions for administrative and clinical tracers and surveyor profiles.

Phase 3: September to November 2008

Kickoff events were held at all UHN sites in early September 2008 with information on the final preparations for the November on-site survey. Presentations introduced the tracer methodology, which surveyors use to evaluate clinical and administrative priority processes (18 were relevant to UHN, e.g., episode of care, medication management, human capital, etc.). One highlight was viewing the clinical tracer video from Accreditation Canada, with a real-life example of an acute care episode of care. The take-home message for staff was, "Each tracer is unique and flexible." The kickoff was followed by presentations across the organization, as in other phases.

Another opportunity for UHN to get exposure to Qmentum came in September when UHN volunteered to be a host site for Accreditation Canada surveyor training with candidates conducting four clinical and administrative tracers at TGH. This day was of substantial benefit to UHN, revealing strategies of how to finalize preparations for the actual upcoming on-site survey and allowing staff a glimpse into the process.

Throughout October, the ART conducted mock tracers. For clinical mock tracers, the ART visited nursing units in pairs and asked sample questions of staff in various roles. In addition, the ART conducted a walk around the unit to assess compliance to issues of privacy and patient safety and to view practices such as medication administration and handwashing. Administrative mock tracers involved meeting with key staff members and reviewing potential questions, while ensuring that relevant documents were up to date and easy to access. Mock tracers were 90 minutes long, including verbal feedback. Questions for the mock tracers were developed from the standards and ROPs specific to each clinical or administrative area. Formal written feedback was given to the lead and unit, department or program leadership at the end of each day. The ART also sent out a summary e-mail of the full week's events, including themes and key areas of opportunity. Teams were encouraged to use sample questions from the tool kit to conduct spot checks and audits within their own areas to measure compliance to the standards. Staff indicated that this process was a great learning opportunity for them.

In early November, the ART consolidated the required on-site documentation as outlined by Accreditation Canada and finalized preparations for the survey including the setup of the surveyor workroom ensuring proper network and phone access and catering requirements for surveyors and for the post debrief celebration to recognize staff for their hard work and participation.

Challenges and Lessons Learned

The Qmentum process presented unique challenges. Two principal challenges were staff engagement and the new on-site survey format.

Staff Engagement

Historically, not all UHN front-line staff took an active role in the accreditation process. One key benefit of Qmentum is the opportunity for front-line staff to be active participants. With nearly 12,000 staff working across UHN, the challenge of engaging staff was daunting and required innovative approaches to communication and collaboration.

The January meeting with UHN public affairs led to a detailed communication schedule outlining a variety of existing media such as the UHN weekly and risk management quarterly newsletters, e-mails from the chief executive officer, a dedicated accreditation intranet site and occasional interviews on a weekly newscast published by the chief nursing executive titled "MFP TV." An accreditation e-mail box was established so that staff could direct questions to any member of the ART. In addition, all accreditation messages were sent from this e-mail address to establish an accreditation team presence within the organization.

Early in 2008, the ART received requests from leads wanting an opportunity to meet with colleagues to discuss issues integral to the process. In response, biweekly Accreditation Team lead forums were established, providing an opportunity to share ideas, ask questions and celebrate achievements. Presenters, including ROP owners, were invited to educate leads on corporate UHN initiatives that supported teams in their accreditation preparations.

On-Site Survey

The format of a Qmentum survey is radically different from that in AIM, so helping staff to understand this new process was integral to its success. Since the Qmentum on-site survey was a completely new process, the ART was initially unsure of how to visualize and coordinate survey week. Being an Accreditation Canada surveyor, the consultant shared insight into the tracer methodology and other elements.

A lesson learned from the surveyor training exercise in September led the ART to recruit clinical directors to act as escorts during tracers. This decision was invaluable as clinical directors were able to answer any high-level questions, were familiar with all areas visited specific to their tracer and could address questions related to communication and linkages keeping in mind that surveyors primarily wanted to talk with front-line staff from many disciplines including volunteers.

Success Factors

Key UHN elements that contributed to a successful Qmentum survey are as follows:

  • Dedicated knowledgeable full-time accreditation resources
  • The involvement of staff, physicians and volunteers at all levels of the organization
  • A commitment of board of directors and leadership team to quality, patient safety and accreditation resulting in a strong quality reporting structure and accountability framework
  • The use of existing committees and communication strategies whenever possible
  • Ongoing support from the Departments of Public Affairs and Shared Information Management Services, among others
  • The integration of accreditation with operations by assigning leads for each of the ROPs to ensure accountability and a consistent corporate approach
  • The embedding of safety initiatives into practice so that staff have a clear understanding of how their role contributes to patient safety
  • The conducting of mock tracers, providing feedback and encouraging staff to use "teachable moments" to test one another on key practices such as handwashing
  • Ongoing dialogue with Accreditation Canada over the three-year cycle
  • UHN's commitment to sustainability by (1) approving a permanent full-time accreditation resource, (2) preparing biannual status reports to senior management, (3) conducting annual mock tracers and (4) continuing the integration of accreditation elements into presentation templates of the board's Quality Committee

Conclusion

UHN's on-site survey was a resounding success. The UHN on-site and forecast reports indicated that UHN met all of the ROPs and, of the 1,521 criteria assessed, only 12 were unmet. UHN has successfully embraced Qmentum in a monumental way and found the experience an overwhelmingly positive one. Staff members were genuinely pleased to be included in every element of the new program and, in particular, speaking with surveyors about the work they do.

As UHN seeks to apply what it has learned over the course of navigating this new process, the primary focus is on sustainability. Given the tremendously positive review from Accreditation Canada, UHN has an obligation to continue to build upon the work that contributed to this level of success. UHN has developed a plan to ensure sustainability and accountability for all elements of Qmentum, is committed to the quality improvement process and will continue to build on successes of the 2008 accreditation survey.

About the Author

Anita Tepfers, BHA, is the manager of patient safety and clinical risk at University Health Network (UHN), Toronto, Ontario.

Christa Hruska, BSc, MHSc, is acting team lead for performance measurement/data quality at UHN.

Justin Stone, RN, BScN, is a patient care coordinator in the Peter Munk Cardiac Program at UHN.

Jane Moser, RN, BScN, MN, is a consultant and surveyor with Accreditation Canada. She has previously held numerous positions within the healthcare field and was most recently the corporate director of nursing practice at UHN.

References

Kearns, S. 2008. "Accreditation Canada's Tracer Methodology." Qmentum Quarterly 1(1): 50-52.

Langlois, J. 2008. "Patient Safety in Action." Qmentum Quarterly 1(1): 14-19.

Nicklin, W. 2008. "Patient Safety: A Common Concern, A Shared Responsibility." Qmentum Quarterly 1(1): 7.

Nicklin, W., T. McLellan and J. Robblee. 2004. "Aim for Excellence: Integrating Accreditation Standards into the Continuous Quality Improvement Framework." Healthcare Quarterly 7(4): 44-48.