Evidence-informed decision-making supports high-quality, efficient healthcare. Programs such as SEARCH Classic (Swift Efficient Application of Research in Community Health) and EXTRA (Executive Training for Research Application) give health system decision-makers the skills and experience required to apply the best evidence to their work. But effectively leading change in how evidence comes to bear on the overall management and delivery of care requires strategies aimed at whole organizations and systems. The Canadian Health Services Research Foundation (CHSRF, EXTRA's managing organization) and SEARCH Canada (the SEARCH Classic program's managing organization) recently launched a jointly commissioned research study to assess organizational mechanisms and the impacts of these programs. Moving away from a focus on individual trainees and their immediate organizational connections, this evaluation builds on the evidence to date that leads to the hypothesis that a critical mass of highly educated, evidence-savvy decision-makers (senior executives in the case of EXTRA; middle- and front-line managers in the case of SEARCH Canada) enhance organizational capacity to use knowledge and ultimately lead to a more systematic use of evidence at the systems level (Champagne et al. 2008).

Reaching a Critical Mass of Trainees

Since their inception, EXTRA and SEARCH Canada have engaged in program evaluation, which focused initially on the quality of programming and the impact on participants' skills, knowledge and motivation. Evaluations have indicated that these programs have successfully increased the capacity of individual decision-makers to undertake evidence-informed decision-making in their host organizations.

Evaluation activities led by Malcolm Anderson and Mélanie Lavoie-Tremblay on behalf of CHSRF, for instance, indicate that decision-makers who took part in the first cohort of the EXTRA program have a greater understanding of research-based evidence and capacity for assessing the quality of this evidence (Anderson and Lavoie-Tremblay 2008; Lavoie-Tremblay and Anderson 2007). In three special journal issues, published between 2007 and 2008, EXTRA candidates (also called fellows) have shared personal accounts of leading evidence-informed change management projects in their organizations (CHSRF 2007b, 2008a, 2008b).

SEARCH Canada has similarly completed over 40 evaluation reports, moving from an early focus on the experience of participants to a later focus on the views of managers and executives and the functioning of the network. These reports have clearly documented both personal and professional impacts of SEARCH Classic participants, including improved technical and team skills, leadership development and career advancement. Organizational impacts appear to be mediated through projects, colleague involvement and decision influence. A major evaluation through a pan-Canadian expert panel in 2006 referenced additional organizational impacts, such as providing rural regions with a recruitment and retention edge, enhanced leadership capacity and improved access to evidence (Grimshaw et al. 2007). That panel also recommended that SEARCH invest in more innovative evaluation modes, particularly those focused on the mechanisms of organizational change. As a result, SEARCH Canada collaborated with CHSRF's EXTRA program to commission this national research study.

With 263 skilled graduates now in the system (163 from SEARCH Canada and 100 from EXTRA), the time has come to increase the evaluation focus to the organizations that sponsor the participants. Of particular interest are those organizations that are home to a number of trainees. For some of these organizations, this number may have reached the "critical mass" necessary to foster a culture of evidence-informed decision-making. It has been known for some time that a critical success factor in evidence-based practice is an organization's willingness to commit resources to ensure research is used to move the decision-making focus from the individual to the organizational level (Lemieux-Charles and Barnsley 2004; Logan and Graham 1998). Although preliminary data from past EXTRA and SEARCH evaluations indicates that trainees have been able to improve their organizational context for research-evidence informed decision-making and have been able to increase their use of research evidence with other professionals in their own organization (Birdsell 2003; McCaffrey 2003), the researchers acknowledge that more research is needed on this front (Lavoie-Tremblay and Anderson 2007).

SEARCH Canada For over a decade, SEARCH Canada has been supporting the public healthcare system by providing learning opportunities, project facilitation, network support and information services to health professionals and researchers in Alberta and elsewhere in Canada. SEARCH creates strong networks that connect evidence to the immediate issues facing decision-makers. The result is a direct increase in the ability of health professionals and health organizations to apply high-quality evidence to practice and management decisions. SEARCH was established as a not-for-profit organization with Alberta's health authorities, universities and research funders; its staff and faculty are respected by health and academic communities across Canada for their unique and highly effective approaches to knowledge transfer and applied research (SEARCH Canada 2009). EXTRA The EXTRA (Executive Training for Research Application) program was initiated in 2004. It is supported by a unique group of pan-Canadian partnering organizations and managed by the Canadian Health Services Research Foundation (CHSRF) and with a grant from Health Canada. Its mission is to increase capacity and leadership to optimize the use of research-informed evidence in Canadian health organizations. It has three overarching objectives: (1) to increase research literacy and research use in health organizations, (2) to encourage collaboration among health professionals and (3) to generate evidence-informed change in Canadian health organizations (CHSRF 2009).

Evaluating the Organizational Impact of Participating in EXTRA and SEARCH Canada

In late 2006, SEARCH Canada and CHSRF initiated discussions to explore their mutual interest and potential options for conducting evaluative research to add to the collective understanding about the impact of their programs. To further these discussions, SEARCH Canada, CHSRF and the Canadian Institutes of Health Research co-sponsored a workshop in mid-2007 to begin planning a joint evaluation initiative. The workshop report highlights the underlying premise of the project - that access to a critical mass of trained decision-makers who have the skills, knowledge and desire to build organizational capacity to use knowledge should lead to a more systematic use of evidence in organizational decision-making (CHSRF 2007a).

A second workshop was held in early 2008 to determine the evaluation questions, key design elements, case-selection criteria, the structure and composition of the research team and timelines for the research proposal. A national research team was established under the direction of co-principal investigators François Champagne, professor of Health Care Management, Health Policy and Health Care Evaluation in the Department of Health Administration at the Groupe de recherche interdisciplinaire en santé (GRIS) at the University of Montreal, and Louise Lemieux-Charles, chair of the Department of Health Policy, Management and Evaluation at the University of Toronto. The team developed a research proposal, Knowledge Creation in Health Care Organizations Involved with the EXTRA and SEARCH Programs: An Evaluation of the Organizational Impact of Participation, which was completed in late 2008 (Champagne et al. 2008). This study has since passed merit review through CHSRF and a research ethics review. The evaluation will also be guided by an advisory group, with representation from SEARCH Canada and EXTRA.

This qualitative evaluation will consist of multiple case studies, each with embedded units of analysis, from the individual to groups of participants to the organization. The criteria used to select the cases reflect a mix of provinces, sizes and complexity of urbanization levels (metropolitan, non-metropolitan and rural), as well as consideration of the extent to which each organization has participated in EXTRA or SEARCH Canada. Six case studies were formally selected for investigation, including three organizations from the newly established Alberta Health Services (formerly, these organizations were the David Thomson Health Region, Chinook Health and Aspen Regional Health), the Saskatoon Health Region, McGill University Health Centre (Montreal) and Capital Health (Halifax). The research team will explore the actions initiated concurrently or following the participation of trainees in the EXTRA or SEARCH Classic programs. In particular, the study will aim to meet three objectives:

1. To understand the nature and extent of the impact organizations have experienced as a result of having mid- and senior-level managers trained through SEARCH Canada and EXTRA, respectively
2. To determine the organizational process through which such impact occurs
3. To explore the contextual conditions facilitating or impeding this impact

It is hoped that the findings from this evaluation will add to the understanding of how training a critical mass of mid- and senior-level decision-makers can contribute to a dynamic knowledge creation process, reinforcing the organizations' learning capacity and leading to beneficial organizational outcomes. As part of this work, the investigators will consider the organizational learning capacity, the organizational support structure and the environment as factors in the observed results.

Conclusion

The final evaluation report is to be completed in late 2009. For more information about this joint evaluation initiative, please contact research coordinator Marie-France Duranceau at marie-france.duranceau@umontreal.ca. For further information about SEARCH Canada, contact Chief Executive Officer Sarah Hayward (sarah.hayward@searchca.net). For further information about EXTRA, contact Director Nina Stipich (nina.stipich@chsrf.ca).

Comments

Be the first to comment on this!