There are limited evaluations of the impact of knowledge translation (KT) activities aimed at addressing practice and policy gaps. We report on the impact of an interactive, end-of-grant KT event. Although action items were developed and key stakeholder support attained, minimal follow-through had occurred three months after the KT event. Several organizational obstacles to transitioning knowledge into action were identified: leadership, program policies, infrastructure, changing priorities, workload and physician engagement. Key messages include: (1) ensure ongoing and facilitated networking opportunities, (2) invest in building implementation capacity, (3) target multi-level implementation activities and (4) focus further research on KT evaluation.
The challenges associated with KT are widely known (Oborn et al. 2013). While KT literature has primarily focused on theories, frameworks and models, there is a lack of information on KT processes (Ward et al. 2009) and the evaluation of KT strategies (Buykx et al. 2012). When evaluations do occur, quantitative measurement tools dominate, not allowing for an interpretative approach that would provide for a better understanding of the KT implementation process (Lafrenière et al. 2013).
Acknowledging this gap in KT evaluation, we explored the impact of an end-of-grant KT event associated with a research project on barriers to integrated care.
The need for integrated care has been emphasized in recent years. Kodner (2009: 7) conceptualizes integration as "designed to create coherence and synergy between various parts of the healthcare enterprise in order to enhance system efficiency, quality of care, quality of life and consumer satisfaction". A recent research project conducted in Alberta Health Services (AHS) aimed to understand the root causes of challenges faced by patients in accessing healthcare services across the continuum of care. The research was fuelled by a joint committee between AHS and Alberta Health (AH) concerned with health system navigation and case management (Jackson et al. 2013). Committee members felt that exploring lived experiences of patients and providers would help identify the underlying causes of challenges to integrated care, which would, in turn, inform strategies to mitigate barriers to integrated care. An advisory committee comprising members from the joint committee, as well as knowledge users in various areas of Alberta's health system, worked in collaboration with the researchers throughout the research process. At the outset of the study, there was an implicit assumption that identified strategies would be embraced and implemented by appropriate stakeholders.
In total, 15 complex patients or their families and 13 of their corresponding providers were interviewed to obtain first-hand accounts of patients' journeys through the health system. Patients were selected from three population groups (mental health, children with special needs and seniors) across rural and urban Alberta, and they were included if they accessed services from at least two programs/services during the six months prior to the interview. Data analysis included a modified change analysis (US Department of Energy 1992) to allow for identification of common root causes and areas for corrective action. Further detail of this research project is available elsewhere (Jackson et al. 2013).
Knowledge translation has been touted as having potential to address the use of knowledge in all sectors and at all levels of decision-making, thereby improving health outcomes and return on research investment (Lafrenière et al. 2013). Canadian Institutes of Health Research (CIHR) (2014) describes knowledge translation (KT) as "a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the healthcare system". KT can be grouped into two main categories: integrated KT and end-of-grant KT (CIHR 2014). While integrated KT provides opportunities for knowledge users to be part of the research process, end-of-grant KT involves disseminating findings to the wider community. A formal KT model was not employed; however, our research project included ongoing interaction between researchers and knowledge users and a five-hour end-of-grant KT event with a larger audience.
For the end-of-grant KT event, key stakeholders (n = 30) included members of our advisory committee as well as other policy makers and senior health service managers with decision-making ability and/or involvement in strategic planning related to integrated care in Alberta. Two study participants also attended (one patient and one frontline provider). The objectives of the event were to share key project findings, validate data interpretations and develop action plans. Prior to the event, pre-reading packages summarizing research findings were circulated to event attendees. The interactive KT event was designed to be knowledge-user-focused – i.e., sharing perspectives, brainstorming and co-constructing plans of action that would be meaningful and implementable for knowledge users throughout AHS. The action plans focused on six areas for corrective action that were identified during data analysis. Each action plan outlined suggested activities, relevant stakeholders, alignment with other organizational initiatives, anticipated outcomes, indicators of success and potential risks and risk-mitigation strategies. Approximately six weeks following the KT event, an event summary (including documented action plans) was circulated to all attendees. The event summary is available elsewhere (AHS 2013).
To investigate the impact of the KT event, attendees and three additional stakeholders (individuals who had not attended the event but had pledged their commitment to move forward identified action items) were invited to participate in a semi-structured interview three months after the event. Interview questions focused on how information from the event had informed interviewees' work/practice, and if the event was an effective strategy for facilitating knowledge to action. In total, 25 participants consented to be interviewed. Interviews were primarily conducted via telephone, lasted approximately 30 minutes and were digitally recorded. A secondary analysis was guided by five pre-determined categories – advanced knowledge, fostered capacity, informed decisions, influenced change and improved outcomes – adapted from research impact frameworks available in the literature (Buykx et al. 2012; CAHS 2009). Because the primary thematic analysis and secondary analysis framework converged on topics of event impact, knowledge-to-action and sustaining momentum, it was appropriate to re-analyze the data (Heaton 2008).
For most interviewees, the research findings resonated and provided evidence to support their perceptions and assumptions. The majority of impact was seen in the categories of advanced knowledge and fostered capacity. For an overview of the five categories, see Figure 1.
Many interviewees stated that the research findings either validated or increased their awareness of integrated care barriers encountered by patients and providers, the impact of those challenges and the importance of addressing system gaps such as communication, collaborative practice and patient/family engagement across the care continuum.
It hit me … just how complex this all really gets and depending on your perspective you can miss other pieces of that complexity.
Interviewees reported that the KT event provided networking opportunities, contributed to a better understanding of what other stakeholders or programs were doing in relation to integrated care, and identified linkages and necessary collaborations between projects/portfolios.
Just the opportunity to meet people and get to know people better has allowed me to sort of pick up the phone, call people and get more support.
Interviewees were keen to sustain the momentum achieved at the KT event. Many expressed a desire for ongoing connection with KT event attendees to share what has been done and what the impact has been, and to identify gaps, needs and next steps. They suggested strategies such as identification of other key stakeholders and development of working groups to take forward specific pieces of work identified as necessary at the KT event. They also recommended continued communication to promote awareness, facilitate collaboration and create synergy.
There was no mention by the interviewees on how the research findings and event discussions informed decisions. However, interviewees highlighted that the KT event validated the need to continue with current work and to embrace a collaborative and integrated organizational approach to the work. Many voiced that it was helpful to now have supporting evidence.
This [research] very much aligns with my work … and it's timely … Data is a pretty powerful thing, it is hard to ignore.
Others acknowledged the importance of a purposeful approach with clear role accountabilities, and suggested that the action plans developed at the KT event needed more how-to detail, leadership commitment and defined accountability.
From my perspective there is no clarity on how to move this forward … so it is hard to harness the passion and energy when there is no road map.
The majority of interviewees reported that no new or revised work related to integrated care emerged from the KT event "as the cogs of the wheel don't move that fast." Yet, a few interviewees spoke to "new eyes" and new insight to make small local changes in their approach to working with other services and programs.
We have initiated at our local level – let's just get around the table and talk … let's learn what each others' challenges are …
Interviewees felt that support from management, awareness of organizational alignment, eagerness of staff for improvement in care and broader circulation of the KT event documentation, would be necessary for change to occur.
The [KT event] report creates a point of reference where you can start to develop a community of interest or a community of understanding.
Interviewees also identified barriers to change including lack of leadership support, restrictive program policies, lack of infrastructure, complexity of healthcare system, changing priorities, workload issues and physician engagement.
I think that our healthcare system is incredibly complicated so just even understanding who does what and how to get the right people to do this level of work is quite complicated.
While limited change had occurred, a few interviewees stressed the need for an evaluation framework to assist with identifying outcome indicators and appropriate measures. There was a desire for a focused discussion to determine "what difference has been made."
Despite participant excitement and the creation of high-level plans of action during our KT event, a three-month follow-up with our participants revealed a lack of uptake or further development of the action plans. Although we had hoped for more, this was in line with Lomas and Brown's (2009) caution that researchers should expect modest impact in health system policy making. What should have been considered was a dynamic KT plan that continued beyond the life of the research project (i.e., provided ongoing interaction with the aim of actively influencing change). Just as partnerships between researchers and knowledge users are important during the research process, sustained partnerships can leverage future use of research findings (Ross et al. 2003).
What does this mean for decision-makers and researchers? Participants identified several organizational obstacles to transitioning knowledge into action including leadership, program policies, infrastructure, changing priorities, workload and physician engagement. Suggested strategies for sustaining momentum were related to communication, collaboration, leadership commitment and defined accountability. Interestingly, our participants who were key stakeholders pointed to "the organization" as needing to step up to the plate – not acknowledging their own role with enabling and sustaining momentum. Not surprisingly, stakeholders who do not believe they have executive support, resources or accountability to implement change will not be empowered to carry the torch. To enable adoption, Greenhalgh et al. (2004) suggest the engagement of champions such as an organizational maverick, who provides autonomy to innovators, and a transformational leader, who gathers support from others in the organization. As well, health system interventions such as integration are complex, as they involve a number of interacting components (Husereau et al. 2014). Thus, a champion role of network facilitator (Greenhalgh et al. 2004) would ensure opportunities for continuous and facilitated interaction between various organizational programs and departments through networks or communities of practice. These ongoing opportunities for exchange can enable a collaborative effort to transitioning knowledge into action at the organizational level. Additionally, decision- and policy makers should create opportunities for researchers to participate in networks and committees to inform decisions (Lavis et al. 2002), while researchers and funding agencies need to consider opportunities that encourage researcher and decision-maker interactions beyond the research process (Ross et al. 2003).
What does this mean for the science of KT? Implementing research findings into practice was a challenge for our participants, and our research team lacked understanding and time to move evidence into practice and policy. While we engaged stakeholders throughout the research process, our knowledge transfer activities focused primarily on a KT event. Ideally, our KT plan should have been guided by a KT model – and specifically a global model that considered determinants, dissemination and implementation. KT related to health services research is complex and requires a multi-level approach to ensure that essential implementation strategies are addressed. Consideration needs to be given to implementation components such as system readiness, organizational structure and communication, leadership and management, human resource capacity, funding and inter-organizational networking (Greenhalgh et al. 2004). Furthermore, investment in building implementation capacity needs to be guided by implementation research (Holmes et al. 2012). Finally, additional research is needed on evaluating the effectiveness of KT strategies, including development of methods for measuring impact. To this end, Van Eerd et al. (2011) encourage further development of KT evaluation tools, especially theory-based and context-independent tools.
The science of KT is still in its infancy. Further work is required to assist researchers and organizations with mobilizing knowledge to action. In particular, our exploration identified that additional research is required to build implementation capacity and identify evaluation methods, and that translating knowledge into practice requires communication, collaboration, leadership commitment and defined accountability.
Utilité d'un événement de transfert de connaissances pour combler les fossés entre politiques et pratique dans le contexte des soins intégrés
Il y a peu d'évaluations de l'impact des activités de transfert de connaissances (TC) qui visent à combler les fossés entre la pratique et les politiques. Nous faisons rapport de l'impact d'un événement interactif de TC, après la fin du financement du projet en question. Bien que des mesures de suivi aient été mises au point et qu'il ait eu un appui de la part des principaux intervenants, on notait un suivi minimal trois mois après l'événement de TC. Nous avons noté plusieurs obstacles d'ordre organisationnel face à la transposition des connaissances en actions: le leadership, les politiques des programmes, l'infrastructure, un changement des priorités, la charge de travail et l'engagement des médecins. Les messages clés sont les suivants: (1) faciliter de façon continue les occasions de réseautage, (2) investir dans le renforcement des capacités pour la mise en œuvre, (3) cibler les activités de mise en œuvre sur plusieurs niveaux et (4) approfondir la recherche sur l'évaluation du TC.
About the Author(s)
Karen Jackson, BScN, MEd, Formerly of Alberta Health Services, Calgary, AB
Omenaa Boakye, BA, MSc, Research & Evaluation Consultant, Alberta Health Services, Calgary, AB
Nicole Wallace, BA, MPsych, Research & Evaluation Consultant, Alberta Health Services, Calgary, AB
Correspondence may be directed to: Omenaa Boakye; tel: 403-943-0721; e-mail: firstname.lastname@example.org.
The authors gratefully acknowledge all participants in this research project. This work was supported by Alberta Health.
Alberta Health Services (AHS). 2013. "Root Cause Analysis of Barriers to Integrated Care: Collaborative Workshop." Summary of Findings. Author. Retrieved August 15, 2015. <http://www.albertahealthservices.ca/assets/info/res/if-res-wre-root-cause-of-barriers-to-integrated-care-workshop-summary-2013.pdf>
Buykx, P., J. Humphreys, J. Wakerman, D. Perkins, D. Lyle, M. McGrail et al. 2012. "'Making Evidence Count': A Framework to Monitor the Impact of Health Services Research." The Australian Journal of Rural Health 20(2): 51–58. doi:10.1111/j.1440-1584.2012.01256.x.
Canadian Academy of Health Sciences (CAHS). 2009. "Making an Impact: A Preferred Framework and Indicators to Measure Returns on Investment in Health Research: Canadian Academy of Health Sciences Assessment Report." Retrieved September 9, 2014. <http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf>.
Canadian Institutes of Health Research (CIHR). 2014. "More about Knowledge Translation at CIHR." Retrieved September 9, 2014. <http://www.cihr-irsc.gc.ca/e/29418.html>.
Greenhalgh, T., G. Robert, F. Macfarlane, P. Bate and O. Kyriakidou. 2004. "Diffusion of Innovations in Service Organizations: Systematic Review and Recommendations." Milbank Quarterly 82(4): 581–629. doi:10.1111/j.0887-378X.2004.00325.x.
Heaton, J. 2008. "Secondary Analysis of Qualitative Data: An Overview." Historical Social Research 33(3): 33–45.
Holmes, B., G. Scarrow and M. Schellenberg. 2012. "Translating Evidence into Practice: The Role of Health Research Funders." Implementation Science 7: 39. doi:10.1186/1748-5908-7-39.
Husereau, D., P. Jacobs, B. Manns, T. Hoomans, D. Marshall and R. Tamblyn on Behalf of the IHE-IHSPR Complex Interventions Working Group. 2014. "Economic Evaluation of Complex Health System Interventions: A Discussion Paper." Edmonton AB: Institute of Health Economics. Retrieved September 11, 2014. <http://www.ihe.ca/publications/economic-evaluation-of-complex-health-system-interventions-a-discussion-paper>.
Jackson, K., N. Wallace, O. Boakye and P. Charland. 2013. "Root Cause Analysis of Barriers to Delivery of Integrated Care: Patients and Providers Exploring the Journey Together." Final Report. Alberta Health Services. Retrieved August 15, 2015. <http://www.albertahealthservices.ca/assets/info/res/if-res-wre-barriers-integrated-care-report-2013.pdf>.
Kodner, D. 2009. "All Together Now: A Conceptual Exploration of Integrated Care." Healthcare Quarterly 13(Sp): 6–15. doi:10.12927/hcq.2009.21091.
Lafrenière, D., V. Menuz, T. Hurlimann and B. Godard. 2013. "Knowledge Dissemination Interventions: A Literature Review." SAGE Open 3(3): 1–14. doi:10.1177/2158244013498242.
Lavis, J.N., S.E. Ross and J.E. Hurley. 2002, "Examining the Role of Health Services Research in Public Policymaking." Milbank Quarterly 80(1): 125–54. doi:10.1111/1468-0009.00005.
Lomas, J. and A.D. Brown. 2009. "Research and Advice Giving: A Functional View of Evidence-Informed Policy Advice in a Canadian Ministry of Health." Milbank Quarterly 87(4): 903–26. doi:10.1111/j.1468-0009.2009.00583.x.
Oborn, E., M. Barrett, K. Prince and G. Racko. 2013. "Balancing Exploration and Exploitation in Transferring Research into Practice: A Comparison of Five Knowledge Translation Entity Archetypes." Implementation Science 8: 104. doi:10.1186/1748-5908-8-104.
Ross, S., J. Lavis, C. Rodriquez, J. Woodside and J.L. Denis. 2003. "Partnership Experiences: Involving Decision-Makers in the Research Process." Journal of Health Services Research Policy 8(Suppl 2): 26–34. doi:10.1258/135581903322405144.
US Department of Energy. 1992. "Root Cause Analysis Guidance Document." Washington DC: US Department of Energy.
Van Eerd, D., D. Cole, K. Keown, E. Irvin, D. Kramer, J. Brenneman Gibson et al. 2011. "Report on Knowledge Transfer and Exchange Practices: A Systematic Review of the Quality and Types of Instruments Used to Assess KTE Implementation and Impact." Toronto: Institute for Work & Health. Retrieved August 15, 2015. <http://www.iwh.on.ca/sys-reviews/kte-evaluation-tools>.
Ward, V., A. House and S. Hamer. 2009. "Developing a Framework for Transferring Knowledge into Action: A Thematic Analysis of the Literature." Journal of Health Services Research & Policy 14(3): 156–64. doi:10.1186/1472-6963-9-12.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed