Integrated knowledge translation (iKT) refers to the engagement of knowledge users (e.g., policy makers, clinicians, patients) as active participants in the research process. Theoretically, this involvement enhances research relevancy and usefulness, thereby supporting health system change. However, evidence to support best practices for iKT is lacking, particularly in a public health context and with non-clinical decision-makers. The objectives of this research were to report how decision-maker involvement in public health iKT research has been described and operationalized and whether the process was evaluated. We conducted a scoping review of published literature from January 2005 to December 2017 and extracted information related to iKT involvement, barriers and facilitators and outcomes. Studies typically did not distinguish between different kinds of knowledge users, making it impossible to comment specifically on decision-makers' involvement. Authors believed knowledge user involvement was beneficial to the quality and potential impact of research activities, although corroborating evaluation data were unavailable. Broad research–knowledge user partnerships spanning multiple projects, as well as flexible involvement of knowledge users, enhanced engagement and supported the iKT process.

Background

"Wicked" problems are characterized by multiple causes within complicated and dynamic social and political contexts (Rittel and Webber 1973). Many public health challenges – inactivity, unhealthy eating, problematic substance use, the effect of the environment on health – are "wicked" (Kreuter et al. 2004), thereby requiring the synthesis, interpretation, translation and mobilization of vast bodies of evidence to arrive at optimal solutions (Brownson et al. 2009; Kitson et al. 2013; Lobb and Colditz 2013). Evidence-informed public health decision-making necessitates the use and adaptation of evidence from applied health research to support public health policy and program development (Boyko 2015; Brownson et al. 2012; Brownson et al. 2006).

One approach to maximizing the effect of evidence on policy and practice is to work with the intended recipients of research evidence: knowledge users (KUs). Participatory models of research (e.g., community-based participatory research, participatory action research) have been used in public health to work with communities to better develop, implement and evaluate research interventions and translate findings into meaningful change (Ahmed and Palermo 2010; Wallerstein and Duran 2010). These methods have been used to develop a sense of community "ownership" of issues and galvanize community action as a method for social change (Israel et al. 1998). Building partnerships between researchers and community stakeholders is a complex and active process, but it can add value to the research process by creating better quality research outputs and enhancing the sustainability of project goals while also building capacity in stakeholder groups (Jagosh et al. 2012).

In Canada, integrated knowledge translation (iKT) is a similar approach that has gained considerable traction (Bowen and Graham 2013; Graham 2012; Kothari and Wathen 2013). iKT has many similarities with participatory methods, including the shared desire to co-create knowledge to improve current conditions (Jull et al. 2017). Both blur the distinction between researchers and research participants; both acknowledge that knowledge development is influenced by values and widen the ambit of "acceptable" knowledge by negotiating perspectives between researcher producers and research users (Jull et al. 2017). Given the history of community-based participatory research, there are also many lessons to inform KT approaches, such as the underlying principles for working together, structures that support this collaboration, the processes for working together and the dynamics of the relationship itself (Lencucha et al. 2010). However, while participatory approaches emphasize community-driven solutions, iKT is intended to generate research-based solutions, and KUs are involved because of their authority to invoke change following the completion of the research (Gagliardi et al. 2016; Kothari and Wathen 2017). Participatory research and iKT also differ in terms of their respective historical motivations (social justice, as opposed to application of knowledge) and social locations (grassroots-/citizen-led, as opposed to health decision-makers) (Jull et al. 2017).

Given the centrality of generating research that are relevant to KUs and can be adopted into practice, there is an expectation of involving KUs throughout the research process, including refining research questions, developing research methods, analyzing findings and sharing results (Graham 2012). Strengthening the relevancy of research findings to KUs can support research-informed decision-making (Innvaer et al. 2002; Oliver et al. 2014); produces research that is more contextually applicable (McLean et al. 2012); improves researcher–KU relationships for future collaborations (Kothari and Wathen 2013); and leads to better social, research and health services outcomes (Gagliardi et al. 2014).

The translation of knowledge into improved health and health services, and iKT by extension, was enshrined in the act of parliament that created the Canadian Institutes of Health Research (CIHR) (The Government of Canada 2000). The push for KT stemmed from two factors: (1) a growing awareness that active efforts were needed for knowledge to be widely used in policy and practice and; (2) a trend in government and public demand for spending to be more in line with, and accountable to, public good (McLean et al. 2013; Tetroe 2007). Activities have included requiring KT plans as a condition for funding, as well as providing resources and training for KT, including iKT (Graham 2012). Another relatively recent innovation was the 2011 launch of CIHR's Strategic Patient-Oriented Research (SPOR) initiative, designed to strengthen the ability of researchers and health system personnel to work with patients to produce research with a greater likelihood of improving health systems and practices (Strategy for Patient-Oriented Research 2014). While still in its infancy and not without areas for improvement (e.g., clarification of mandate, evaluation of outcomes), the SPOR initiative has been effective at engaging stakeholders and building interest and support for patient-oriented research (KPMG LLP 2016).

Canada is also by no means the only country to create structures and supports to foster relationships between health system partners and encourage the movement from research to influence health system change. The CLAHRC collaborative in the UK comprises multiple university–healthcare organization partnerships and has yielded important insights about the structures and processes, which support the adoption of evidence into practice (Fitzgerald and Harvey 2015; Rycroft-Malone et al. 2016). Other examples of programs or systems enhancing how researchers and other system stakeholders and service users work together include the Australian Prevention Partnership Centre (Wilson et al. 2014) and the World Health Organization-directed Evidence-Informed Policy Networks (EVIPNet) spanning continental communities (World Health Organization 2015).

Policy and decision-makers (PDMs) play a vital role in public health policy development and initiation and are key KUs to consider when conducting public health research. Although the use of research evidence by PDMs can strengthen health system performance and improve individual health outcomes, there are many barriers to its use including poor access to relevant, timely, context-sensitive and understandable research (Innvaer et al. 2002; Lavis et al. 2003; Lavis et al. 2012; Oliver et al. 2014; Orton et al. 2011). While iKT is intended to overcome many of these issues, it also presents additional challenges: iKT activities often require considerable time commitments, may not produce results that lead to immediate policy or practice change and are often not rewarded by traditional academic or PDM advancement structures (Kothari and Wathen 2013). Even in cases of nationally funded iKT research, KU engagement can be "tokenistic" (Sibbald et al. 2014).

While evidence regarding how iKT activities are currently being structured to enhance engagement with KUs is sparse (Kothari and Wathen 2017), there are some emerging best practices. A recent scoping review on iKT in healthcare noted that facilitators reported by multiple authors included multiple – and varied – opportunities for KUs to contribute to the research, as well as strong leadership and a phased approach to allow time for building shared understanding and early successes (Gagliardi et al. 2016). These are similar to the "3 Ts" reported by Wathen and colleagues (2011) from their four-year KT and exchange project with stakeholders in the area of violence prevention; talk, trust and time were key factors to effectively engaging with stakeholders and enhancing their receptivity to acting on research findings (Wathen et al. 2011). Other potential models for iKT, such as consensus-based approaches (i.e., where researchers and KUs work together to solve problems and determine how their answers can be used) and interorganizational networks, offer new ways for thinking about how iKT research can be successfully conducted (Kothari and Wathen 2017).

However, evidence-informed policy making is more nuanced than simply getting the right evidence into the right hands. Consider public vaccination programs: While PDMs can use evidence to help choose a course of action (e.g., What vaccine does scientific evidence suggest will be most effective at immunizing people from the flu this year?), there are other decisional factors that cannot be found in the research literature (e.g., How much more are we willing to pay for a more effective vaccine?). Moreover, PDMs must also consider how a course of action can be most successful in a given setting and consider factors such as equity (e.g., Should we prioritize access for vulnerable populations?) and workforce planning (Are the benefits of a new vaccination protocol worth the costs of new training and safety requirements?), as well as ponder if more far-ranging and fundamental shifts are worth pursuing (e.g., What are the implications of a mandatory vaccination program for healthcare providers?). PDMs must thus consider and prioritize many factors in addition to evidence when deciding on public policy, including political priorities, the political capital required to affect change, the timeline of the commitment, public values, stakeholder input and the optimal return of investing limited resources (Andermann et al. 2016; Cairney and Oliver 2017; Lavis et al. 2009; Liverani et al. 2013; Moat et al. 2013). For example, the "3I+E" framework proposes that Ideas, Interests, Institutions and External Factors are the drivers of policy decisions (Lavis et al. 2012); in this conceptualization, research evidence constitutes a single kind of "idea" for how the world should be. Models that conceptualize evidence uptake as a transfer problem between research producers and research users have been criticized for their ignorance of these political factors and processes, and are therefore of limited utility to the policy community (Cairney et al. 2016).

Moreover, given the complexity of health policy making, how PDMs might use evidence is likely to be different from other KU groups (e.g., clinicians) and thus must be supported differently. For example, with near-perfect knowledge of a condition and confounding variables – e.g., allergies, interactions with other medications, risk of complications – a clinician should be able to determine a course of action with a high degree of confidence. This reductionistic logic is challenged in policy settings, where PDMs must consider not just what to do about a perceived problem, but what the problem is, how best to implement a solution that will be feasible within a contested ideological jurisdiction and whether to prioritize this problem at the expense of limited resources (Cairney and Oliver 2017). Depending on the "phase" of decision-making, PDMs may use evidence to prioritize areas of interest or dispute claims of interest groups for attention (agenda setting), reduce uncertainty and improve confidence in options as well as reduce duplication of services (policy development) or provide a basis for ongoing program improvement and an impetus for accountability (monitoring and evaluation) (Lomas and Brown 2009). In addition, while PDMs can use evidence instrumentally to directly inform a policy decision, they may also use it to conceptually inform their thinking of an issue or symbolically to justify a course of action that was already decided ("decision-informed evidence making") (Amara et al. 2004; Weiss 1979).

Accordingly, successful iKT with PDMs signifies more than just a willing audience for research; aligning the goals and objectives of knowledge producers and KUs is intended to influence the research process such that it is tailored to support the needs of policy makers, and findings are more readily integrated into the decision-making process. Given that a key component of iKT is engaging those who have the authority to apply research findings to a problem, and that the basis of this process is developing a shared understanding so that evidence generated is more likely to be of use amidst other decision factors, learning from existing iKT descriptions stands to support future public health research. Given the primacy of iKT research in Canada as indicated by dedicated resources and funding opportunities (Graham 2012), as well as evidence that meaningful KU engagement is not always achieved (Sibbald et al. 2014), a better understanding of how to successfully conduct iKT research is beneficial to the public health research community. As such, the objectives of this study were to explore how PDMs' involvement in public health iKT research has been described and operationalized in the public health research literature and whether there is convincing evidence for the impacts of an iKT approach.

Methods

To understand how PDMs have been involved in public health iKT research, we conducted a scoping review. A scoping review follows a transparent and replicable process of mapping the literature in a systematic way and is useful for describing unexplored or developing topics and identifying gaps (Arksey and O'Malley 2005; Daudt et al. 2013; Levac et al. 2010). An iterative approach was used to conduct the scoping review, where we critically reflected at every step of the process to adapt our approach (e.g., inclusion criteria, data charting) to best answer the question based on the mapping of the literature (Arksey and O'Malley 2005).

First, a research librarian was consulted to assist in the development of the search strategy for PubMed, CINAHL and EMBASE databases. The searches used the "AND" operator to identify articles across three areas: knowledge translation, public health and policy making (see Appendix A for detailed strategy, including construction of search areas). The search strategy was initially pilot tested to ensure that specific key articles were identified (Arksey and O'Malley 2005). Searches were limited from January 2005 (two years before the earliest known use of the term iKT the authors could identify) (Graham and Tetroe 2007) to December 2017. We also hand-searched the table of contents of four relevant journals (Milbank Quarterly, Implementation Science, Health Research Policy and Systems and Evidence & Policy). In addition to searching databases, a search was carried out through Google Scholar in January 2016 using the term "integrated knowledge translation," as a mix of search engines enables greater capture of articles that may otherwise be missed (Rothfus et al. 2016). Protocol papers that met inclusion criteria were also forward-searched via Google Scholar to identify completed works.

All citations were imported into a reference manager program (RefWorks) where duplicates were removed. Two reviewers independently applied inclusion and exclusion criteria to citation titles and abstracts to select relevant studies for full-text review (Appendix A). As per scoping review methodology, our inclusion criteria were adapted based on our findings. For example, we included overviews of research initiatives even if these did not report results of the primary studies, as they provided valuable detail on the iKT processes used. We also broadened our definition of PDM to include individuals involved across stages of the policy making process, including decision-making (e.g., "politician," "civil servant") (Lavis et al. 2013) as well as policy implementation and evaluation (e.g., "administrator," "manager") (Fafard 2008). Disagreements were resolved through discussion and consensus, with reviewers erring on the side of inclusion to ensure that all relevant studies were captured. Two authors (LL and AB) then independently performed full-text screening.

Data extraction was based on factors or variables identified in the iKT literature, such as how KUs were involved in the research process (e.g., informing the research question; collecting, analyzing and interpreting data, disseminating findings) (Bowen and Graham 2013; Graham 2012; Shen et al. 2016). Other extracted characteristics of the iKT process included the development of the researcher–KU relationship (e.g., trust, mutual learning) (Sibbald et al. 2014) and evaluation of the iKT process (Bhattacharyya et al. 2013). The data extraction form was pilot-tested prior to commencing the full search and adapted iteratively to capture emergent salient details, such as the kind of study (e.g., protocol, single project or overview of a larger research initiative) and challenges and facilitators to the iKT process (Appendix B).

Quality appraisal was conducted using the Mixed Methods Appraisal Tool (MMAT) (Pluye et al. 2011) to describe the quality of identified studies. The MMAT has been previously used in scoping reviews in the field of KT and is useful when included studies span a variety of research designs (Dagenais et al. 2013).

Results

A total of 3,848 citations were screened, of which 232 were advanced for full text review. Twenty-six articles were selected for data extraction; during this process, six were excluded because of their setting (e.g., clinical care), leaving 20 articles in the final review (see Figure 1 for PRISMA Flow Diagram). Included studies spanned a variety of subject areas and designs (see Table 1). There were three types of article formats: protocols, single studies and descriptive reports providing "overviews" of research programs employing an iKT approach.

When the affiliation of PDMs was specified, they often represented provincial governments or health authorities, although other policy actors representing the federal government, research funders or policy-related services or agencies (e.g., World Health Organization, Public Health Agency of Canada) were also included (Table 1). Only one project (Oelke et al. 2015) worked solely with PDMs as part of their iKT process, although other stakeholders were to be included at a later phase. Other KUs included health providers, patients and their families, members of the public and representatives from other KT and community health organizations.

With one exception, authors described the involvement of KUs as one entity rather than describing the individual roles of the different groups of KUs in a given project (McGrath et al. 2009). It was therefore not possible to determine the contributions of PDMs relative to other types of KUs; as such, we report on iKT with KUs in general rather than PDMs specifically.

Rationale for iKT

All studies provided reasons for involving KUs (i.e., assumptions about the theoretical merits of iKT), though it was occasionally unclear whether this could be attributed to an iKT approach or to something else (e.g., using community-based participatory research principles). While some introduced the value of KU involvement as a general enhancement to the research process (e.g., multiple perspectives) (Bullock et al. 2010; Henderson et al. 2014; Iyer et al. 2015; Pelletier et al. 2011; Porter et al. 2012; Valaitis et al. 2012; Welch et al. 2015), others anticipated KUs to assist in research design, such as shaping research questions or deciding research priorities (Bornstein et al. 2017; Hayden et al. 2015; Kothari et al. 2014a; Naqshbandi Hayward et al. 2016) and/or informing methods (e.g., intervention content) (Boivin et al. 2011; McGrath et al. 2009; Wathen et al. 2011). Many believed that involving KUs would improve the use of their research (i.e., enhanced relevance and utility of research findings and adaptability to user contexts) and its spread (i.e., timely and facilitated dissemination, uptake or use of research findings, as well as overcome resistance to new ideas). The majority of rationales stated that KUs would assist in their dissemination or uptake, although in a few cases, some KUs were expected to help implement the research or interpret the findings. This included reducing potential resistance to new ideas (McGrath et al. 2009), understanding their preferences for how best to translate or exchange findings (Henderson et al. 2014) and ensuring relevancy to KU needs (Bullock et al. 2010; Henderson et al. 2013; Kothari et al. 2014a, b; Naqshbandi Hayward et al. 2017; Oelke et al. 2015). In two cases, KUs had requested and funded the research (Bullock et al. 2010; Hayden et al. 2015).

Engaging KUs in Research Process

Authors' descriptions of KU involvement across all aspects of the iKT research were often unclear, making it difficult to discern the level of engagement. Based on unequivocal descriptions of involvement, KUs helped to: (1) develop research questions (Bornstein et al. 2017; Bullock et al. 2010; Hayden et al. 2015; Naqshbandi Hayward et al. 2016; Oelke et al. 2015; Pelletier et al. 2011; Valaitis et al. 2012), (2) inform methods (Bornstein et al. 2017; Bryant et al. 2011; Bullock et al. 2010; Fortin et al. 2016; Hayden et al. 2015; Henderson et al. 2013, 2014; Iyer et al. 2015; McGrath et al. 2009; Porter et al. 2012; Wathen et al. 2011; Welch et al. 2015), (3) collect and analyze data (Boivin et al. 2011; Bornstein et al. 2017; Bryant et al. 2011; Fortin et al. 2016; Iyer et al. 2015; Naqshbandi Hayward et al. 2016; Porter et al. 2012; Valaitis et al. 2012), (4) interpret results and craft the overall message (Boivin et al. 2011; Bornstein et al. 2017; Bullock et al. 2010; Fortin et al. 2016; Hayden et al. 2015; Kothari et al. 2014b; Naqshbandi Hayward et al. 2016; Oelke et al. 2015; Porter et al. 2012; Valaitis et al. 2012; Wathen et al. 2011); and (5) share findings and move research into practice (Boivin et al. 2011; Bornstein et al. 2017; Bullock et al. 2010; Fortin et al. 2016; Hayden et al. 2015; McGrath et al. 2009; Naqshbandi Hayward et al. 2016; Oelke et al. 2015). Out of these five kinds of opportunities, studies reported an average of two involvement activities and ranged from describing none (Kothari et al. 2014b; Rosella et al. 2014) to all five (Bornstein et al. 2017). However, some studies, such as Rosella and colleagues (2014), did mention working with KUs to guide evaluation, which was not captured in our data extraction.

In addition to meetings with KUs (e.g., face-to-face, teleconference, project launch), engagement activities included exchange forums or think tanks, as well as using existing planning events to work with KUs. Other iKT activities included workshops, online communities and using knowledge brokers. Surveys were used to elicit specific information from KUs, while consultation and small work groups and individual interviews were used to support larger KU engagement efforts; Henderson and colleagues (2014) differentiated between information sharing and formal and informal input-seeking, suggesting that different methods of engagement serve different functions. More general methods of KU contribution included executive or advisory committees, site-specific or local advisory groups and networking events. Strategies for sharing information, plans and research proposals with KUs included regular updates or reports/newsletters, distributing summaries or briefs, media press releases, conferences, presentations, website content and teleconference updates.

Less than half the studies explicitly described the contribution KUs made to the research, such as prioritizing research questions (Hayden et al. 2015), providing contextual information that helped frame the research findings (Bullock et al. 2010), developing quality indicators (Bryant et al. 2011), planning implementation blueprint (Boivin et al. 2011) and centralizing the voices required for determining success (Naqshbandi Hayward et al. 2016). Three studies noted that their methodology evolved or was flexible as a result of working with their KUs (Bornstein et al. 2017; Fortin et al. 2016; Hayden et al. 2015). Table 2 describes the challenges, barriers and facilitators to iKT identified in the included studies. It was unclear whether KUs were co-authors on publications.

Managing Relationships and Challenges

Despite the importance of the relationship with KUs, few studies described existing relationships with KUs or how the iKT partnership developed. Only one study reported on the relationship they had with their main KU (Bullock et al. 2010), although two reported on how their relationship with KUs had changed over time and led to new initiatives (Bornstein et al. 2017; Fortin et al. 2016). The challenges of working with KUs reported by a minority of authors fell into three broad categories: logistics and coordination (e.g., resource use, time, information sharing), negotiating tensions and building shared understanding (e.g., aligning research and policy considerations), and external constraints (e.g., changes in personnel and political priorities). While solutions were not offered for all challenges, authors offered advice as to how they overcame some challenges and would structure future projects.

The time and resources required to do iKT optimally were substantial, although face-to-face meetings were seen as worth the time and cost (Fortin et al. 2016; Hayden et al. 2015; Henderson et al. 2013, 2014; Kothari et al. 2014b; McGrath et al. 2009; Valaitis et al. 2012). Authors recommended budgeting for these activities (McGrath et al. 2009) and that engagement is considered as the project is conceptualized (Henderson et al. 2014). Meeting the information needs of multiple KU partners was also challenging, both in terms of providing timely updates as well as balancing different information needs without privileging some KUs (Bullock et al. 2010; Kothari et al. 2014b; Wathen et al. 2011). Guiding documents such as a memorandum of understanding (Bullock et al. 2010) or governance structure (Naqshbandi Hayward et al. 2016) can make reporting relationships explicit, thereby enhancing clarity and mitigating the risk of miscommunication or not meeting expectations.

The importance of good communication extended to building a shared understanding of the work. Many authors reported challenges with low involvement of KUs (Fortin et al. 2016; Wathen et al. 2011), ensuring usefulness of research to the KU organization (Kothari et al. 2014b; McGrath et al. 2009; Wathen et al. 2011) and balancing methodological rigour with involvement (Bullock et al. 2010). Again, careful planning was thought to maximize the role of KUs in the project, such as considering how KU involvement could vary according to their willingness or availability (Henderson et al. 2014) as well as across research stages (Valaitis et al. 2012). Technology (e.g., web-based conferencing platforms) was thought to be useful at engaging stakeholders, although knowledgeable support staff were required, and problems with organizational information technology security requirements were encountered (Valaitis et al. 2012). Demands for research to inform program or policy decisions while the research is ongoing could also be serviced by drawing on existing research (Kothari et al. 2014b) or through alternative reporting strategies that can still support decision needs without sacrificing research integrity (McGrath et al. 2009). It was also challenging to situate the iKT portion within a larger policy or program environment; while a memorandum of understanding can help all parties determine how the iKT process aligns with ongoing work, this may take considerable time (e.g., 1–2 years) to complete (Bullock et al. 2010). Also, while some KUs may feel they do not have the expertise to offer meaningful contributions (Henderson et al. 2014), their input is vital for determining how research findings can be successfully integrated into health system practices (McGrath et al. 2009). The development of trust (Bullock et al. 2010; Fortin et al. 2016; Naqshbandi Hayward et al. 2016; Wathen et al. 2011) and mutual learning (Bornstein et al. 2017; Fortin et al. 2016) helped create relationships with the potential for future collaboration.

Finally, external factors may be the most capricious and deleterious to the success of iKT projects. Changes in personnel – both within the research team as well as with KU organizations – requires time and effort to ensure that institutional memory is maintained (Bornstein et al. 2017), trust is built anew and incoming project members are brought up to speed (Bullock et al. 2010). It is possible to maintain relationships through organizational restructuring, although the research team is required to adapt and update their training to continue to understand KU interests and align their work accordingly (Bornstein et al. 2017). Unfortunately, other challenges were left unanswered (e.g., identifying the right stakeholders, competing priorities impeding KU engagement, conflicting priorities, negotiating credit for outputs), so these serve as issues to be aware of for researchers wishing to engage in iKT.

Research Impact

IKT research required extra time and resources, notably for coordinating meetings, maintaining communication and developing tools. Tensions between researchers and KUs sometimes complicated the research process, such as when the research evidence does not support a desired direction (Wathen et al. 2011). In spite of these challenges, the vast majority of completed projects noted a variety of benefits in line with their rationales for conducting iKT research, such as ensuring research was relevant to KUs and enhancing the sharing and use of findings (Bornstein et al. 2017; Bryant et al. 2011; Fortin et al. 2016; Hayden et al. 2015; Kothari et al. 2014b; McGrath et al. 2009; Porter et al. 2012; Valaitis et al. 2012; Wathen et al. 2011). Other benefits included enhancing methodological feasibility (Henderson et al. 2013, 2014), building partnerships for future collaborations (Bullock et al. 2010; Hayden et al. 2015; Kothari et al. 2014b; McGrath et al. 2009; Valaitis et al. 2012), fostering ownership over knowledge creation (Naqshbandi Hayward et al. 2016) and overcoming differences to better understand each other's constraints and areas of interest (Fortin et al. 2011; Pelletier et al. 2011; Wathen et al. 2011).

Only one study evaluated the iKT process, although its focus was on the partnership process rather than whether iKT affected research use or health outcomes (Kothari et al. 2014b). Those reporting on the ability of KUs to act on research findings believed it to be strong because the research was commissioned by them, or they had actively disseminated the research. Only one study (Wathen et al. 2011) examined knowledge use, finding that research was much more likely to be used conceptually or symbolically than instrumentally (Weiss 1979) and that the kind of use depended on where they were in the decision-making process.

Quality of Studies

The overall quality of studies was low to medium, with an average MMAT score of 50%. However, only 8 out of the 20 studies met the screening criteria for going through quality appraisal (e.g., clear research questions or objectives, data collected address questions or objectives), mainly because of the format of the article (i.e., descriptive articles do not have clear questions or objectives). Therefore, quality appraisal results should be interpreted accordingly. There was variation between study types in terms of being eligible for MMAT scoring as well as the actual score received: "project" reports (4/7 eligible for scoring) had an average MMAT score of 62.5%; "overview" reports (1/9) had a score of 75% and "protocol" reports (3/4) had a score of 25%.

Discussion

This scoping review explored public health iKT research involving PDMs and identified various rationales and strategies for engaging KUs in research. iKT provided means to incorporate KU insight, preferences and concerns to produce feasible, relevant and useful research with the potential to develop future collaborations. The researcher–KU collaboration was enhanced by taking the time to develop trust and mutual respect (Bullock et al. 2010; McGrath et al. 2009; Wathen et al. 2011). Meaningful engagement is crucial to this end: research is more likely to be used by KUs if they have been consulted to accurately determine their needs, rather than have knowledge that may not meet these needs "translated at them" (Bowen and Graham 2013). Because trust and mutual respect can also be seen as indicators of collaboration, this likely reflects a trust-building cycle: some collaboration is necessary to build trust and respect, which in turn fosters a deeper relationship.

The importance of relationship-building has been noted in the KT literature (Bowen and Martens 2005) as well as reviews or co-creation in community health services (Greenhalgh et al. 2016) and broader interorganizational partnership literature (Winters et al. 2016). It is unclear why relationship-building was only reported in a handful of included studies; this may reflect the belief that relationship-building is not a methodological consideration worth reporting or that there were no conscientious relationship-building activities incorporated in the iKT process; in either case, the reader is not provided with guidance on how to build relationships. This is reflected in both a recent call for an iKT research agenda (Gagliardi et al. 2017) as well as a protocol for a longitudinal program of research examining how iKT works under different circumstances and partnerships (Graham et al. 2018).

Allowing KUs to decide their level of engagement and be flexible about their commitments was thought to let them contribute to the best of their abilities (Bullock et al. 2010; Henderson et al. 2014; Wathen and MacMillan 2015; ). Different KUs bring differing levels of ability and expertise to the table, suggesting that careful tailoring of involvement opportunities, rather than seeking to maximize involvement of KUs in all phases of the iKT process, is helpful (Henderson et al. 2013; Wathen and MacMillan 2015). Given the different ways that KUs can be involved (e.g., developing the research question, informing methods, implementing the study, interpreting results and disseminating findings), there is substantial opportunity to determine where KU interests lie and how to best leverage them. Ten of the included studies took the form of "overviews" of programs of research or networks; as these contained multiple projects, they offered multiple opportunities for KU involvement and thus may be better designed to allow KUs to modify their involvement as time and interest permit. Recent Canadian work exploring how engagement was defined among leadership within a policy and service delivery organization found that across hierarchical levels, engagement was defined similarly (Norris et al. 2017). Facilitating participation at the individual level (e.g., determining willingness and varying level of involvement accordingly), connecting around a purpose (i.e., shared vision or goal), and providing opportunities for meaningful interaction and dialogue all align with our findings regarding the importance of a meaningful relationship with KUs. The "spectrum of public participation" offered by the international association of public participation (IAP2; iap2.org) has been used to support patient engagement in research (Bellows et al. 2015), and could assist researchers to determine the optimal level of involvement for PDM KUs.

Further, as relationships change with experience, the "maturity" of the partnership (i.e., degree of shared history and understanding) can help determine the appropriate degree of KU involvement (Kothari et al. 2011) (although this was not associated with outcomes in a recent iKT scoping review; Gagliardi et al. 2016). Existing literature on health services partnerships, such as task and role negotiation, leadership, and conflict management, may also be useful in guiding researcher-PDM collaboration and operationalize iKT processes (Curry et al. 2012; Winters et al. 2016). While formal arrangements (e.g., memorandums of understanding) can help address potential issues, informal mechanisms (e.g., good working personal relationships between researchers and KUs) can support formal mechanisms and thus should not be undervalued (Bullock et al. 2010). Regardless of specific methods used, emphasis on principles of engagement to guide the research process, understanding perspectives beyond the academic world, and drawing on the experiences and perspectives of KUs are crucial for avoiding failure (Greenhalgh et al. 2016).

It is also worth noting that the rationale of involving PDMs in iKT implies the shared goal of using the resulting research to change policy and practice. When the research question is straightforward and the results provide clear direction, uptake is supported. However, PDMs may refuse to act on evidence with which they disagree (e.g., conflicts with their experience; Wathen et al. 2011). Notwithstanding an initial agreement between researchers and PDMs on the shared value of the research, other values may conflict (e.g., scientific rigour needed for publication as opposed to the imperative to act within a timeline and available resources, respectively) (Estabrooks et al. 2012; McGrath et al. 2009). In this way, iKT may not be as useful for public health problems that require urgent action or are contentious; after all, research is a single input that decision makers must consider in their calculation for action.

Goals also need to be shared at the appropriate level of responsibility. The labels of "policy maker" or "decision-maker" may be somewhat misleading, as this group comprises different roles (e.g., analysts, advisors, decision-makers) and functions (e.g., evidence gathering, advising, decision-making), each of which may have differing needs and goals (Morestin 2017). Consider, for example, junior policy analysts and senior executive directors within a ministry of health. Each has different responsibilities (e.g., gathering evidence to inform a policy options paper as opposed to weighing the pros and cons of different options to make a decision they are accountable for) and likely differs in training and familiarity with accessing, appraising, synthesizing and applying research evidence. Thus, the ultimate goals of each, their role within the same iKT project and the best ways to nurture these relationships, are likely to be different.

These relationship considerations are integral to fostering strong communication and building a shared understanding. One of the benefits of iKT is that two groups – researchers and KUs – gain a better understanding of each others' "worlds" and thus are better able to accommodate and support each other (Kothari and Wathen 2013). Exposure to PDMs helps researchers understand the nuances that influence their decisions and better prepares them to contribute research to the policy process, including how to best communicate research, to whom and at what time (Cairney et al. 2016; Kothari and Wathen 2013). This understanding and mutual respect also prepares the grounds from which future collaborations spring, and a number of authors spoke to how iKT helped sow these seeds (Bryant et al. 2011; Hayden et al. 2015; Henderson et al. 2014; Kothari et al. 2014b; Naqshbandi Hayward et al. 2016; Valaitis et al. 2012). In this way, while the iKT research itself may not yield desired results, the relationships built therein may be a fruitful consolation.

To determine to what extent iKT creates future collaborations and long-term health improvement, evaluating iKT strategies is critical to understanding its effects (Gagliardi et al. 2017). However, only one study evaluated the iKT process (Kothari et al. 2014b) with a concentration on the effects of partnerships between members of a research network, rather than specific research and health outcomes. Evaluating the effects of partnership beyond process measures has been a long-standing difficulty (Dowling et al. 2004; El Ansari et al. 2001; Winters et al. 2016), and research linking iKT to more intermediate and distal outcomes (e.g., research uptake) is considered an important next step (Gagliardi et al. 2017). Given how crucial relationships are to the success of iKT, qualitative data of the research process (e.g., reflective notes, observations, interviews) are needed for evaluating the processes inherent to partnership research (Sarkies et al. 2017; Waterman et al. 2001). Taking a time-series approach to evaluation in order to identify specific activities and key events that helped to develop the research–KU partnership has been recommended as a way to better evaluate the iKT process (Gagliardi et al. 2016). Regardless of the methods, our finding of a lack of iKT evaluation is shared by others (Camden et al. 2015; Gagliardi et al. 2016).

Finally, as has been noted in other reviews of iKT research (Gagliardi et al. 2016), poor reporting on KU engagement represented an important gap in the current research evidence, as it was difficult to determine what methods were used to involve KUs in the research and how they contributed across stages of the research process. Despite many articles providing overviews or descriptions of iKT activities, few related how KUs were engaged to outcomes of interest such as process outcomes (e.g., stakeholder satisfaction, future collaborations) or health outcomes (Kothari et al. 2014a; Wathen et al. 2011). Moreover, few reported on factors that have been identified as important to iKT, such as if there were existing relationships with KUs, whether they influenced methodology evolution, if they were familiar with iKT and saw value in it, whether trust and mutual learning developed and how arising issues were managed (Bhattacharyya et al. 2013; Bowen and Graham 2013; Sibbald et al. 2014). Additionally, despite the importance of both individual and organizational contextual factors in enabling iKT (Gagliardi et al. 2014), individual characteristics such as knowledge, attitude and motivation were not described. It is important for iKT researchers moving forward to share how they attempted to address these factors and what strategies were successful in order to build best practices (Gagliardi et al. 2014; 2017; Kothari and Wathen 2017).

While not all of the studies included in our review reported in detail on their iKT processes and activities, our findings align with recent work in this area. Extracting information on the iKT process was challenging, and it remains unclear to what extent the iKT process actually achieves its intended outcomes (Gagliardi et al. 2016). They also reported that despite the assumption that a lack of funding impeded the iKT process, working with KU funders did not appear to reduce barriers or improve outcomes relative to study without this funding dynamic (Gagliardi et al. 2016). In one case we examined, this funding dynamic was a challenge, as the funder already had an idea of the research goals – a situation that might be a bit more collaborative if neither party held the metaphorical purse strings (Bullock et al. 2010). While only one of our studies noted the challenge of identifying the right stakeholder (Henderson et al. 2014), Camden et al. (2015) reports on both targeted and open strategies for recruiting stakeholders and that it was easier when they were formally affiliated with an organization and/or had a clear job description. The authors note power sharing within these relationships, and particularly researchers relinquishing power to make others feel more empowered to participate. Both our study and Camden et al. (2015) also stress the importance of an up-front negotiation process, such that a clear, shared understanding of the research is set; this includes rewriting scientific materials so they are more readily understood and that different KU groups might have different needs and understandings of others' needs. All of our studies noted that there is a lack of evaluation data on iKT (Camden et al. 2015; Gagliardi et al. 2016). Where our study adds to the literature is the observation that "overview" programs seemed to be better equipped to support iKT than one-off projects and, as a result, provided more detailed reports of plans for engagement and relationship structure, enabling factors as well as the long-term implications of these relationships (Bornstein et al. 2017; Bullock et al. 2010; Henderson et al. 2014; Naqshbandi Hayward et al. 2016; Wathen et al. 2011). Our review is also the only one to not just describe challenges, but also how teams overcame them.

Our work also aligns with broader iKT literature. For instance, the iKT facilitators noted that the studies we reviewed lend support for a recent conceptual framework to assess organizational capacity for iKT (Gagliardi and Dobrow 2016), including dedicated support resources for iKT, enabling opportunities for interaction between researchers and KUs and time for iKT activities and relationship-building. In particular, our findings highlight the importance of the process of conducting iKT: if capacity to develop, build and maintain relationships is absent, iKT initiatives will likely falter and eventually fizzle out. Future research could further explore how relationships that support iKT are developed, and how to enable knowledge systems where KUs can come to researchers with their questions as needed (Gagliardi et al. 2017; Kothari and Wathen 2017). A KU-initiated research is much more likely to result in benefits beyond research project objectives (e.g., sustained change) compared to researcher-driven, or even co-produced research questions (Bornstein et al. 2017; Bush et al. 2017).

There are some limitations to our work. While we decided to focus on iKT given its prevalence in the Canadian KT community, we did not capture studies employing similar approaches (e.g., community-based participatory research, Engaged Scholarship, T2 research). Thus, while we have tried to draw from this body of literature to help contextualize our findings, we may have missed key insights on involving PDMs in public health research. Our intention with this work was to explore the operationalization of the concept of iKT within the Canadian context, particularly given the emphasis that CIHR has placed on iKT research and the variable quality of these relationships in Canada (Sibbald et al. 2014). While our low capture of international literature may limit the applicability of these findings in other jurisdictions, many of our findings are consistent with international literature on partnership in research, including the importance of partnership management (Greenhalgh et al. 2016) and the time required to develop these (Rycroft-Malone et al. 2016), and clarifying contributions from KUs (Kitson et al. 2013).

Second, given the small number of included studies, the minority of which provided rich descriptions of their iKT workings, our conclusions are largely based on a few richly described studies (notably Bullock et al. 2010; Henderson et al. 2014; McGrath et al. 2009; Wathen et al. 2011) from the perspective of researchers and research teams. It is unclear how much of PDMs' perspectives are reflected in these reports, particularly their perspectives on the engagement process. Nonetheless, the findings provide important foundational evidence regarding the current state of iKT in public health literature while drawing attention to the "black box" of researcher–policy maker partnerships. iKT is not a singular action, and given the centrality of partnerships within iKT, the mechanics of relationship-building and maintenance must be elucidated to enable critical appraisal and advance the state of the art.

Finally, we did not consult experts in the field with our findings after completing the scoping review. While consultation with research consumers can be an optional final step in scoping reviews (Arksey and O'Malley 2005), others believe it should be a requirement (Levac et al. 2010). This would have further strengthened our methodological rigour, as it would have enabled public health PDMs to reflect on their own experiences with iKT research and determine if our findings reflected their experience – and presumably similar experiences of other PDMs.

Conclusions

Our scoping review of PDM involvement in public health iKT research resulted in a few conclusions of note. First, overall vague reporting around the contributions of KUs in general, and the roles of PDMs in particular, made it impossible to determine the role of PDMs relative to other KUs involved with the research. Thus, it remains unclear what PDMs contribute to public health iKT research relative to other types of KUs, and how this particular relationship can be better supported. While the general principles we have synthesized are likely to strengthen the iKT process (e.g., meaningful engagement, flexible opportunities, developing trust), it is up to each team to determine the optimal engagement process. In this way, our work aligns with the broader partnership literature in that investing in the process of relationship-building and management is critical to developing and achieving shared goals.

Our findings also indicate that "overview" programs of research typically appear to have more dedicated resources for supporting iKT development and are thus more successful at building meaningful relationships with KUs. The majority of challenges that iKT teams experience can be overcome through patience and creativity. Future research on this topic should include detailed descriptions of how partnerships are built and maintained, and how challenges such as identifying the "right" KUs and navigating conflicting priorities are addressed. For example, given that public health issues often have a variety of stakeholders at different levels, future iKT research should consider sharing how they dealt with the logistical and representational challenges of managing diverse groups of KUs. Not all partners have an equal stake in the research, and successfully managing the differences between stakeholders is important for aligning the strengths of each relationship towards shared goals.

Application des connaissances intégrée auprès des décideurs en santé publique : un examen de la portée de la littérature

Résumé

L'application des connaissances intégrée (ACi) est une approche qui met à contribution les utilisateurs des connaissances (p. ex., responsables de politiques, cliniciens, patients) en tant que participants actifs au processus de recherche. Théoriquement, ce type de participation accroît la pertinence et l'utilité de la recherche, ce qui favorise le changement dans le système de santé. Toutefois, il y a un manque de données pour étayer les pratiques exemplaires en ACi, en particulier dans le contexte de la santé publique ainsi qu'auprès des décideurs des milieux autres que cliniques. L'objectif de cet examen est, dans un premier lieu, de rendre compte de la façon dont est décrite la participation des décideurs dans les recherches qui mettent en pratique l'ACi et, ensuite, de voir dans quelle mesure le processus a été évalué. Nous avons mené un examen de la portée de la littérature publiée entre janvier 2005 et décembre 2017, puis nous avons extrait les renseignements liés à la participation, aux obstacles, aux facteurs favorables et aux résultats de l'ACi. Les études considérées ne font pas nécessairement la distinction entre les divers types d'utilisateurs des connaissances, ce qui rend impossible un compte rendu sur la participation particulière des décideurs. Les auteurs estiment que la participation des utilisateurs des connaissances présente un avantage pour la qualité et l'impact potentiel des activités de recherche, bien que les données venant corroborer ce fait ne soient pas disponibles. De vaste partenariats multi-projets réunissant chercheurs et utilisateurs des connaissances, de même qu'une flexibilité dans la participation des utilisateurs des connaissances, permettraient d'améliorer l'engagement et d'appuyer les processus de l'ACi.

Comments

Be the first to comment on this!