Healthcare Quarterly, 15(Special Issue) December 2012: 58-62.doi:10.12927/hcq.2013.23148
Linking Evidence And Quality
Supporting the Use of Research Evidence in the Canadian Health Sector
Interventions to support evidence-informed decision making have increased in recent years, but they are often fragmented across different clinical, management and policy environments. Many of these efforts also place varying emphasis on supporting the use of research evidence, with some choosing to focus more on expert knowledge and/or media coverage and others focusing on supporting the use of actionable messages arising from high-quality, relevant and optimally packaged research evidence. In this paper, we profile five Canadian contributions – EvidenceUpdates, Rx for Change, Health-Evidence.ca, Health Systems Evidence and the McMaster Health Forum – that allow providers, managers and policy makers to efficiently find and use research evidence when they need it. These contributions are critical for supporting both local and global efforts to provide optimal and cost-effective care, improving the quality of care and strengthening health systems.
Research evidence is an important input into decision making for both healthcare providers and for health system managers and policy makers. Research evidence can inform decisions about which programs, services and drugs to provide as well as decisions both about health systems (i.e., strengthening or reforming health system governance, financial and delivery arrangements within which programs, services and drugs are provided) and within health systems (i.e., how to get cost-effective programs, services and drugs to those who need them) (Lavis et al. 2010).
Notwithstanding this potential, there are notable examples of research evidence not being used (or used inconsistently) and/or decisions and recommendations being made that do not reflect the conclusions of high-quality research evidence. For example, in clinical practice, studies have found significant deficits in adherence to recommended care processes (McGlynn et al. 2003; Schuster et al. 1998) as well as prescribing practices (Shrank et al. 2006). At the level of policy making, an examination of the use of research evidence in recommendations made by World Health Organization (WHO) departments found that the development of recommendations rarely drew upon systematic reviews and concise summaries of findings (despite WHO guidelines emphasizing the use of systematic reviews) (Oxman et al. 2007). Similarly, a review of recommendations made by the WHO and the World Bank in five health-related policy domains found that of the eight publications examined only two cited systematic reviews, and of the 14 recommendations made only five were consistent with both the direction and nature of findings from systematic reviews of effects (Hoffman et al. 2009).
In Ontario, the need to inform decisions about patient care and strengthening the health system using the best available research evidence (thereby avoiding situations as outlined above) has been made explicit in The Excellent Care for All Act, 2010 that was proclaimed in June 2010 (Ministry of Health and Long-Term Care [MOHLTC] 2012a, 2012b). Indeed, the MOHLTC has provided clear signals that it is prioritizing the use of research evidence to inform the development of policy by requiring training for civil servants in finding and using research evidence, incorporating assessments of the use of research evidence as part of performance reviews, and mandating the use of a "Research Evidence Tool" that requires civil servants making submissions to the minister or to cabinet to explicitly document the key sources for research evidence that were searched and declare that relevant findings were used to inform the submission. However, such internal "user-pull" efforts need to draw on broader efforts that allow providers, managers and policy makers to efficiently find and use research evidence when they need it.
The field of knowledge translation attempts to support the use of research evidence to inform practice, management and policy. There are many terms used to describe the same or similar processes (Graham et al. 2007; Straus et al. 2009), with the terms "implementation science" and "research utilization" often used in Europe, and "diffusion" and "dissemination" commonly used in the United States (Straus and Haynes 2009). A cross-sectional study using data from 2006 documented the number and frequency of terms used in 12 healthcare journals to describe knowledge translation and found that 100 different terms were used across the 581 articles that described knowledge translation research (McKibbon et al. 2010).
Despite the diverse terms used, the field of knowledge translation is focused on moving beyond the passive dissemination of research evidence to more effectively supporting its use (Straus et al. 2009). The field faces four important challenges in doing so: (1) research evidence competes with many other factors in decision-making processes; (2) providers, managers and policy makers may not value research evidence as an input to decision-making processes; (3) the available research evidence may not be relevant to the issues or context at hand; and (4) research evidence is not always easy to use (Lavis et al. 2006). While efforts to address these challenges through knowledge translation interventions have increased in recent years, they are often fragmented across different clinical, management and policy environments. In addition, many efforts place varying emphasis on supporting the use of research evidence, with some choosing to focus more on expert knowledge and/or media coverage (EvidenceNetwork.ca 2012; HealthyDebate 2012) and others focusing more on supporting the use of actionable messages arising from high-quality, relevant and optimally packaged research evidence (Straus and Haynes 2009).
Increasingly, efforts that have a focus on supporting the use of research evidence (as opposed to expert opinion or other forms of evidence) draw on systematic reviews (or summaries of systematic reviews), given the reduced bias and increased precision achieved by synthesizing the global pool of evidence about a particular topic. In addition, systematic reviews (and especially summaries of reviews) constitute a much more efficient use of time for busy healthcare providers, managers and policy makers, given that all of the studies have already been identified, quality appraised and synthesized in one document. We profile below several Canadian contributions to supporting evidence-informed practice, management and policy, both locally (e.g., toward the focus of this special issue – policy development to build a culture of quality in Ontario's hospitals as part of the Excellent Care for All Act) and globally (e.g., toward developing global guidelines to support evidence-informed policies about health systems).
Efforts toward Supporting Research Use by Healthcare Providers
Three examples of comprehensive knowledge translation efforts for providers as well as health professional leaders and managers engaged in supporting evidence-informed practice are focused, respectively, on providing access to research evidence (systematic reviews and primary research) to support evidence-based clinical decisions (BMJ Evidence Centre 2012; Haynes 2005), systematic reviews focused on bringing about behaviour change in prescribing and medicines use (Canadian Agency for Drugs and Technologies in Health 2012; Weir et al. 2010) and systematic reviews about public health interventions (Dobbins et al. 2010; Health-Evidence 2012). First, EvidenceUpdates (http://plus.mcmaster.ca/EvidenceUpdates), an initiative of the BMJ Group and McMaster University's Health Information Research Unit, provides a searchable database of high-quality research evidence, an e-mail alerting system and key evidence-based resources such as synopses and summaries of research evidence. The citations in EvidenceUpdates are identified from 120 premier clinical journals, quality appraised and then rated for clinical relevance and interest by at least three members of a worldwide panel of practising physicians. The alerting system allows users to receive periodic updates to citations meeting minimum levels of clinical relevance in their areas of clinical interest (e.g., primary care or internal medicine). As Haynes (2005) indicates, the EvidenceUpdates service allows users to easily keep up-to-date with "need to know" studies and reviews by reducing approximately 50,000 articles per year in approximately 120 premier clinical journals to the most salient one to two articles per month (or 12 to 24 per year), which amounts to a substantial noise reduction of 99.96% (Haynes 2005).
The second resource, Rx for Change (http://www.rxforchange.ca), provides a comprehensive repository of systematic reviews evaluating the effectiveness of interventions to change clinical practice to support evidence-based prescribing and medicines use (Weir et al. 2010). Rx for Change is primarily intended for those making decisions about which interventions to include in quality improvement programs (e.g., health professional leaders or managers). The database was created to make it easier for these decision makers to access, assemble and assess research evidence in this domain, given the large volume, wide dispersion and variable quality of the available research evidence. In addition to ensuring the database remains up-to-date and comprehensive, Rx for Change includes several features that distinguish it as a unique knowledge translation tool, including (1) categorizing reviews according to whether they evaluate interventions directed at consumers or professionals; (2) quality appraising each eligible review using the AMSTAR tool (Shea e al. 2007) and only including reviews meeting a minimum quality standard; (3) providing user-friendly one-page summaries highlighting the study characteristics, key findings and conclusions; (4) providing summaries of overall findings from reviews about each grouping of interventions (e.g., audit and feedback, computerized reminders, and so forth); and (5) providing a table of results from the individual studies (including links to each study).
Lastly, Health-Evidence.ca provides a comprehensive repository of systematic reviews of the effectiveness of public and population health interventions, accompanied by e-mail updates that periodically alert users to new reviews in their areas of interest (Health-Evidence 2012). The database aims to support evidence-informed decision making in public health organizations, and therefore its primary target audience includes both providers (e.g., public health nurses, outreach workers, and so forth) and managers and policy makers responsible for making decisions related to public health (Dobbins et al. 2010). Health-Evidence.ca allows users to search for systematic reviews using their comprehensive taxonomy of topics related to public health, and each record provides a quality appraisal score as well as an outline of the review focus, type of review, intervention studied, population characteristics and intervention strategy. Lastly, brief summaries of the key findings from some of the systematic reviews contained in the database are provided.
Efforts toward Supporting Research Use by Health System Managers and Policy Makers
Increasingly, efforts to support linking research to policy strive to address the two factors that emerged with some consistency in systematic reviews of factors influencing research use by health system managers and policy makers, which include the timing and timeliness of research evidence being made available and interactions between researchers and policy makers (Innvaer et al. 2002; Lavis et al. 2005a, 2005b). A key strategy for addressing the former involves facilitating the retrieval of optimally packaged, high-quality and high-relevance systematic reviews, while for the latter, engaging policy makers and stakeholders in deliberative dialogues has emerged as a key strategy.
To facilitate the timely retrieval of research evidence, Health Systems Evidence (www.healthsystemsevidence.org) provides the world's most comprehensive and continuously updated repository of research evidence about governance, financial and delivery arrangements within health systems, and about implementation strategies that can support change in health systems (many of which are drawn from Rx for Change) (McMaster Health Forum 2012). Where once the supply of systematic reviews addressing these types of questions seemed very limited, Health Systems Evidence now (as of July 2012) includes 54 review-derived products (30 evidence briefs for policy and 24 overviews of systematic reviews), 1,590 systematic reviews of effects (including 416 Cochrane reviews), 284 systematic reviews addressing other types of questions and 218 systematic reviews in-progress. The database also contains a continuously updated repository of economic evaluations related to health system arrangements and implementation strategies, descriptions of health system reforms and descriptions of health systems. In addition, Health Systems Evidence contains a number of features designed to help policy makers and stakeholders efficiently find and use research evidence. These features include links to independently produced user-friendly summaries (where available), scientific abstracts and full-text reports (when publicly available); quality appraisal scores for systematic reviews (using the AMSTAR tool) (Shea et al. 2007); and listings of the countries in which the studies included in the synthesis were conducted. Health Systems Evidence has also recently incorporated Canada's Evidence-Informed Healthcare Renewal (EIHR) Portal to provide policy makers and stakeholders with a comprehensive inventory of policy-relevant documents that can support healthcare renewal in Canada.
To facilitate interactions between policy makers and researchers, a number of groups (e.g., the McMaster Health Forum in Canada and the Evidence-Informed Policy Networks in Africa, Asia and the Americas) have begun to experiment with convening deliberative dialogues. In Canada, the McMaster Health Forum (www.mcmasterhealthforum.org) convenes stakeholder dialogues with a broad array of policy makers, stakeholders and researchers to work through a pressing health problem, options for addressing it and key implementation considerations. Dialogues at the forum are informed by an evidence brief that draws upon the best available data and research evidence to define the policy problem/issue, identify and describe what is known about possible policy and program options, and identify key implementation considerations for these options.
Deliberative dialogues provide unique support for evidence-informed decision making by fostering the interplay of the best available data and research evidence with the tacit knowledge, views and experiences of those who will be involved in or affected by the issue. The preparation of evidence briefs for deliberative dialogues are also an example of how each of the resources outlined above can be used to efficiently identify the best available data and research evidence about pressing health problems. For example, EvidenceUpdates, Rx for Change and Health Systems Evidence were recently used by the McMaster Health Forum as the primary sources used to identify research evidence to inform an evidence brief as part of the Quality Improvement in Primary Healthcare Project in Ontario (Lavis 2010). Specifically, the evidence brief used each of these resources to mobilize the best available research evidence about (1) the problem faced in supporting quality improvement in primary healthcare in Ontario; and (2) three options for addressing it (collaboratively developing principles for quality improvement, developing coordinating structures and processes to support quality improvement, and scaling-up existing quality improvement initiatives) and implementation considerations. The resulting evidence brief and dialogue summary, like all Forum products, can be downloaded from the Forum website.
Supporting the use of research evidence to inform practice, management and policy has been significantly enhanced by several synergistic efforts to support the use of high-quality, relevant and optimally packaged research evidence. The resources outlined above (i.e., "one-stop shopping" resources for research evidence and deliberative dialogues) are critical for supporting efforts to provide optimal cost-effective care and for making evidence-informed decisions toward strengthening health systems such as those that are the focus of this special issue about moving forward in Ontario with The Excellent Care for All Act.
With respect to strengthening knowledge translation efforts to support evidence-informed practice, management and policy both in Canada and globally, there are several key areas requiring further investigation. First, despite the many promising interventions, there are few rigorous evaluations of knowledge translations interventions, particularly those designed for managers and policy makers (Mitton et al. 2007; Perrier et al 2011). However, randomized controlled trials evaluating two of the databases highlighted here (EvidenceUpdates and Health-Evidence.ca) have found that efforts to provide tailored and targeted messaging about relevant research evidence increased the utilization of evidence-based information (Dobbins et al. 2009; Haynes et al. 2006). Second, while systematic reviews are an important tool for knowledge translation, their utility for providers, managers and policy makers can be improved by including more detailed descriptions of the interventions and ensuring they remain up-to-date. Lastly, there is a need to continue to diversify knowledge translation, which could include clinical support systems that automatically retrieve findings for clinicians (Straus and Haynes 2009), interventions for rapid decision support when research is needed in a timely manner such as rapid response units (Canadian Agency for Drugs and Technologies in Health 2012; Ottawa Hospital Research Institute 2011; The Ontario HIV Treatment Network 2012) or efforts toward developing global guidelines to support evidence-informed policies about health systems (Lavis et al. 2012). Additional efforts could also include training for the science and practice of knowledge translation such those currently being led by KT Canada (Straus et al. 2011).
About the Author
Michael G. Wilson, PhD, is assistant director of the McMaster Health Forum, a member of the Centre for Health Economics and Policy Analysis, McMaster University, an assistant professor (part-time) in the Department of Clinical Epidemiology and Biostatistics, McMaster University and scientist, Health Policy and Knowledge Transfer and Exchange at the Ontario HIV Treatment Network. His research is focused on supporting the use of research evidence in health systems with specific interest in community-based organizations and health system policymakers. Mike also helped lead the creation of Health Systems Evidence (www.healthsystemsevidence.org) and continues to contribute to the continuous updating of it.
John N. Lavis, MD, PhD, is director of the McMaster Health Forum, associate director of the Centre for Health Economics and Policy Analysis, a professor in the Department of Clinical Epidemiology and Biostatistics, and associate member of the Department of Political Science, all at McMaster University. He is also an adjunct professor of Global Health in the Department of Global Health and Population, Harvard School of Public Health. His principal research interests include knowledge transfer and exchange in public policy-making environments and the politics of health systems. John also led the creation and oversees the continuous updating of Health Systems Evidence (www.healthsystemsevidence.org).
Jeremy M. Grimshaw, MBChB, PhD, FRCGP, FCAHS, is a senior scientist, Clinical Epidemiology Program, Ottawa Hospital Research Institute, a full professor in the Department of Medicine, University of Ottawa and a Tier 1 Canada Research Chair in Health Knowledge Transfer and Uptake. He is director of Cochrane Canada and co-ordinating editor of the Cochrane Effective Practice and Organisation of Care group. He is also the principal investigator of Knowledge Translation Canada (KT CANADA), a CIHR and CFI funded interdisciplinary network of over 50 knowledge translation researchers from six academic health science centres in three provinces. His research focuses on the evaluation of interventions to disseminate and implement evidence-based practice. He also led the creation and oversees the continuous updating of Rx for Change (www.rxforchange.ca).
BMJ Evidence Centre. 2012. EvidenceUpdates. Retrieved February 14, 2012. <http://plus.mcmaster.ca/EvidenceUpdates/>.
Canadian Agency for Drugs and Technologies in Health. 2012. Rx for Change. "Retrieved October 24 2012. < http://www.cadth.ca/en/resources/rx-for-change>.
Canadian Agency for Drugs and Technologies in Health. 2012. Rapid Response. Retrieved October 24, 2012. <http://cadth.ca/en/products/rapid-response>.
Dobbins, M., K. DeCorby, P. Robeson, H. Husson, D. Tirilis and L. Greco. 2010. "A Knowledge Management Tool for Public Health: Health-evidence.ca." BMC Public Health 10(1): 496.
Dobbins, M., S. Hanna, D. Ciliska, S. Manske, R. Cameron, S. Mercer et al. 2009. "A Randomized Controlled Trial Evaluating the Impact of Knowledge Translation and Exchange Strategies." Implementation Science 4(1): 61.
EvidenceNetwork.ca. 2012. Retrieved February 13, 2012. <http://evidencenetwork.ca/>.
Graham, I.D., J. Logan, M.B. Harrison, S.E. Straus, J. Tetroe, W. Caswell et al. 2007. "Lost in Translation: Time for a Map?" Journal of Continuing Education in the Health Professions 26(1): 13–24.
Haynes, R.B. 2005. "bmjupdates+, a New FREE Service for Evidence-Based Clinical Practice." Evidence-Based Nursing 8(2): 39.
Haynes, R.B., J. Holland, C. Cotoi, R.J. McKinlay, N.L. Wilczynski, L.A. Walters et al. 2006. "McMaster PLUS: A Cluster Randomized Clinical Trial of an Intervention to Accelerate Clinical Use of Evidence-Based Information from Digital Libraries." Journal of the American Medical Informatics Association 13(6): 593–600.
Health-Evidence. 2012. Retrieved February 14, 2012. <http://www.health-evidence.ca/>.
HealthyDebate. 2012. Retrieved February 12, 2012. <http://healthydebate.ca/>.
Hoffman, S.J., J.N. Lavis and S. Bennett. 2009. "The Use of Research Evidence in Two International Organizations' Recommendations about Health Systems." Healthcare Policy 5(1): 66–86.
Innvaer, S., G.E. Vist, M. Trommald and A.D. Oxman. 2002. "Health Policy-Makers' Perceptions of Their Use of Evidence: A Systematic Review." Journal of Health Services Research and Policy 7(4): 239–44.
Lavis, J.N. 2010. Issue Brief: Supporting Quality Improvement in Primary Healthcare in Ontario. Hamilton, Canada: McMaster Health Forum.
Lavis, J.N., H.T.O. Davies, A.D. Oxman, J.-L. Denis, K. Golden-Biddle and E. Ferlie. 2005a. "Towards Systematic Reviews That Inform Health Care Management and Policy-Making." Journal of Health Services Research and Policy 10(Supplement 1): S1:35–48.
Lavis, J.N., A.C. Hammill, A. Gildiner, R.J. McDonagh, M.G. Wilson, S.E. Ross et al. 2005b. A Systematic Review of the Factors that Influence the Use of Research Evidence by Public Policymakers. Final Report Submitted to the Canadian Population Health Initiative. Hamilton, Canada: McMaster University Program in Policy Decision-Making.
Lavis, J.N., L. Lomas, M. Hamid and N.K. Sewankambo. 2006. "Assessing Country-Level Efforts to Link Research to Action." Bulletin of the World Health Organization 84(8): 620–8.
Lavis, J.N., J.-A. R⊘ttingen, X. Bosch-Capblanch, R.A. Atun, F. El-Jardali, L. Gilson et al. 2012. "Guidance for Evidence-Informed Policies about Health Systems: Linking Guidance Development to Policy Development." PLoS Medicine 9(3): e1001186.
Lavis, J.N., M.G. Wilson, J.M. Grimshaw, R.B. Haynes, M. Ouimet, P. Raina et al. 2010. "Supporting the Use of Health Technology Assessments in Policy Making about Health Systems." International Journal of Technology Assessment in Health Care 26(4): 405–14.
McGlynn, E.A., S.M. Asch, J. Adams, J. Keesey, J. Hicks, A. DeCristofaro et al. 2003. "The Quality of Health Care Delivered to Adults in the United States." New England Journal of Medicine 348(26): 2635–45.
McKibbon, K.A., C. Lokker, N. Wilczynski, D. Ciliska, M. Dobbins, D. Davis et al. 2010. "A Cross-sectional Study of the Number and Frequency of Terms Used to Refer to Knowledge Translation in a Body of Health Literature in 2006: A Tower of Babel?" Implementation Science 5(1): 16.
McMaster Health Forum. 2012. Health Systems Evidence. Retrieved February 14, 2012. <http://www.healthsystemsevidence.org/>.
Ministry of Health and Long-Term Care. 2012a. Excellent Care for All: About the Excellent Care for All Act. Retrieved March 9, 2012. <http://www.health.gov.on.ca/en/ms/ecfa/pro/ecfa_act.aspx>.
Ministry of Health and Long-Term Care. 2012b. Excellent Care for All: Ontario's Excellent Care for All Strategy. Retrieved August 12, 2012. <http://www.health.gov.on.ca/en/ms/ecfa/public/>.
Mitton, C., C.E. Adair, E. McKenzie, S.B. Patten and B. Waye Perry. 2007. "Knowledge Transfer and Exchange: Review and Synthesis of the Literature." Milbank Quarterly 85(4): 729–68.
The Ontario HIV Treatment Network. 2012. Rapid Response Service. Retrieved July 2, 2012. <http://www.ohtn.on.ca/Pages/Knowledge-Exchange/Rapid-Response-Service.aspx>.
Ottawa Hospital Research Institute. 2011. Knowledge to Action. Retrieved October 24, 2012. <http://www.ohri.ca/kta>.
Oxman, A.D., LN. Lavis and A. Fretheim. 2007. "Use of Evidence in WHO Recommendations." The Lancet 369(9576): 1883–9.
Perrier, L., K. Mrklas, J.N. Lavis and S.E. Straus. 2011. "Interventions Encouraging the Use of Systematic Reviews by Health Policymakers and Managers: A Systematic Review." Implementation Science 6(1): 43.
Schuster, M., E. McGlynn and R.H. Brook. 1998. "How Good Is the Quality of Health Care in the United States?" Milbank Quarterly 76(4): 517–63.
Shea, B., J. Grimshaw, G. Wells, M. Boers, N. Andersson, C. Hamel et al. 2007. "Development of AMSTAR: A Measurement Tool to Assess the Methodological Quality of Systematic Reviews." BMC Medical Research Methodology 7(1): 10–6.
Shrank, W.H., S.M. Asch, J. Adams, C. Setodji, E.A. Kerr, J. Keesey et al. 2006. "The Quality of Pharmacologic Care for Adults in the United States." Medical Care 44(10): 936–45.
Straus, S., M. Brouwers, D. Johnson, J. Lavis, F. Legare, S. Majumdar et al. 2011. "Core Competencies in the Science and Practice of Knowledge Translation: Description of a Canadian Strategic Training Initiative." Implementation Science 6(1): 127.
Straus, S. and R.B. Haynes. 2009. "Managing Evidence-Based Knowledge: The Need for Reliable, Relevant and Readable Resources." Canadian Medical Association Journal 180(9): 942–5.
Straus, S.E., J. Tetroe and I. Graham. 2009. "Defining Knowledge Translation." Canadian Medical Association Journal 181(3–4): 165–8.
Weir, M., R. Ryan, A. Mayhew, J. Worswick, N. Santesso, D. Lowe et al. 2010. "The Rx for Change Database: A First-in-Class Tool for Optimal Prescribing and Medicines Use." Implementation Science 5(1): 89.
Be the first to comment on this!Please Sign In or Create an Account to add comments