Healthcare Policy

Healthcare Policy 1(2) January 2006 : 43-48.doi:10.12927/hcpol.2006.17874
Perspectives on Evidence, Synthesis and Decision-Making

Informing Policy Making and Management in Healthcare: The Place for Synthesis

Catherine Pope, Nicholas Mays and Jennie Popay

Abstract

Research synthesis has an important role supporting the transfer of knowledge between researchers and healthcare decision-makers. But if our goal is to make knowledge more useable and context specific, then extending the scope of systematic reviews or producing syntheses with policy makers and managers may be insufficient. Dialogues, partnerships and reinterpretations of evidence in context will help us achieve this goal.

[To view the French abstract, please scroll down.]

Lomas (2005) and Lavis et al. (2005) provide a timely contribution to debates about how to make research evidence available and useful to healthcare managers and policy makers. Both papers argue that the relatively well-developed methods for systematic reviewing - used, for example, within the Cochrane Collaboration - do not address the "broader contextual factors of the managers' and policy makers' world" (Lomas 2005: 59). Lavis et al. suggest that managers and policy makers ask more complex questions that go beyond "What works?" (i.e., questions of effectiveness) and include questions about relationships, mechanisms and meaning. Lavis et al. call for a better repository of knowledge, in essence adapting and extending the Cochrane Library's systematic reviews, to make it more suited to the needs of policy makers and managers. Lomas adds a further twist to this argument by suggesting that the synthesis of research knowledge for policy makers and managers requires different kinds of input (notably, from interpretive social science) and a closer partnership between researchers and managers/policy makers.

So, What's Wrong with Systematic Reviews?

Before we rush into a program of synthesis, it is worth pointing out that many synthesis methods are emerging - the techniques (how to do it) and definitions (what it is) are being developed (Mays et al. 2005; Dixon-Woods et al. 2005). Lomas differentiates summative and interpretive approaches to synthesis. He argues that summative approaches centre on questions of effectiveness, while interpretive approaches are more closely allied to the needs of policy makers and managers. Unfortunately, this implies that policy makers and managers do not need summative accounts. We would suggest that effectiveness reviews may be a necessary, but not sufficient, aid to policy makers and management (they still need to know "what works"). Moreover, there are examples of Cochrane-style systematic reviews aimed at exactly the broad, complex policy-type questions that both Lomas and Lavis et al. identify (Garcia et al. 2002; Harden et al. 2004). There is a danger, in overstating the distinction between summative and interpretive approaches, that we ignore the usefulness of particular types of review (i.e., systematic reviews of effectiveness) and the presence of interpretive work within existing systematic reviews.

For us a more important distinction, inspired by the pioneering work of Jack Dowie (2001, 2002), is the difference between reviews or syntheses providing knowledge support and those providing decision support. Lavis et al. focus on the problem of providing knowledge support. They argue that methods used by the Cochrane Collaboration need to be extended to make reviews more generalizable in order to answer the kinds of questions that policy makers and managers ask. We would further argue that a range of types of review and interpretive and summative syntheses (such as narrative synthesis, meta-ethnography and cross-case analysis, described in Mays et al. 2005) have the potential to provide the kinds of knowledge support that Lavis et al. recommend.

The decision-support approach is distinct from knowledge support because it seeks to go beyond research synthesis and to take on some of the tasks entailed in the decision-making process, for example, incorporating weightings that represent values or judgments. Summative and interpretive approaches might have a place early in this process, as in a literature review of qualitative and quantitative studies to make the inferences that inform a Bayesian analysis. In an example of this approach, findings from qualitative research about parents' reasons for having their children immunized (or not) were used to inform a statistical analysis of the factors that influenced this behaviour (Roberts et al. 2002). For policy makers and managers, this approach provided a more comprehensive picture of the potentially important factors than would have been available if only effectiveness data had been included.

Partnerships

Both Lomas (2005) and Lavis et al. (2005) make compelling arguments for moving away from researcher-driven knowledge translation towards co-production of knowledge and a partnership between managers/policy makers and researchers. At present, policy makers and managers are accused of failing to utilize research evidence. This charge gives rise to an impression that they exist in an evidence-free vacuum. Clearly, managers and policy makers do use evidence. What they don't do so often is use the particularized kinds of evidence that some researchers recognize (e.g., systematic reviews). As Burns points out (2005: 53), these decision-makers have little tradition of using the library. But they utilize intelligence from a variety of sources, including formal research evidence, albeit in an adapted form. One of the reasons for this is that researchers often do not provide evidence that is timely and accessible to policy makers and managers (Popay in press).

Some of the current difficulties in getting knowledge into policy and management practice relate to presentation. At a very basic level, there is a strong case for "jargon-busting" - avoiding discipline- or profession-specific terminology and acronyms, and encouraging the use of "plain English" or French (e.g., the Plain English Campaign) to get our messages across. Lavis et al. champion the 1:3:25 report format pioneered by the Canadian Health Services Research Foundation (CHSRF). This format has proved useful, but there is a danger in becoming over-prescriptive and assuming that standardized formats are a quick fix for knowledge translation. At the heart of Lomas's paper is a plea for closer relationships between policy making and management. Reports in 1:3:25 format, or a larger Cochrane Library with policy-relevant add-on reviews, will not deliver this. We need to think about making the dialogue between researchers and policy makers/managers work (Elliott and Popay 2000). Inevitably, this dialogue will consider making and re-making partnerships in local contexts. One way might be the process adopted by the CHSRF Policy Synthesis Program (CHSRF 2000) whereby researchers and policy makers/managers meet to discuss the content and format of reviews and syntheses. These kinds of critical conversations help to establish what it is that both sides want from the partnership.

There are emerging methods for synthesis that can contribute to the dialogue between research and policy making and management. Synthesis can promote knowledge transfer, but it is not simply an advance on other kinds of literature reviewing; rather, it is a key aspect of this broader activity. Some synthesis approaches allow the inclusion of forms of evidence, such as qualitative research, which have previously been considered too small-scale or too contextualized to inform policy making or management. Others have the potential to become mixed-method approaches, enabling the inclusion of evidence from qualitative and quantitative research and from non-research sources.

Decision support requires a different kind of engagement. It is likely that any synthesis or review would require serious adaptation to meet the demands of decision-makers. It may not be possible to use existing reviews or syntheses for this purpose. In many ways, decision support requires an even closer partnership between research and policy making or management.

Challenges

There are issues that neither Lomas (2005) nor Lavis et al. (2005) address about who should engage in this business of knowledge translation. We need to recognize the different skills required for different approaches - summative or interpretive, knowledge or decision support. The development of transparent, formalized methods for systematic reviewing has enabled researchers from a variety of backgrounds (clinical/non-clinical, research/informatics) to undertake such reviews. Contemporary work developing methods for synthesis suggests that these approaches may require discipline- or methodologically specific expertise (e.g., work on meta-ethnography has highlighted the need for expertise in qualitative methods).

Decision support is quite different from reviewing or synthesis and, again, requires appropriate skills. Engagement with decision-making processes is likely to require input from a team, extending beyond a partnership between researchers and policy makers/managers to include other types of decision-makers, stakeholders and experts.

The Way Ahead

At the heart of the debate about informing policy making and management in healthcare is a paradox: the more we attempt to make knowledge useable and context specific, the more difficult it becomes to draw on a repertoire of reviews or a stock of knowledge. Both Lomas (2005) and Lavis et al. (2005) emphasize that, in the business of policy making and management, context matters. What they are both arguing for in their different ways - Lavis et al. with their extended version of the Cochrane Collaboration and Lomas with his call for co-produced research synthesis - is for knowledge that is relevant. Ultimately, there may well be a place for new forms of research synthesis, as well as for systematic reviews, in informing management and policy, but local partnerships, critical dialogues and reinterpretation in context will be what make a difference in the world of healthcare management and policy making.


 

Informer l'élaboration de politiques et la gestion dans les soins de santé : la place de la synthèse

Résumé

La synthèse de recherche a un important rôle de soutien à jouer dans le transfert des connaissances entre les chercheurs et les décideurs du domaine des soins de santé. Toutefois, si nous voulons rendre les connaissances plus utilisables et plus spécifiques au contexte, l'élargissement de la portée des examens systématiques ou la production de synthèses en collaboration avec les décideurs et les gestionnaires peuvent ne pas suffire. Les dialogues, les partenariats et la réinterprétation des preuves en contexte nous aideront à atteindre ce but.

About the Author

Catherine Pope, PHD
Reader in Health Services Research, School of Nursing & Midwifery
University of Southampton, England

Nicholas Mays, FFPH
Professor of Health Policy, Health Services Research Unit
Department of Public Health and Policy
London School of Hygiene and Tropical Medicine
London, England

Jennie Popay, BA, MA
Professor of Sociology and Public Health
Institute for Health Research
University of Lancaster, England

Acknowledgment

Correspondence may be directed to: Catherine Pope, Reader in Health Services Research, School of Nursing & Midwifery, University of Southampton, Highfield, Southampton SO17 1BJ, United Kingdom; tel.: +44 (0)2380 598293; fax: +44 (0)2380 598308; email: cjp@soton.ac.uk.

References

Burns, A. 2005. "Recognising the Needs and Roles of Key Actors." Journal of Health Services Research and Policy 10 (Supplement 1): 53-54.

Canadian Health Services Research Foundation (CHSRF). 2000. < http://www.chsrf.ca >.

Dixon-Woods, M., S. Agarwal, D. Jones, B. Young and A. Sutton. 2005. "Synthesising Qualitative and Quantitative Evidence: A Review of Possible Methods." Journal of Health Services Research and Policy 10: 45-53.

Dowie, J. 2002. "Research Implications of Science-Informed, Value-Based Decision Making." European Journal of Oncology Library 2: 1-9.

Dowie, J. 2001. "Towards Value-Based, Science-Informed Public Health Policy: Conceptual Framework and Practical Guidelines." Background paper for The World Health Report 2002. Geneva: World Health Organization.

Elliott, H. and J. Popay. 2000. "How Are Policy Makers Using Evidence? Models of Research Utilisation and Local NHS Policymaking." Journal of Epidemiology and Community Health 54(6): 461-468.

Garcia, J., L. Bricker, J. Henderson, M.-A. Martin, M. Mugford, J. Nielson and T. Roberts. 2002. "Women's Views of Pregnancy Ultrasound: A Systematic Review." Birth 29: 225-250.

Harden, A., J. Garcia, S. Oliver, R. Rees, J. Shepherd, G. Brunton and A. Oakley. 2004. "Applying Systematic Review Methods to Studies of People's Views: An Example from Public Health Research." Journal of Epidemiology and Community Health 58: 794-800.

Lavis, J., H. Davies, R. Gruen, K. Walshe and C. Farquhar. 2005. "Working Within and Beyond the Cochrane Collaboration to Make Systematic Reviews More Useful to Healthcare Managers and Policy Makers." Healthcare Policy 1(2): 21-33.

Lomas, J. 2005. "Using Research to Inform Healthcare Managers' and Policy Makers' Questions: From Summative to Interpretive Synthesis." Healthcare Policy 1(1): 55-71.

Mays, N., C. Pope and J. Popay. 2005. "Systematically Reviewing Qualitative and Quantitative Evidence to Inform Management and Policy-Making in the Health Field." Journal of Health Services Research and Policy 10 (Supplement 1): 6-20.

Plain English Campaign. No date. Retrieved December 8, 2005. < http://www.plainenglish.co.uk/
introduction.html#Anchor-Wha-35253
>.

Popay, J. (ed.). In press. Moving Beyond Effectiveness: Methodological Issues in the Synthesis of Diverse Sources of Evidence. London, UK: National Institute for Health & Clinical Excellence.

Roberts, K.A., M. Dixon-Woods, R. Fitzpatrick, K.R. Abrams and D.R. Jones. 2002. "Factors Affecting the Uptake of Childhood Immunisation: A Bayesian Synthesis of Qualitative and Quantitative Evidence." Lancet 360: 1596-1599.

Comments

Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed