Commentary: Whose Views Count in Evidence Synthesis? And When Do They Count?
Four recent pieces in Healthcare Policy reveal some disagreement on when and how to involve decision-makers in the process of evidence synthesis. This commentary proposes varying roles for researchers versus managers or policy makers at each of three different stages of synthesis and at the actual point of decision. It also raises the issue of how poorly current processes accommodate the broader conception of evidence held by most managers and policy makers.
[To view the French abstract, please scroll down.]
Including my own article in the last issue, Healthcare Policy has now published four views on research synthesis for managers and policy makers. As Lavis and colleagues (2005) point out, there is much upon which we agree. For instance, there is no argument that the task, while demanding transparency and bias reduction, is different from that of summarizing research for clinicians; or that context is crucial in determining ultimate applicability; or that the questions are about more than "what works." However, the fun of sorting out the issue of synthesis for managers and policy makers lies in debating the differences - in working through the challenge of the best way to improve evidence-informed decision-making.
How much should researchers compromise in their conception of "evidence"? And how much should decision-makers compromise in theirs, when it comes to synthesizing evidence for decision-making?
On this score, there appears to be some disagreement among the four authors published in Healthcare Policy, particularly about the relative roles for researchers on the one hand and managers or policy makers on the other. In my earlier article (Lomas 2005), I saw them as equal partners in a co-production role throughout the process. Lavis and colleagues (2005) - and, for a knowledge rather than decision-support synthesis, Pope et al. (2005) - seem to see decision-makers as adjunct input to a researcher-dominated exercise. Greenhalgh and Russell (2005) put policy makers in the driver's seat, opening an avenue for researchers' input, while decision-makers control the traffic lights at all the major junctions.
We have probably all been guilty of too much shorthand on this. In all likelihood, the relative roles of researchers and decision-makers (whether managers or policy makers) should change with the stage of the process.
At the initial stage of summarizing the research - the systematic review stage or knowledge support synthesis - the researcher plays the lead role with a lot of help from the decision-maker in formulating (and potentially re-formulating) the question. At the stage of extracting implications from the summarized research - defining the key general messages - the researcher still takes the lead, but is aided by the decision-maker. At the stage of creating recommendations for policy or management - advising on action for a specific context - the decision-maker takes the lead, tempered by the researcher's caution around evidence. Finally, the manager or policy maker must actually make the decision - alone, but with help from whatever "dialogue," "argumentation" or other political processes are used.
An illustration of this approach at work is provided by a synthesis process that was successfully concluded recently by a Quebec research collective using very similar relative roles as those described above through the different stages (Pineault et al. 2005). This process recognizes that summing up the research evidence is more than a checklist exercise and requires interpretation, largely by researchers. But it also recognizes that, coming from the other direction, there is interpretation by decision-makers as they sum up the relevant "colloquial evidence" from their context (Lomas et al. 2005). Evidence-informed decision-making is finding a way to synthesize the two forms of evidence - "science" from the researchers and "colloquial knowledge" from the decision-makers.
Researchers and decision-makers have to meet halfway for this task in what Greenhalgh and Russell (2005) describe as "a new rationality of policy-making … in which the skills of argumentation are acknowledged, promoted and reflected upon rather than dismissed as underhand, biased or 'anecdotal.' " The compromise on what counts as "evidence" for the synthesis cannot all be on the side of the decision-maker; researchers' evidence can inform but should not determine the decision. Perhaps the way forward is to find a way for decision-makers' evidence to be incorporated into science - the "new rationality" - rather than our historical drive to fit science into decision- making.
De quelles opinions tient-on compte dans la synthèse des preuves? Et quand en tient-on compte?
Quatre récents articles publiés dans Politiques de santé révèlent un certain désaccord quant au moment et à la façon d'amener les décideurs à participer à la synthèse des preuves. Au lieu de faire participer les gestionnaires ou les décideurs, ce commentaire propose de varier les rôles joués par les chercheurs à trois différentes étapes de la synthèse et au moment de la décision elle-même. Il soulève aussi la question de l'incapacité des processus actuels de tenir compte de la conception plus vaste des preuves qu'ont la plupart des gestionnaires et décideurs.
About the Author(s)
Chief Executive Officer
Canadian Health Services Research Foundation
AcknowledgmentCorrespondence may be directed to: Jonathan Lomas, Chief Executive Officer, Canadian Health Services Research Foundation, 1565 Carling Avenue, Suite 700, Ottawa, Ontario K1Z 8R1; tel. 613-728-2238; email: email@example.com.
Greenhalgh, T. and J. Russell. 2005. "Reframing Evidence Synthesis As Rhetorical Action in the Policy Making Drama." Healthcare Policy 1(2): 34-42.
Lavis, J., H. Davies, R. Gruen, K. Walshe and C. Farquhar. 2005. "Working Within and Beyond the Cochrane Collaboration to Make Systematic Reviews More Useful to Healthcare Managers and Policy Makers." Healthcare Policy 1(2): 21-33.
Lomas, J. 2005. "Using Research to Inform Healthcare Managers' and Policy Makers' Questions: From Summative to Interpretive Synthesis." Healthcare Policy 1(1): 55-71.
Lomas, J., T. Culyer, C. McCutcheon, L. McAuley and S. Law. 2005. "Conceptualizing and Combining Evidence for Health System Guidance." Retrieved December 6, 2005. < http://www.chsrf.ca/other_documents/pdf/
Pineault, R., D. Tousignant, P. Roberge, P. Lamarche, D. Reinharz, D. Larouche, G. Beaulne and D. Lesage. 2005. "Research Collective on the Organization of Primary Care Services in Quebec." Summary report. Retrieved December 6, 2005. < http://www.chsrf.ca/research_themes/pdf/
Pope, C., N. Mays and J. Popay. 2005. "Informing Policy Making and Management in Healthcare: The Place for Synthesis." Healthcare Policy 1(2): 43-48.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed