There has been a strong push over the last decade for health services researchers to become "relevant," to work with policy makers to translate evidence into action. What has been learned from this interaction? The pooled experiences of health services researchers across the country, including those at the Manitoba Centre for Health Policy (MCHP), suggest five key lessons. First, policy makers pay more attention to research findings if they have invested their own funds and time. Second, researchers must make major investments in building relationships with policy makers, because there are inevitable tensions between what the two parties need and do. Third, researchers must be able to figure out and communicate the real meaning of their results. Fourth, health services researchers need a "back-pocket" mindset, as they cannot count on immediate uptake of results; because the issues never go away, evidence, if known and easily retrievable, is likely to have an eventual impact. Finally, getting evidence into the policy process does not come cheaply or easily, but it can be done. The overriding lesson learned by health services researchers is the importance of relationship-building, whether in formalizing contractual relationships, building and maintaining personal trust, having a communications strategy or increasing the involvement of users in the research process.

In the world of health services research, one hopes to see research evidence become action in the form of a new policy, program or decision. Sometimes these hopes are realized. Of course, researchers are well aware that research evidence is only one factor in decision-making - there are also the political realities of the day, economic constraints, lobbyists, habits, traditions and values (Davies 2004; Davies 2005). Sometimes the "tectonic plates" of researchers and decision- makers move slowly past each other with little noticeable change in the landscape for decades. Other times there is a great deal of friction, resulting in major tidal waves or volcanic eruptions on the policy scene, or in the relationships between these two groups. What are the lessons that health services researchers have learned at the interface? How have relationships changed between researchers and decision-makers over time?

In Canada, there are many health services researchers and centres working with policy makers and planners to translate¹ research evidence² into action. This paper is intended to share the collective wisdom of researchers interacting with public policy decision-makers. Some examples are drawn from the experience of the Manitoba Centre for Health Policy (MCHP), a unit within the Department of Community Health Sciences in the University of Manitoba's Faculty of Medicine. Other examples are drawn from leaders in health services research across the country, who were interviewed by telephone, in person or through email by one author (PJM).

Lesson #1: If They Build It, They Will Come³

Many health services research organizations have discovered that policy makers pay more attention to research findings if they have invested their own funds and time. As Noralou Roos, founding director of the Manitoba Centre for Health Policy, states:

In the 20 years before the establishment of MCHP, my colleagues and I functioned as typical academics. Our work was widely published academically, but no one in local organizations or in any position of responsibility at Manitoba Health [the provincial department of health] paid attention to the results. But when Manitoba Health invested its own funds, they began to pay attention to what was actually being produced.

The importance of a long-term contract cannot be underestimated. It allows continuity across staffing changes in key positions within government, or even changes in the government itself. New people, and new governments, require time to understand the value of an arms-length research organization producing publicly released evidence for decision-making. As Stephen Bornstein, director of the Newfoundland and Labrador Centre for Applied Health Research at Memorial University, notes:

There's no memory in government. People in the upper positions (like ADMs or DMs) rarely last more than two years, so you start from scratch each time. I've found that it has less to do with the type of government than it has to do with the person in the role, especially in a smaller province, and whether or not this person can formulate research questions.

Charlyn Black, director of the Centre for Health Services and Policy Research at the University of British Columbia, also comments on looking for the key individuals:

I'd say you have to work with decision-makers who are committed to understanding and working with evidence, and who value evidence enough to make the commitment to work with academics, because the work we do is fundamentally different from the internal issues of government.

An ongoing contractual relationship with government also guarantees stability to maintain a highly skilled and specialized workforce. This is difficult to sustain through fluctuating peer-reviewed grant funding. However, if a health services research centre relies solely upon government funding, it may be perceived and criticized as a paid consultant rather than a rigorous research institute. A track record of external funding from traditional academic sources is essential to a centre's ongoing credibility in the research world. For example, MCHP receives about half its funding from Manitoba Health, with the rest coming from peer-reviewed granting agencies such as the Canadian Institutes of Health Research (CIHR).

Lesson #2: Relationships-The More We Get Together, the Happier We'll Be (or Not)

If policy makers fund health services research, the "tectonic plate" interface can be fraught with friction. It sometimes seems there is no good news in health services research. If, for example, complication rates drop from 10% to 5%, the headlines will scream, "Hospitals injure five out of every 100 patients they admit." Such headlines lead to demands for more funding for the healthcare system and calls for the minister to take action. Inhouse research can be kept internal within governments, but the assumption that the same rules will apply to independent academic research can create problems. How does a researcher maintain an acceptable boundary? Policy makers want rapid responses to queries, research they can use, links to groups that will give their work credibility and avoidance of embarrassment to the government. Researchers want time in which to study a question thoroughly or work on creative new approaches, and the ability to keep their academic integrity and freedom to publish. Researchers know their work must hold up in the "court" of peer review.

It is no wonder, then, that tensions could - and indeed, probably should - arise between researchers and policy makers. So, how does a researcher balance these conflicting pressures?

MCHP has developed some experience in dealing with this situation through a series of contracts with Manitoba Health. After completing one "confidential" project in the early 1990s, MCHP decided not to do future confidential projects and included a clause in its contract stating that all work would become publicly available. After three years of experiencing delays in scheduling a joint release of reports, MCHP further negotiated a clause stating that MCHP can release its report at any time after 60 days of the draft report's delivery to government. During this period, MCHP also briefs stakeholders on the results. While the government, opposition leaders and other stakeholders are given the final report just prior to public release, the news release and four-page summary sheets stay internal to MCHP until release.

Strong relationships between researchers and decision-makers can ease the tectonic plate pressure buildup. Beyond the more formalized contractual relationships, there is the need for ongoing personal relationship-building. As Stephen Bornstein reflects:

Deal with top levels of government on a regular basis, with the "easy stuff." Cultivate the relationship. For example, we have a bi-monthly seminar on research ideas. They talk about what they would like to hear about, and then we put together an information session on this topic. It builds up a sense of trust.

It must be recognized that such relationship-building takes time and commitment on the part of both the researcher and the decision-maker. As Ingrid Sketris, Professor at Dalhousie University and Researcher at IMPART (Initiative for Medication Management, Policy Analysis, Research and Training), observes:

You need ongoing communication, and it takes a long time to build and nurture the relationship. When I don't nurture the relationship - if I've been too busy, or away for several weeks - then there's a greater chance of miscommunication. And newer researchers or students need support in communicating with decision- makers. Decision-makers are not necessarily gentle in their criticism.

Greg Stoddart, founding coordinator of McMaster University's Centre for Health Economics and Policy Analysis, echoes these comments:

The overriding message for health services researchers is that there is no substitute for personal contact. You need to adjust your schedule, make time, meet with people personally, sit and talk. It's pretty tough to get the contacts if you're new. Also, you can't always predict the needs of policy-related research in terms of timing. So you need to adjust your schedule. You may have a great chance to do the research, but you need to get it done by next month!

Sometimes it is difficult to maintain informal personal relationships between researchers and policy makers. Greg Stoddart describes one example of changing a relationship from the personal to the structural:

One great example of "institutionalizing the interaction" is the late Bernie O'Brien's work with McMaster University, St. Joseph's Health Care and the Ministry of Health and Long-Term Care in Ontario. They created the Program for Assessment of Technology in Health (PATH). This provides health technology assessment findings in real time to policy makers, with four or five "deliverables" throughout the year. The information is brought to the table, and the policy makers make the decision. So PATH managed to institutionalize researcher-to-decision-maker relationships with less dependence on the personal contact.

MCHP has a slightly different approach, including regularly scheduled meetings with the deputy minister of health and a high-level bureaucrat identified as the liaison between MCHP and the Ministry of Health. This liaison acts as the official go-between and has effectively produced an "institutionalized" relationship that promotes mutual understanding.

Lesson #3: Don't Let the Message Get "Lost in Translation"

Various theories and frameworks have been described to help demystify knowledge translation (KT) and improve translation of research knowledge (Lyons and Warner 2005; Bowen et al. in press; Lavis et al. 2003). Grimshaw et al. (2004) attest to the importance of relationships and tailored KT activities for each stakeholder group. As John McLaughlin, senior scientist and head of the Program in Epidemiology and Biostatistics at the Samuel Lunenfeld Research Institute, points out:

The "tower of Babel" - we've all climbed it. There are different languages that separate groups like policy makers and researchers use. So we all need to plan and evaluate the medium, the message and the messenger to get our research evidence into decision-making.

Like many other health services research centres, MCHP has taken many different approaches to KT, including full reports and summaries, Web-based materials, presentations and media releases. As Les Roos, Director of the Repository at MCHP, observes:

There are many different layers of research communication - the public, the decision-makers locally, provincially, nationally and internationally, other researchers, practitioners - and each may need a very different mode to translate this research.

The best "mode" for provincial and regional decision-makers may be the face-to-face briefing or the interactive workshop. Pre-release briefings to key policy makers help avoid confusion and misinterpretation of the results. Moreover, it gives them a chance to digest the information, prepare thoughtful responses and be ready to answer the media and the public. As Ingrid Sketris notes, "If you have unexpected or difficult findings, you need to give the decision-makers enough breathing space to deal with them."

Policy making takes place in a public context, and researchers need to be aware of the importance of the public "sound bite." What is the essence of the research study, in five bullet points or less? The search for these "golden nuggets" requires researchers to go beyond the executive summary and identify the critical messages.

At MCHP, we have talented writers whose job it is to remain true to the science while distilling the message into what we term the "four-pager," written in lay language rather than in academic terms. These four-pagers have proven their usefulness in a variety of ways, including handouts for workshop attendees and mail-outs to media, the public and, particularly, to other researchers who want a quick read of the findings before deciding whether to delve into the full report. For high-profile reports, MCHP writes an "op-ed." This is a newspaper submission of half a page or less, written by MCHP and providing the distinct advantage of "getting the story right." However, there have been downsides. When reporters expect an op-ed, they sometimes focus their coverage on critical responses to the report, or bury the story on a back page. Often the media want to play up the unusual, outrageous or highly controversial finding, whereas the researcher wants to portray the "usual" view (the mean, median or "big picture") and would qualify the unusual or controversial finding.

Communications training and mock media sessions are helpful to researchers prior to public release, and even prior to government briefings, to ensure that the four or five key messages are refined and practised. Practising those "golden nuggets" is critical. As Ingrid Sketris succinctly states, "Practise the messages that you want to get across. Anticipate the reactions of the other 'players' and practise answering them."

Different communications strategies are required for different groups, and ways to create relationships between academics and the policy makers are critical. Another successful way in which MCHP has "translated" health services research is through creating special supplements in journals such as Medical Care (in 1995 and 1999), Canadian Journal of Public Health (in 2002 and 2005) and Healthcare Management Forum (in 2002). Supplements bring together a series of complementary papers, with the whole having more impact than a series of individually published pieces. Moreover, the forewords in these supplements are written by high-profile national and provincial figures, and help situate health services research as important to policy makers.

Health services researchers are experts in the realm of odds ratios, complex tables with 95% confidence intervals, multiple regression modelling and age- and sex-adjusted rates. Most decision-makers are not experts in any of this. But they listen to stories. Or, as writer and political activist Muriel Rukeyser once stated, "The universe is made of stories, not atoms." Story-telling has been used for millennia as a teaching tool. Health services researchers need to learn the art of evidence-based story-telling.⁴ This may mean drawing a simple graph rather than presenting data in complex tables, or using an analogy to drive home the point. At MCHP, we have worked extensively with planners from each regional health authority through a five-year, CIHR-funded research collaboration referred to as The Need to Know™ Team (directed by Patricia Martens). We challenge the planners within our team to look for the stories in the data when we produce collaborative research reports. For example, in our project on mental illness (Martens et al. 2004), one story that speaks to policy makers is the high percentage (75%) of nursing home clients who had a diagnosis of mental illness within the five years previous to admission. This information begins discussion around the staffing and services available to nursing home residents. The Need to Know™ Team members also facilitate roundtable discussions on MCHP research reports at our annual MCHP Rural and Northern Healthcare Day, to find the evidence-based stories that speak to the policy makers for that region.

Evidence-based story-telling can emphasize research results. For example, by constructing two "virtual Winnipeg schools," one with 100 adolescents in the poorest socioeconomic area and one with 100 adolescents in the richest, we brought home the scale of disparity documented in the tables and graphs (Martens, Brownell et al. 2002). In the poorest classroom you would find eight students who had been hospitalized for respiratory infection in the first year of life compared to three in the richest; 28 versus 12 would be living in a lone-parent family; 41 versus 11 would have parents lacking a high school education; and 28 versus 4 would have changed schools at least once during the year.

But ultimately, researchers must be cognizant of the tensions between effective KT and academic rigour. As Louise Potvin, scientific director of the Centre Léa-Roback sur les inégalités socials de santé de Montréal, observes:

We are seeing a perversion in health services research - increased KT perhaps, but a perversion. We are attracting more attention, more dollars to do research. But researchers try to go for the headlines. Is it better to be cited by The Economist, or by Lancet? I personally prefer Lancet. As researchers, we are to produce knowledge. The perversion in decision-making is reliance on and expectations of single studies, rather than the slow building up of evidence in a scientific way. This builds false expectations on both sides.

Lesson #4: Keep a "Back-Pocket" Mindset about Evidence Because the High-Profile Issues Never Go Away

Health services researchers need to cultivate a "back-pocket" mindset. We cannot be discouraged if our evidence is not immediately adopted by policy makers. We do, however, need to ensure its wide dissemination and its ongoing accessibility for when the issue re-emerges. For example, an MCHP report on rural hospital performance indicators (Stewart et al. 2000; Martens, Mitchell et al. 2002) found excess hospital capacity in some areas, and rural hospitals with either low occupancy rates or with clients better suited to long-term care. No real changes followed this report's release. However, three years later, policy makers were reviewing the evidence in this report, and it is increasingly likely that the evidence will be used for action. As Greg Stoddart aptly points out:

The upside of working in the researcher/public policy interface is that the researcher is committed to making a difference in the world. But all of us have had lots of experience where it doesn't, where other things are more binding - like values versus information/evidence. Policy makers are free to impose values, and values may override some evidence. Therefore, research may not necessarily carry the day. But in the best possible world, you as a health services researcher may have improved policy making, and hopefully improved the health of the population.

Researchers with "back-pocket" mindsets will be able to reintroduce research that stands the tests of time into public or government debates, long after the original evidence was gathered. And in the policy arena, timing is everything - even the timing of knowing when to bring research out of the "back pocket." As Charlyn Black observes, "Be creative and flexible with bringing evidence forward, so that it can play a role in an evolving context. There's opportunity for research findings to have multiple lives as the context changes."

Lesson #5: Sound and Fury, or Making a Difference? A Lesson about User Involvement

Figure 1 illustrates a hypothesized grid of researcher/user involvement and its relationship to policy influence. In the south-west quadrant, low user involvement and poor research design results in research that will probably be ignored. The most dangerous quadrant is likely the south-east, where the anecdote reigns supreme. There may be a high degree of user involvement but a low degree of research validity. An example of this situation is anecdotal "evidence" (usually based on only one case) brought to a board or planning group. In the north-west quadrant, researchers create highly valid research but do not engage their users in any way, either before, during or after, and the evidence may simply sit on the shelf. The north-east quadrant is hypothesized as effective in producing evidence-based change, where a high degree of both user and researcher involvement from start to finish ensures highly valid research and highly policy-relevant research. These factors yield the highest probability that the evidence will be translated into action. However, researchers need to be aware of the time and resource implications of establishing a sense of trust to enable this type of collaboration (Bowen et al. in press; Denis and Lomas 2003). Since 2001, CIHR's The Need to Know™ Team of MCHP academics and regional planners has come to consensus on research topics of particular relevance to rural and northern policy makers and planners. The team co-creates the research and ensures its dissemination and application at the regional level. However, this process takes funding - in this case, essentially $650,000 annually when including the three research "deliverables" supported under the Manitoba Health contract over the five-year period.

Conclusions

The basic lessons from the tectonic plates of researcher/policy maker interactions are simple - finding a way to have policy makers feel a sense of vested interest in results, building relationships, communicating results, producing the evidence despite the setbacks and involving the users in the research process itself. Are we getting any better at this? John McLaughlin states:

At one level, there is a recognized primacy of evidence-based decision-making in our culture. But on the other level, the search for evidence is no longer a priority. But if evidence isn't right there, decisions still have to be made right then. Resources need to be assigned to doing reviews, yet doing reviews and guidelines isn't something you can get tenure on.

According to Renee Lyons, director of the Atlantic Health Promotion Research Centre, the most profound changes in the past decade are the concept of KT and of interdisciplinary research:

The two major differences in the past five years of CIHR are in collaboration across nodes, and in knowledge translation emphasis. But the dance is not always smooth between researchers and decision-makers or granting agencies. Our research group has actually produced a tool to enable people to examine the KT potential of their research.

Other researchers feel that some progress has been made in certain research areas, but progress (if any) has been painfully slow on other fronts. Greg Stoddart comments:

Has researcher-decision-maker interaction changed over time?... I don't think there's a real overall trend. It ebbs and flows. It varies over time by ideology of the government in power. Sometimes the agenda is based upon values, and facts don't always fit the script. However, there does seem to be an increased interest in cost-effectiveness, the cost of new drugs and new technologies. I'm also pleased that there is an increase in language around the social determinants of health. But somehow the topic of health human resources seems to go no further ahead, and sometimes backwards, as does the issue surrounding the financing of the healthcare system.

Still others wonder if we perceive differences only because of our own personal, ongoing relationships over time. Louise Potvin suggests:

When you ask if we have more, less or different interactions between health services researchers and decision-makers, it seems to many people that there is an increase. But this may actually be a "cohort effect" as we all age. On the other hand, there does seem to be an opening up of researchers to engage with those not in the research community in the last decade or so. But what is "impacted" - the way people think? Behave?

So, how do we measure impact in health services research? Often our research is only one factor amidst a complex environment of political or structural change, making it difficult to attribute any change to our evidence alone. Charlyn Black notes:

If you do get decision-makers to make change based on your research, it's important to recognize that there are opportunities to then evaluate the impact of these changes. We need to build in potential to critically evaluate the impact of research evidence when it has been used to influence change, as part of an evolving research agenda between ourselves and policy makers.

It is debatable whether the relationship between researchers and policy makers has changed over time. It is even debatable if we can measure whether better relationships change the quality of decisions made. Yet, it is clear that an overriding lesson learned by health services researchers is the importance of relationship-building, whether in formalizing contractual relationships, building and maintaining personal trust, having a communications strategy or increasing the involvement of users in the research process. Easing the friction at the "tectonic plate" means ensuring research credibility within the real-world realm of policy making, and it is only through this frictional contact that we increase the probability that our evidence will be understood and will lead to policy action.

Comments

Be the first to comment on this!