Health services research (HSR) is commonly conceived as an applied discipline whose success is defined by its tangible impact on policy, practice or both. In Canada there has been a concerted effort to engage decision-makers in informing the research agenda. While it is admirable to aspire to practical utility, the HSR community has no control over the ultimate disposition of its work. Furthermore, the conditions for change must be present if the pathway from relevant, high-quality research to application is to be relatively smooth and immediate. In such cases, the changes may have occurred regardless of whether the research to support them took place. An examination of some widely renowned HSR reveals that timely and significant impact is relatively rare. Moreover, research that fundamentally changes how we view the world plays out over decades; it would be impossible to act on it in the short term, and in some cases it is not clear what ought to be done. The implications are that the first duty of HSR is to seek truth, and that funding and decision-making communities should define "useful" broadly, from a longer-term perspective. Taking the wide and the long view will in the end generate a greater return on investment in HSR than focusing too narrowly on contemporary preoccupations.

Health services research (HSR),¹ like all research, is supposed to improve the human condition. Cultures that support and produce research tend to be more prosperous and free than those that don't. That research in general makes a difference is rarely disputed; in fact, the question is rarely asked. But it is asked of HSR for two main reasons. First, the discipline is only a few decades old, but there is already good evidence that the insights of comparative effectiveness studies and thousands of clinical practice guidelines are widely ignored. Second, HSR has come to advertise itself as a practical discipline that aims for tangible and fairly immediate impact (the very name of the Manitoba Centre for Health Policy implies that its primary activity – research – is instrumental). The public and governments tend to support basic science research as an intrinsic good that will pay off somewhere, some day, in unanticipated ways. They rarely cut HSR such slack: here, it's show me the impact.

Impact is not so easy to define. If only studies that result in a tangible change in the way health services are delivered, or the outcomes they achieve, can claim to have made a difference, HSR has almost always fallen short. But the definition is far too narrow. HSR can influence thinking, culture, policy, behaviour and practice. It takes place in an economic, social and political context, not in a linear, rationalist world where systematically acquired evidence explains all choices and results. Power, tradition, interests and uncertainty all affect what is done and what is achieved. Furthermore, health systems are riddled with perverse incentives and are principally organized for the convenience and profit of the supply side rather than the public or patients. Long-standing problems remain unresolved despite the efforts of managers, leaders and politicians, some of whom are genuinely interested in a more evidence-informed world.

The question is thus less whether HSR has made a difference, but what HSR should aspire to achieve. The argument here is that HSR should seek truth. It should shed light on phenomena inaccessible to common sense or intuition – the counterfactuals that confound simplistic analyses and glib prescriptions, and force a re-examination of the status quo. It should attempt to answer questions that are both broad and deep, mirroring the experience of people and organizations in complex adaptive systems. To be actionable, it should ask how and why things are as they are in addition to describing what they are. Producers of HSR are rarely decision-makers in the health system, but they can and should be attuned to how decisions are made and the vagaries of the political arts. The case here is not to exempt the choice of HSR topics or the design of studies from all considerations of relevance or potential use. It is to define "useful" and "important" in a more nuanced way.

A short paper cannot fully articulate the argument, but it can be illustrated by an examination of the impact of what are widely regarded as blockbuster HSR studies. Table 1 identifies examples of groundbreaking studies, their findings and, in very general terms, their impact.

Are there any patterns in this small sample of renowned research findings? It would appear that those with the greatest apparent impact – the Hormone Replacement Therapy and hospital utilization studies – deal with discrete phenomena where behavioural change is relatively straightforward; where (in the case of HRT) users of services can act unilaterally; where findings evoke embarrassment, fear, hope, shame or other strong emotional response; and where the environment is more manageable (hospitals). Those with the least apparent concrete and tangible impact deal with complex phenomena; require concerted and multi-pronged initiatives to address; threaten traditional cultures, elites, egos and interests; and do not suggest an obvious blueprint for action.

Nothing, of course, is that simple, and experiences and research suggest further possible explanations for the presence or absence of a tangible and attributable response to research. People generally discount future, anonymous health benefits quite steeply in comparison to potential here-and-now health benefits that would accrue to identifiable people. (It is easier to raise money to send Alice Wilson to Loma Linda Hospital for a double-organ transplant than for programs that would help those under-achieving kids in poor Winnipeg neighbourhoods.) Hence population health studies are, other things being equal, less likely to engender a rapid, tangible response than some clinical studies (complexity is, of course, another major barrier to change).

But let's frame the question differently. Aside from natural disasters, intellectual insights and ideas have been responsible for all the great changes in human systems. The biggest ideas had no immediate and quantifiable impact. The Copernican and Newtonian revolutions did not change manufacturing, inspire armies or rearrange the means of production in any observable way. Yet, they changed the world and spawned an explosion of ingenuity in untold ways to enormous and irreversible effect. The new knowledge changed everything – but not in a mechanical, causal, transparently traceable fashion. Analogously, it is no longer possible to think that healthcare alone (or even mainly) determines health status; that need alone drives spending; or that regulation and licensure are sufficient to guarantee quality.

The health and healthcare conversation is vastly different today than it was 30 years ago, precisely because of the contributions of HSR writ large. Atul Gawande's (2009) compelling account of the Dartmouth studies in The New Yorker became required reading in the White House. As Clyde Hertzman has noted, some people are able to ignore convincing evidence in reaching their own conclusions about school performance but most, eventually, do not. In a world where there is a touted new cure, drug and technology every week, and where the media have an insatiable appetite for healthcare gadgetry and magic, there is nonetheless broad awareness that healthcare and health are conceptually distinct, and that the former explains rather little of the variance in the latter. HSR findings created this awareness, and the implications are potentially enormous.

All this suggests a paradox: the more concrete and obvious the relevance and impact of HSR, the less likely it is to be paradigm-altering. Typically, change happens because the antecedent conditions are already in place; research findings may be little more than confirmatory, or may provide a small additional tailwind to journeys already underway. Conversely, truly game-changing research is unlikely to change the game any time soon because it bumps up against the powerful forces of tradition, hierarchy, inertia, vested interests and complexity. Thus, the HSR that may make the biggest difference in the long run is highly unlikely to make any concrete difference in the short run because genuinely original and creative research is so difficult to absorb, let alone apply.

As a result, HSR would do well to resist selling itself as a purely responsive, relevant and useful service industry, its ear attuned to "Listening for Direction" and driven by the preoccupations of decision-makers faced with daily and vexing dilemmas. It has been clear for millennia that you cannot derive the ought from the is, and HSR is about the is. Often, decision-makers and other actors find the low-hanging fruit inedible, for whatever reason: there is a lesson in the widespread failure of healthcare workers to wash their hands in hospitals 160 years after Semmelweis's elegant and definitive research. There are no algorithms for estimating "knowledge demand," and not all wisdom lies in crowds (even crowds of talented decision-makers). You never know when that long-neglected research will emerge from the dusty shelf when the decision-making context changes.

Finally, this perspective suggests some reflection on another contemporary truism: you can't manage what you can't measure. That may be true in healthcare, but it is less true in estimating the worth of research. In my own efforts to assess the impact of HSR, it became clear that there are no reliable measures of the real or potential impact of the most interesting and relevant studies. It is easier (though not always easy) to recognize an important and original idea than to divine its likely impact in the foreseeable future. Researchers cannot control anything beyond the conception, design and execution of their research, and we do not live in a particularly evidence-seeking health culture. Before judging HSR on its utility, it would be helpful to learn how decision-makers perceive their role in leaving so much ostensibly useful research on the shelf. And before the wider community decides that putting itself in charge of the HSR agenda will yield a greater return on investment, it might profit from its own accounting of how it has used the research it has commissioned in the past. This would be a valuable complement to the work the Canadian research community has already done to advance the science of measuring return on investment (Panel on Return on Investment in Health Research 2009).

Lest this argument be misconstrued, I should add that I have spent my own HSR career in an applied health research agency and as a consultant whose livelihood derives from doing others' bidding. It is my instinct to want to do things that will make a difference, but that presumes that those able to make a difference have the intention and the will to do so. There are infinitely more false negatives (not "using" research that should inform policy and practice) than false positives or true positives. That is the way of the world, and in such a world, it is ultimately more useful to attempt to enlighten than to steer. That is what the best HSR has always done, and when insights and ideas take hold, the world changes, and the return on the initial investment is immense, albeit unquantifiable – for now at least, and possibly for a very long time.

Comment la recherche sur les services de santé a-t-elle donné lieu à des changements?

Résumé

On conçoit habituellement la recherche sur les services de santé (RSS) comme une discipline de recherche appliquée dont le succès se mesure par son impact concret sur les politiques et la pratique. Au Canada, il y a eu des efforts concertés visant à engager les décideurs à éclairer le programme de recherche. Bien qu'il soit admirable d'aspirer à une telle utilité pratique, le milieu de la RSS n'a aucun contrôle sur l'utilisation finale de son travail. De plus, les conditions nécessaires au changement doivent être présentes si on souhaite que le parcours entre une recherche pertinente de haute qualité et son application soit relativement souple et immédiat. Dans ce cas, les changements peuvent avoir eu lieu indépendamment de la recherche visant à les appuyer. L'examen de certains projets de RSS de grande renommée révèle qu'il est plutôt rare d'observer un impact significatif et opportun. De plus, l'effet des recherches qui transforment fondamentalement les choses s'étale sur plusieurs décennies; il est donc impossible d'agir à court terme et, dans certains cas, on ne sait pas clairement ce qu'il faut faire. En conséquent, le principal devoir de la RSS est de rechercher la vérité alors que les milieux de financement et de prise de décisions doivent adopter une définition plus large de la notion d'« utilité », et ce, dans une perspective à plus long terme. Cette approche élargie permettra de générer un rendement sur l'investissement dans la RSS plus important qu'une vision étroite centrée sur les préoccupations quotidiennes.

Comments

Paul Tanse wrote:

Posted 2011/03/02 at 02:22 PM EST