Healthcare Policy

Healthcare Policy 1(2) January 2006 : 14-20.doi:10.12927/hcpol.2006.17871
The Undisciplined Economist

Kafka, New Orleans, the OARs and the KT Boundary

Robert G. Evans

Abstract

Reverse the flow of Kafka's fable of the Imperial Message and you have the problem of Knowledge Transfer - the message is not getting through.

The dying emperor in Kafka's tale has sent you a message of great importance, entrusted to his strongest herald. But the throne room, the palace, the city are so huge and so crowded with people that the message can never reach you. Reverse the tale and you have the problem of Knowledge Transfer (KT) - brutally illustrated by the fate of New Orleans. Important information, of unquestioned validity, soundly based in research and easy to understand, simply does not get through to practice. A recent evaluation of the Ottawa Ankle Rules makes the point all too clearly. Indeed, the KT problem is worse than Kafka's. Economically motivated people and organizations actively distort the messages - and try to trip the herald - while grossly oversimplified frameworks of understanding include no language in which the messages can be expressed. "More research" is not the answer.

[To view the French abstract, please scroll down.]

An Imperial Message

The Emperor - so they say - has sent a message, directly from his death bed, to you alone, his pathetic subject, a tiny shadow that has taken refuge at the farthest distance from the imperial sun. He ordered the herald to kneel down beside his bed and whispered the message in his ear. He thought it was so important that he had the herald speak it back to him. He confirmed the accuracy of the verbal message by nodding his head. And in front of the entire crowd of those witnessing his death - all the obstructing walls have been broken down, and all the great ones of his empire are standing in a circle on the broad and high-soaring flights of stairs - in front of all of them he dispatched his herald.

The messenger starts off at once, a powerful, tireless man. Sticking one arm out and then another, he makes his way through the crowd. If he runs into resistance, he points to his breast, where there is a sign of the sun. So he moves forward easily, unlike anyone else. But the crowd is huge; its dwelling places are infinite. If there were an open field, how he would fly along, and soon you would hear the marvellous pounding of his fist on your door. But instead of that, how futile are all his efforts. He is still forcing his way through the private rooms of the innermost palace.

Never will he win his way through. And if he did manage that, nothing would have been achieved. He would have to fight his way down the steps and, if he managed to do that, still nothing would have been achieved. He would have to stride through the courtyards, and after the courtyards through the second palace encircling the first, and then again, through stairs and courtyards, and then, once again, a palace, and so on for thousands of years. And if he finally burst through the outermost door - but that can never, never happen - the royal capital city, the centre of the world, is still there in front of him, piled high and full of sediment. No one pushes his way through here, certainly not someone with a message from a dead man. But you sit at your window and dream of that message when evening comes. (Translated by Ian Johnston, Malaspina College-University, Nanaimo, BC)

Reverse the flow of Kafka's fable and you have the problem of Knowledge Transfer (KT). You, the humble researcher beyond the outer fringe of power, have a message of great importance for the emperor. The message is crystal clear, and the (living) emperor needs and would very much want to hear it. But so crowded is the public discourse with other issues, other priorities, other people's messages, that it is impossible for your message to get through. Human rationality is bounded, and your message is outside the boundary. Which brings us to New Orleans.

"Just Because I Don't Care …"

From the perspective of KT, what is most interesting about the drowning of New Orleans is not the disaster itself, nor the slow and fumbling response, nor even the gathering of commercial vultures as the event reaches the end of its media life. Rather, it is that the disaster was so predictable and so widely and accurately predicted. From sophisticated computer simulations and engineering studies, to articles in high-end magazines - Scientific American, National Geographic - the message was spelled out unambiguously in letters 10 feet tall.

Nor was that message at all difficult to understand. The various interacting natural and human processes that made New Orleans a disaster waiting to happen may be complex and subtle, but the central points do not require advanced training. Much of New Orleans is below sea level, some nearly 20 feet below. The city has survived behind levees that were known to be inadequate to withstand a major hurricane. The Gulf of Mexico is a major hurricane track. When (not if) a hurricane hit, the city would be drowned. There would be great loss of life, massive disruption of lives and colossal property damage. One did, it was, and there was.

If the ultimate test of good science is successful prediction, then the analysis of New Orleans' predicament was very good science. The KT was a complete and utter failure. But that failure cannot be charged to the messenger, or to the message. The message was clear, and the heralds tried their best. But the streets of the capital, the courtyards and corridors of the palace, and especially the throne room, were thronged, jammed tight with people and their uncountable multitudes of concerns. Whatever message receptor sites there might have been were already occupied. (The arteries of imperial power, suggests Morris Barer, were clogged with fatter pork.) The herald and his portentous message could not get through.

Rowing Against the Current

Now let's narrow the focus and consider another clear and unambiguous message: orders of magnitude less dramatic than the destruction of a city, but nonetheless with powerful implications for KT in healthcare. Over a decade ago, a research team at the Ottawa Civic Hospital (Stiell et al. 1992, 1994) generated and began to disseminate the Ottawa Ankle Rules (OARs). These constituted a rigorously developed and extensively tested algorithm for diagnosing ankle injuries in the Emergency Department (ED) that permits clinicians to rule out fractures through a simple (and small) set of careful observations. Radiography, the standard response to ankle injury, was - in a high proportion of presenting cases - simply unnecessary.

Injured ankles make up a significant share of ED workload; universal implementation of these rules could thus reduce radiology load and costs, as well as saving patient and clinician time. Perhaps more important, the development of such a simple and successful clinical decision rule (CDR) held out the prospect of a much broader array of similar CDRs, a program that the Ottawa team have subsequently taken up with vigour.

And the result?

These rules are transforming the approach to the assessment of these injuries and, after training, can be used by clinicians from a range of backgrounds (including medical, nursing and paramedic staff), in both hospital and community settings. (Heyworth 2003)

But in 1999, Cameron and Naylor told a different story from Ontario:

Although participants gave highly positive appraisals of the Ottawa Ankle Rules and the educational sessions, there was no reduction in the use of ankle radiography for the 10 hospitals that received the educational sessions. … Even when a dissemination strategy is well received and involves a widely accepted clinical guideline, the impact on behaviour in clinical practice may be small.

Or, indeed, nil. So the recently published study of uptake by the Ottawa group (Brehaut et al. 2005) should not be a total surprise. Their survey of a sample of ED physicians found that while 99.2% reported familiarity with the OARs, "82.4% had not reviewed the rules in months or years, and only 30.9% were able to correctly [sic] remember the components of the rule."

Perhaps it is just as well that "only 42.2% reported basing their decisions to order radiography primarily on the rule," though that is exactly the purpose for which the OARs were designed and are very effective. 89.6% reported applying the OARs (very rarely consulting memory aids) "always" or "most of the time." But most applied these rules in combination with other clinical observations. Unfortunately, these observations were "non-rule factors that are not related to the presence of a fracture … and factors that add no more predictive value over and above the rule."

These findings essentially repeat the message of Jonathan Lomas and his colleagues (1989): "Do practice guidelines guide practice?" Well, no. Lomas et al. surveyed Ontario clinicians to determine their responses to the SOGC guidelines for caesarian section, guidelines motivated by rates of intervention that were generally agreed to be excessive. They then matched (with subjects' permission) responses with actual practice as reflected in OHIP billing. Briefly, a majority of respondents said that they knew of the guidelines, agreed with them and followed them in practice. But in fact, they did not. Inappropriate interventions continued unabated.

The point of this excursion into CDRs is not to bash ED clinicians, but to suggest a parallel between these two egregious examples, from radically different settings, of complete failure of knowledge transfer. In both cases the message was simple, clear and about as solidly grounded in evidence and analysis as one could hope. Both messages were consequential and were widely disseminated within the relevant communities, and their implications for "What is to be done?" were direct and unambiguous. Nothing happened.

"Do I Have the Party to Whom I Am Speaking?"

We now have a nice, new Canadian journal of health policy. KT is both part of its purpose and part of its subject matter. But Kafka's story implies that simply (!) generating sound research findings with clear policy implications and disseminating them widely may nonetheless achieve nothing. (Bit of a nihilist, Kafka was, but that's one way to avoid disappointment.)

Health researchers' messages, of course, address not one emperor but several different "policy maker" communities. The experience with the OARs, however, underlines heavily a point made long ago by Lomas, that every clinician is a policy maker. Clinical policy, the sum and resultant of day-to-day decisions, is at least as significant for health system performance as is the "high policy" of politicians, senior bureaucrats and administrators - and even, occasionally, judges. (A.P. Herbert's character Albert Haddock argues that a judgment of the House of Lords is equivalent to an act of God because it, too, is something that "no reasonable man could have expected.")

Official policies can have a powerful impact on the context of clinical policy, but their effects, for good or ill, ultimately flow through clinical decisions. Have we implicitly written off direct communication with clinicians as "not our department" - or perhaps wholly ineffective in the absence of substantial contextual change? Do we then bet our chips solely on reaching non-clinicians?

At the journal's launch, though, Paul Jacobson argued rather vigorously that our journal will not reach even those official policy makers who are really critical for effective KT. A broadened conversation with health administrators and bureaucrats remains within a shared framework of understanding. Our messages may be clear, sound and well understood within that community, but the real levers of power are elsewhere - in the hands of politicians and senior finance officials who are outside our conversation.

Their absence was powerfully illustrated by Lavis et al. (2003) in surveying the penetration within federal and provincial bureaucracies of current concepts of population health. Those ideas were widely disseminated and had been taken up across health, social services and labour ministries - but not in finance. What could fiscal policy possibly have to do with population health? Some thought that the survey had been sent to them in error.

Lavis's findings reinforce Kafka's point. The failure of KT was not traceable to a confused message or to inarticulate messengers. People in other ministries "got the message" with no apparent difficulty, and members of finance departments are surely, on average, no less intelligent. Why has nothing gotten through?

Kafka's hordes have, I think, their analogy in the powerful and elaborately articulated framework of understanding peculiar to economics and predominant in finance departments and the business community generally. Such frameworks provide a filter for information, defining what is and particularly what is not to be attended to. Conventional economic analysis, especially as practised in North America, provides "off-the-shelf" explanations generated from a priori theory for both the determinants of health and the dynamics of healthcare systems. These typically incorporate little, if any, knowledge of the actual subject matter and are correspondingly grossly oversimplified when not just plain wrong.

Findings from health services research, solidly rooted in the real world of health and healthcare, do not fit into the predetermined conceptual categories of the conventional economic framework - no receptor sites - so in a real sense cannot be heard. They can be heard by those working within alternative, much looser and more flexible frameworks of understanding - but they are generally farther from the throne.

Research Be Damned! We're Trying to Make a Buck!

Worse, the crowd in the throne room includes some - small in numbers, perhaps, but very heavily resourced - with a strong economic interest in blocking or distorting the messages from research and substituting self-serving myths. The pharmaceutical industry is the most notorious example, but private insurers have an obvious interest in undermining universal public coverage. No private payment, no private insurance. Commercial diagnostic enterprises can be indifferent to OAR-type decision rules, only so long as they have no effect. Imagine the impact on costs, and on health policy generally, if all access to MRI had to be justified by some explicit, evidence-based prospect of improved patient outcomes.

All cost savings are threats to someone's income. For publicly traded corporations, a reduction in expected future earnings translates directly into reduced share values. Remember Nortel - capital markets are brutally unforgiving.

Demand creates its own supply, and these commercial interests have supported the growth of a specialized private disinformation industry - "liars for hire" would be impolite, call them marketers by other means - to promote public policies furthering those corporate interests and to deflect or discredit threatening research findings. All perfectly normal, in a for-profit world. These activities have little or no penetration among the health research community, but have been very effective in exploiting the intellectual vulnerability of those pre-conditioned to hear their selective, simplistic and grossly distorted messages. The Chaoulli decision provides a spectacular example, but any randomly selected product of standard economics training should serve as well.

The "(K)retaceous-Tertiary (KT) Boundary" refers not to knowledge transfer, but to a thin layer of iridium-enriched clay marking a discontinuity between these geological periods. It is generally interpreted as the consequence of a really bad day in the Yucatan. The crossing of this KT boundary was a decisive break in evolutionary history. But the Age of Mammals, previously a bunch of evolutionary no-hopers, would have been impossible without the elimination of the dinosaurs. While they remain in place - unexamined habits of thought and behaviour, fed and reinforced by entrenched economic interests - KT will be a dubious battle.


 

Kafka, la Nouvelle-Orléans, les OAR et la frontière de l'AC

Résumé

L'empereur mourant du récit de Kafka vous a envoyé un message de la plus haute importance et l'a confié à son plus fidèle messager. Mais la salle du trône, le palais et la ville sont si vastes et si populeux que le message ne vous parvient jamais. Racontez cette histoire en sens inverse et vous avez le problème de l'application des connaissances (AC) - crûment illustrée par le destin de la Nouvelle-Orléans. Les renseignements importants dont la validité est sans équivoque, qui sont fondés sur des travaux de recherche solides et qui sont faciles à comprendre, n'arrivent tout simplement pas à se frayer un chemin jusqu'à la pratique. Une récente évaluation des « Ottawa Ankle Rules » n'illustre que trop clairement ce point. Le problème de l'AC est en réalité bien pire que celui de Kafka. Les personnes et les organismes ayant des enjeux économiques s'emploient activement à déformer les messages - et à essayer de faire trébucher le messager - tandis que des cadres théoriques trop simplifiés n'incluent aucun langage dans lequel exprimer les messages. « D'autres travaux de recherche » ne sont pas la solution.

About the Author

Robert G. Evans
Professor of Economics
University of British Columbia
Vancouver, BC

References

Brehaut, J.C., I.G. Stiell, L. Visentin and I.D. Graham. 2005 (October). "Clinical Decision Rules 'in the Real World': How a Widely Disseminated Rule Is Used in Everyday Practice." Academy of Emergency Medicine 12(10): 948-956.

Cameron, C. and C.D. Naylor. 1999. "No Impact from Active Dissemination of the Ottawa Ankle Rules: Further Evidence of the Need for Local Implementation of Practice Guidelines." Canadian Medical Association Journal 160(8): 1165-1168.

Heyworth, J. 2003 (February 22). "Ottawa Ankle Rules for the Injured Ankle: Useful Clinical Rules Save on Radiographs and Need to Be Used Widely." Editorial. British Medical Journal 326: 405-406.

Lavis, J.N., S.E. Ross, G.L. Stoddart et al. 2003 (April). "Do Canadian Civil Servants Care about the Health of Populations?" American Journal of Public Health 93(4): 371-379.

Lomas, J., G.M. Anderson, K. Domnick-Pierre et al. 1989. "Do Practice Guidelines Guide Practice?" New England Journal of Medicine 321(19): 1306-1311.

Stiell, I.G, G.H. Greenberg, R.D. McKnight et al. 1992. "A Study to Develop Clinical Decision Rules for the Use of Radiography in Acute Ankle Injuries." Annals of Emergency Medicine 21: 384-390.

Stiell, I.G, R.D. McKnight, G.H. Greenberg et al. 1994. "Implementation of the Ottawa Ankle Rules." Journal of the American Medical Association 271: 827-832.

Comments

Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed