The role of evidence in improving the quality of care and the sustainability of our healthcare system is part of a number of healthcare reform efforts including, among others, the Triple Aim Framework developed by the Institute for Healthcare Improvement and the Excellent Care for All Strategy in Ontario. In this interview, Charles Wright (CW) speaks with Les Levin (LL) about the ways that evidence can be developed and brought to bear on healthcare decision-making. Dr. Levin has led work for a number of years – first within the Ontario Ministry of Health and Long-Term Care and lately at Health Quality Ontario – that has led to substantial cost avoidance and quality improvement. These experiences re-enforce the importance of a close connection with decision-makers but also underscore the receptivity of decision-makers to evidence that is appropriately placed within a local context.

Charles Wright, in conversation with Les Levin

CW: Let me ask you the overview question – what would you say is the purpose of your organization? Tell me what it is and why it exists.

LL: The Medical Advisory Secretariat (MAS) is a component of Health Quality Ontario (HQO), and its mandate is to assemble evidence-based analysis, both within the unit and in collaboration with academia, especially at the University of Toronto, University Health Network and McMaster University. The purpose is to develop objective, scientifically rigorous evidence-based analysis on new and existing health technologies.

CW: It also forms a close relationship with what's called the Ontario Health Technology Advisory Committee. Could you describe that relationship?

LL: It's been described as a dyad. Through the collaboration with MAS, evidence that is contextualized by expert panels assembled by the province is reviewed by the Ontario Health Technology Advisory Committee (OHTAC). The latter actually produces recommendations based on evidence, or guidance based on evidence. So, the evidence goes through two contextualization processes. The first is through expert panels set up by the Province. Then OHTAC is responsible for prioritizing the requests for health technology assessments, which come from the Ministry of Health and Long-Term Care and from other stakeholders in the health system. These stakeholders are predominately, at this point anyway, from hospitals, but also from community-based health services. It's up to the MAS to assemble the evidence over a 16-week period for single technologies, which are then presented back to OHTAC for its recommendations.

CW: How does MAS go about fulfilling its mandate to produce evidence-based reviews and analysis and recommendations, and then go through OHTAC?

LL: I think there are two components. One is a systematic review, a search of the literature relating to the technology. It's a scientifically rigorous process, done according to a template that's set up for these reports. It aligns evidence through a hierarchy of quality. We use GRADE, which has been universally adopted as a validated process for assigning quality to evidence; it's not just looking at the broad comprehensive analysis of evidence and comparative, where the technology is compared to other technologies. It's also assigning quality, which is a very important part of the decision-making process. If you have moderate- to high-quality evidence, there's little chance that further research is going to change your confidence in the estimate. That's the systemic review component. That's always contextualized, as I said earlier, with expert opinion. Experts are asked to provide their opinion iteratively, when the analysis is under way.

The second component, which is also coordinated by but not undertaken by MAS, is the economic analysis. That's done by two health economic units in the province – one through THETA, another through PATH. The former, THETA is at the University of Toronto and the latter is McMaster University.

Using the full-blown resources of both those health economy academic units and working closely with MAS as the systematic review is under way, we'll develop a full economic analysis, usually budget impacts and cost-effective analysis.

CW: Would you say that MAS holds the reviews and that OHTAC adds a wider perspective on the implications in all of the systems and society?

LL: Absolutely. OHTAC, as a very large stakeholder and an expert group, is able to provide that kind of unique perspective, so evidence itself is only part of that. I know there's a question coming later on the decision-making process, but OHTAC applies much more than an evidentiary lens and a health economy lens on these issues. It also provides the relevance of the evidence to the whole health system.

CW: How do you decide how to prioritize what you have time to engage in?

LL: The prioritization is actually undertaken by OHTAC. It's surprising how rarely we've had to use the template that allows OHTAC to really prioritize in an objective way. It could be effects, it could be a societal perspective, but the actual... or it could be a diffusion pressure; but the prioritization is undertaken by OHTAC. MAS enters the prioritization process by declaring what resources it has to undertake the analysis. My recollection is that at least 90% of analyses that have been requested of OHTAC, where OHTAC has regarded the analyses as being relevant, have... We've had the resources to deal with those.

CW: Apart from the scientific evidence, what are the other issues that affect the final recommendations?

LL: This goes to what OHTAC developed through a sub-committee process – the decision determinants – where there was a literature review and a discussion with key decision makers in the health system, and where the decision determinants were finally agreed to by OHTAC. There are four major determinants. One is the quality of evidence, of effectiveness and safety; the second is value for money, which is the economic analysis I referred to earlier; the third is societal and ethical considerations, which is more qualitative research; and the fourth is feasibility at a macro level. There's no detailed feasibility study, but the perspective of the health system from the stakeholders that make up OHTAC gives a rough estimate of the feasibility of adopting the technology.

CW: All of this is designed to put the scientific recommendation into a real world and local context so that it can be implemented? All of that is designed to get the science plus the real world and the local context more ready for the possibility of a policy decision or what to do with the recommendations?

LL: Yes, and the issue of a policy decision is terribly important, because universally, the traction between evidence and policy is not very strong.

CW: Exactly.

LL: But in the case of OHTAC, 85% of their recommendations have had traction on policy.

CW: What's the process now? Once the Advisory Secretariat has made the draft recommendations, I understand they're sent out to the public; they all come back and eventually OHTAC makes a recommendation. What happens to the conclusions and the recommendations?

LL: There's a defined process within Health Quality Ontario. The chair of OHTAC, at his discretion, may decide to take the recommendation and send it back to where it came from with the full recommendations. If the request came from the ministry around a single technology because they were examining a fee code, for example, or from the Ontario Medical Association because they were examining a fee code, the chair of OHTAC may send it back to – let's say the OMA – with evidence and a recommendation in response to the request. If the evidence-based analysis is much more broadly based or complex, or it's a sensitive issue regarding the technology being evaluated, or there's a potential need for an investment by the ministry related to the adoption of the technology, then it will usually go to the Board of Health Quality Ontario. HQO would do one of two things. It may decide to implement some of the recommendations itself, For example, if it needed to develop quality performance indicators and monitor, track and report on them, Health Quality Ontario could do this. If HQO decided to move at least part of the recommendation into quality, such as evidence-based payment or quality-based payment, it could do that as well. What is more common is that within the Effective Care for All legislation, the chair of the Board of HQO has direct access to and can provide advice to the ministry regarding the adoption of certain technologies or mega-analyses, which would be more aligned to disease conditions or health states. Those go through the ministry for further analysis and implementation.

CW: That brings to mind the inevitable question. To what extent are you involved in the actual implementation?

LL: Well, until now we have not been involved, but increasingly we're being asked to at least develop a macro implementation plan. What that means – it's still being worked out actually – is, at least we have defined the key players who would be involved in the implementation. For example, if there is a recommendation that a certain technology be adopted and it has an implication regarding the fee code or the fee schedule, then we would identify that as one of the implementation components. The recommendation would go back to the Decision Determinants Committee or it would be identified or flagged as something that the Decision Services Committee may wish to address. If it needs an investment in hospital infrastructure, the ministry would need to look at it and address it with hospitals. If it's a safety issue, the ministry may want to set up a special safety committee to look at implementing the recommendation. That's all being assembled now as part of the macro implementation roadmap for the ministry to consider, but the actual detailed implementation is something that one would expect the hospitals, in collaboration with the ministry or the community-based healthcare system, to implement for the LHINs.

CW: How specific does it get with the recommendations? On the basis of the evidence, such and such should be done, to paraphrase, do orthopedic surgeons have to look at it, or do hospital CEOs have to get involved? Does it ever get to that level of detail?

LL: It doesn't. It's not really prescriptive in that sense, but the recommendation may, for example, state that if the technology is really complex it may be limited to facilities where there sufficient volumes to maintain excellence for the delivery of the service that's applied to the technology. So far, it's never been more prescriptive than that.

CW: All this has to do with health technology assessment and the implications in the healthcare system, but how does it relate to the whole issue of quality and improving quality in healthcare.

LL: One could argue that the delivery of any service – or the access to any service or technology or clinical intervention – that has not gone through the scrutiny of evidence of effectiveness could be potentially dangerous, or it certainly would be construed as a waste of money. In terms of safety and effectiveness, if those two components of quality are important to driving the quality agenda, then I think it's terribly important. In fact, the way HQO has been set up, there is an expectation that the evidentiary platforms will be the basis for moving some of the quality agenda forward. But there's a component of the quality agenda that doesn't require the full-blown evidentiary platform. I think we are just trying to find a balance as we speak and doing the formative part of HQO's existence.

CW: What values or obstacles do you see in the way of implementing good evidence-based recommendations into policy and funding decisions?

LL: What values to uptake? We're living in a jurisdiction in which the traction of the evidence and the translation to policy is probably a world-beater at the moment, and it's looking increasingly that way. We are living in a province where the translation of evidence to policy is probably the best in the world; even something that's cost-effective doesn't mean it's not going to cost a lot of money. I think that's the first issue of financial imperative.

The second point is a potential barrier. We need to recognize that policy decision makers, especially when encumbered by fiscal constraint, will cherry-pick all the negative recommendations for implementation. That has not happened in Ontario, but there is a real risk it could happen. It would be a barrier to the uptake of effective technologies that are costly, even if they improve patient outcomes.

The other barrier is our funding system. It is globally allocated to hospitals and community-based healthcare. That's a significant problem, because where you have to pay for the technology at the front end, even though it's going to result in downstream events avoided and costs avoided, there's a reluctance to invest. We're finding that increasingly with non-drug technologies. There's a silent change taking place in the health system for technologies that can be very effective – non-drug technologies – at the front end; but because we've become – I call it mortgage junkies – we invest in chronic diseases amortized over 20-year periods with a compounding effect of drugs. It's cheaper to do that for the same reason that it's more reasonable to buy a house that way. You don't feel the pain.

CW: Why should governments enthusiastically support health technology assessment?

LL: Because it's the only transparent, credible, consistent and fair way to make decisions, everything else aside, and it's defensible. As we get into tougher decision-making modes or fiscal constraint, without evidence it's going to become increasingly difficult to make those decisions in a fair, credible, transparent and consistent way.

CW: Lastly, are there any changes you would like to see in the field of health technology assessment and the way it's done in general throughout Canada, just to bring it close to home?

LL: I think that two major innovative developments have taken place in the MAS/OHTAC dyad. The first has been addressing uncertainty in evidence. We could do one of two things. We could either walk away, so we don't have the quality evidence we need for really important technologies and just leave it to passively diffuse in the system. Or, we can say that at this point, we don't have the evidence we need so we're going to evaluate this in real time in Ontario. We've already done this through field evaluation studies that have been very important and hugely successfully by any international standards. If we can do this in collaboration with other provinces, it would be terrific, but it does take an investment.

The second is in mega-analysis. Instead of looking at single technologies, there's one lesson we've learned: if you are going to make a decision on a single technology, it needs to be made in the context of all the other technologies that could be used instead of the newer technology. We're not looking for more costly technologies, but the only way you can really make a determination is through comparative effectiveness analysis around disease states or disease conditions and health states. We look for major drivers in some of these disease conditions; we just aggregate the drivers, look at the different technologies around these drivers and re-aggregate based on quality of evidence and health economic analysis. This is one of the key developments in Ontario that has been of considerable interest, certainly to macro decision makers and policy makers. I think that's going to be our future. I think that's exactly where the future of health technology assessment lies. So two things – the field evaluation and the mega-analysis.

There's a third component that I'll touch on very briefly, and that is taking the whole evidence-generating machinery – including decision making – into the pre-market arena and working with industry, regarding industry as a research and development part of our health system and applying that in the pre-market state. That's up and running in Ontario now. OHTAC is very involved in that process – to provide the health system lens to the relevance, the disruptive effect, the patient outcomes and all kinds of ethical oversight, if you like, of that process in the pre-market space. I think that's evolving, and I think it's already taking off in a substantial way in Ontario.

Comments

Unknown Unknown wrote:

Posted 2013/07/01 at 04:56 PM EDT