A key principle underlying the Excellent Care for All Act was the importance of evidence in guiding decisions across the healthcare system. The Canadian Agency for Drugs and Technologies in Health (CADTH) has led pan-Canadian efforts for several years to bring evidence to decisions about what will be covered and what will not be covered in Canadian healthcare. In this interview, the CEO of CADTH – Brian O'Rourke (BO) – speaks with Charles Wright (CW) about a number of the challenges and opportunities inherent in bringing evidence to healthcare decision-making. A key point throughout the interview is the range of efforts necessary to support decision-makers as they try to bring evidence into coverage and other potentially controversial decisions.

Charles Wright, in conversation with Brian O'Rourke

CW: Let me start by asking how you would describe the general purpose of your organization – The Canadian Agency for Drugs and Technology and Health?

BO: More affectionately known in this country as CADTH – that's the acronym to get out of your lips! We're a health technology assessment agency. In its broadest sense, that means we inform; we provide information to policy makers in Canada regarding health technology. It can include pharmaceuticals, medical devices, medical/surgical procedures and diagnostic tests.

CW: Could you describe some of the methods, the processes, by which you fulfil that mandate?

BO: The difficult challenge with health technology assessment is trying to narrow down what you're going to look at. There are way too many technologies –pharmaceuticals, procedures, and so forth – to look at them all. So there's a very structured process for getting to what we're going assess.

Our primary customers – and we call the ministries of health our customers – are usually faced with some uncertainty regarding health technologies, and we help close that evidence gap to where they feel comfortable making a decision. To do that, we've put a structured process in place. Most countries around the world that have a health technology assessment agency, or an organization that provides that type of function, follow a similar process.

First, you need to plan and prioritize the types of assessments you'll do. In this country, there are about 25,000 drugs, and I've heard the device industry say there are over a million different medical devices. It's impossible for us to do an assessment and provide evidence on all of them, so planning and prioritization are first and foremost.

Once we come up with an agenda or a list of what we're going to assess, we get into production mode, where we get our research staff and clinicians together to look at the evidence and to produce reports or recommendations or tools. Then you need a formal process to disseminate that knowledge, to transfer it to the policy makers.

This all requires scientific oversight. At CADTH, we have a chief scientist. Her job is to be our eyes and ears with respect to the methodologies we use, the quality of our products, some of the staff training and education – both within CADTH and externally – and the evaluation. So plan, prioritize, produce and disseminate, with some good scientific oversight.

CW: That's a good overview. Let's go to some of the issues in more detail. What sort of expertise do you require in this process?

BO: First would be staff within the organization. It's important to note that we are a not-for-profit corporation; none of us in this organization are public servants. We're independent, arms-length from the ministry and from industry, so we provide good, independent, evidence-based information. Our internal staff are a good, broad mix of clinicians, physicians, pharmacists and nurses – a lot of PhD- and master's-trained scientists in epidemiology, pharmacology, public health, biochemistry, and so forth. We also have research assistants – typically with a bachelor's degree, who provide support to our scientists. We have a number of health economists on staff and a really strong component of information specialists or librarians; I think we've got one of the strongest groups of librarians in the country to dig up the information we're looking for in both the known literature and the grey literature. We've got project managers as well, who help us keep our projects on track; timeliness is an important component of what we do. Then, a number of people come to us with an expertise in knowledge mobilization, people who understand how to develop tools or how to get that research into a format that's understandable from a policy maker's perspective or a clinical perspective. That's tour internal mix of staff.

We also rely heavily on expert advisors, so we have a number of advisory committees. We have a Drug Policy Advisory Committee composed of senior drug plan officials from all the participating jurisdictions. There's a Policy Forum; this is not a CADTH group but a group of senior officials from across the country on the medical devices and medical procedures side, and they provide advice to us. We have two distinctive expert committees. Our Canadian Drug Expert Committee is a mix of specialist physicians, family practice physicians, pharmacists, health economists, and so forth, who provide expert recommendations based on the work we do. They deal mainly with our drug portfolio. Our Health Technology Expert Revue Panel is a new committee we put together last year; it looks at all the work we do on non-drug technologies – devices and procedures.

We're also linked quite well with the academic community. There's a network called the Health Technology Analysis Exchange. We provide secretariat support to that group. Again, it's another means to getting some good expert advice from the academic community and other producers of health technologies.

CW: It's a very broad scope of relevant professional expertise.

BO: Absolutely, and for every report that we're doing, we typically also contract an expert who's a specialist in that particular area.

One other thing I should mention as well is patient input. We incorporated patient input into our processes for our Common Drug Review in about May 2010. That's an important aspect of how we do our work now.

CW: Going back to your comments about prioritization, how do you decide which issues to engage in? What constitutes your prioritization template, if you like?

BO: We get products into our portfolio mix in two ways. One is a reactive kind of approach, and one is more proactive. On the reactive side, we have two programs that produce reports and recommendations. One is the CADTH's Common Drug Review. As a new drug, a new molecule first receives market authorization from Health Canada – the regulator. Then, the manufacturer has to make a submission to us. We do our work and make recommendations on whether the drug plans should list that product. There are between 30 and 40 new molecules marketed in this country every year that we look at and make recommendations for. We also have a rapid response service, again reactive, where our customers – the ministry and people working in the health authorities in hospitals across the country – can request information from us. They have a specific question on a technology, and we provide them with whatever level of information they're looking for, again to close that evidence gap. Sometimes they need information tomorrow, so we're able to do some very quick analysis for them. Sometimes they can wait a week or a month or even three months.

Then there's the proactive side, where we scan the horizon. What are the drugs, the technologies, the procedures in the pipeline that policy makers will be faced with making decisions on in a year's time or five years' time? We listen a lot to our customers, to deputy ministers, to senior officials, to health authorities. We've positioned a liaison officer in each of our participating jurisdictions. They are our eyes and ears on the ground, working closely with ministry officials and health authorities.

Over the last couple of years, we've tried to focus on some priority themes, again through scanning the horizon, listening to customers, looking at their business plans, their throne speeches, and so on. We've narrowed the type of work down into a number of clinical areas. For this fiscal year, for example, we've got five priority themes: cardiovascular and cerebral vascular, infectious diseases, mental health, endocrine disorders with a focus on diabetes, and neurological disorders. We look at our priority themes every year. Our board approves them on an annual basis as part of our business planning.

We've created theme leads within our staff, individuals who have a focus on all the work we're doing in that particular theme. They're linked with the clinical societies within that theme.

Once we have those proactive items, we have a Portfolio Committee, a central intake process where a cross-representation of our staff get together weekly to prioritize all the projects we could be working on. We have a very structured process for that as well, looking at how relevant the work is, when we could do it, what impact it would have, and whether there any risks in doing or not doing it. We identify a list of potential topics and then sit down, scope them out, prioritize them, and work with our advisory committees to develop the actual portfolio of projects.

CW: In your process of review and analysis, what kind of output do you provide? Is it conclusions? Or is it recommendations, and if so, is it who should do what? What's the format?

BO: We have gone through a major transformation at CADTH over the last few years. It grew up primarily as a research-based organization under its old name and mandate, COOHTA, the Canadian Coordinating Office for Health Technology Assessment. We sent out research-based reports with conclusions – usually peer-reviewed reports – but there weren't any hard expert recommendations to policy makers. That left them wanting more. When we introduced our Common Drug Review, we also introduced the expert committee process, and now we make recommendations to the drug plans on whether or not they should list these new drugs. We've introduced that as well for our medical devices and our procedures and diagnostics, with this Health Technology Expert Review Panel. More and more, policy makers are looking for hard recommendations for their policy-making needs. More and more, our format is expert recommendations, tools and guidance documents to go along with them, but all with an expert approach.

CW: What happens to your conclusions and recommendations? You write up the project; you make it available. Is that targeted in any way? Is there any process, or does it just go out there in the hope that it will be picked up as much as possible?

BO: In the past it was one of those processes where we would produce this very large scientific-based report – very good reports, don't get me wrong – but the process was much more passive. We would put the report on our website or maybe disseminate it and hope that it ended up on a policy maker's or a clinician's desk and that they might do something with it.

We're taking a much more proactive approach now. When our Common Drug Review recommendations are released, within about five days the Expert Committee makes the recommendation, and it goes directly to the drug plans. The drug plans, of course, are still the decision makers. We are not; we're still a recommending body. But about 92% of the time, when those drug plan managers make a decision, it's consistent with the recommendation we provided.

Similarly, in all our other work, we now develop a knowledge exchange, or a knowledge mobilization strategy or plan, for every project. At the start of the project we know who's looking for that information, how we could best translate that information, who are the groups we should be working with and how we can help implement the recommendations that might come out of a report. It's a much more active approach. Again, I'll make it abundantly clear that we're not the decision makers; there are many other factors that policy makers within the ministries have to consider in making this decision. But our work goes a long way to supporting them in their evidentiary needs.

CW: Yes, I understand; that was going to be my next question. Once the work is done – the scientific work, the conclusions and recommendations, and wherever they've gone – how involved are you in implementation, whether it's something hospital executives should be doing, or heads of clinical departments, or policies the ministries should be following?

BO: We're involved to some extent. I think it's a growing area for us. It's a fine line to walk as well, because we're not the ones involved with implementing policy, so we tend to work closely with individual jurisdictions to support whatever efforts they'd like us to get involved with. Our board has made it clear that they would like us to be a little more involved with implementation support if we can; and its implementation support of course, not implementation. We've worked with some jurisdictions that have academic detailing programs in place, to provide them with information they can use to go out and talk to clinicians.

Any of the work we do typically involves both a policy change and a practice change. We're trying to develop tools and products that speak to both of those groups, and also to the patients. We did some work on smoking cessation drugs, for example, and we produced documents for policy makers; we produced documentation for clinicians; and we produced documents for patients – and lots of different tools. We're starting to delve more into implementation support, but we recognize that there's a fine line between implementation support and implementation.

CW: How do you feel about the influence your work has had or is having on policy and funding, and executive decisions out there?

BO: I think it's a growing influence. We've always had some influence. On the drug side it's been there since the Common Drug Review was established in 2003. But we're having influence on more and more of our projects. For example, we did some work on autism services a few years back, and the Province of Saskatchewan was looking at their policy and how much funding they were putting in to enhance their services. Following our work, they set aside $2.5 million in new funding to enhance their autism services.

As well, we did a report last year for the Province of New Brunswick on magnetic resonance imaging (MRI). They were at the point where they needed to replace all their MRI machines, and they were faced with a tough decision on whether or not to go with the standard magnet-strength – 1.5 tesla machines – versus the more academic and newer 3.0 tesla. The newer machines cost more, so they would have been able to purchase more of the standard. If they went with the higher cost, they wouldn't have enough MRIs for the hospitals in the province, so they were faced with a very tough decision. They asked us to look at the evidence. We put an expert panel together, involving radiologists and clinicians, and we provided recommendations; they based their decision on our recommendations.

We've done some work on robotic surgery and on smoking cessation drugs. More and more we're starting to see that the policy makers require good evidence to support their decision making, and they are relying on organizations like CADTH to help them in that regard.

CW: Yes, indeed. Another aspect, though, with the growing interest in quality and even the issue of quality-based funding arising for the future, is, how do you make a connection between technology assessment and quality of healthcare services?

BO: That's a fundamental concept of the type of work we do. I don't think we ever go into a project simply to say we're about just saving money. Most of the time when we put our experts together, or even our clinicians and scientists internally, the first thing they're looking at is the efficacy, or the effectiveness, of this new technology. We're comparing it with existing technologies. Will it work? Is it of value to the clinicians and the patients? We look at the effectiveness and the harms before the cost-effectiveness and appropriateness, and maybe some of the ethical components too. To me, this whole aspect of quality is fundamental.

What it does well is, if we do a report and find, for example, that there might be overuse of a particular technology and we make recommendations to limit use of that technology, this would perhaps create some space for other technologies that might provide higher-quality care. We're always looking at the type of work we do from two aspects. One is health outcomes or safety and effectiveness; the second is cost-effectiveness or the sustainability of the healthcare system. I think there's a really close link to the quality agenda that most provinces are really focusing on now.

CW: Yes, I think that's the way things are working out for the future, more and more. Sometimes you must look at your work, your suggestions, your conclusions and your recommendations, and be a little frustrated that they're not taken up more quickly or in as much depth as you would like. What do you see as the obstacles to responding more quickly and effectively to good recommendations on health technology assessment?

BO: Some of my staff who've been working a project know the evidence and ask, "Why isn't the province doing something with this great work we've done, even when they have experts that have agreed to this or come to some consensus?"

We've got a classic story with our self-monitoring of blood glucose – the test strips that diabetics use. We primarily focused on type 2 diabetics not using insulin, and we came up with all kinds of good recommendations. We did an economic analysis, and our outcome essentially said, "If practice were to change to reflect the evidence that we demonstrated, over the next three to four years we could save between $450 million and $1.2 billion in this country on those test strips. Stopping coverage of the test strips would eliminate that amount of money that then could be spent on more effective things." So it can be frustrating, but I understand it.

Again, it goes back to the types of things that require change – both to policy and to practice. Practice change can take an extended period of time. We've found that you need to get key stakeholders involved, and early and often. They're the clinical groups, the patient groups, maybe the clinical societies, and the policy makers. You need to understand what their true needs are. Is this relevant to their work? Are they prepared to make some policy changes? Or do other factors need to be considered, like emotion? We know that emotion trumps good policy and good evidence. Are there affordability issues that would require significant changes to a health system? I use two words a lot with my staff: "Be patient but be persistent." These changes will happen. If the evidence is sound, the change will occur, but it does take time.

We're looking at new methods, at what other organizations are doing in a global perspective, and at what the general research community does to effect policy and practice change.

CW: Yes, thanks. Now, you're in the business of evaluating technology and drugs. What is the process for evaluating what you do in your organization?

BO: We've got a number of systems in place. We've been subjected to many evaluations over the years as an organization, because we are a not-for-profit corporation, but our funding comes from both the federal government and the participating provinces and territories. Naturally, evaluation processes have to happen. We went through a very extensive evaluation in 2009. The Conference of Deputy Ministers asked for an independent evaluation of our organization, and they looked at everything – our governance, our product and services mix, how efficient we are in getting our work done and the funding model we use. That resulted in a number of recommendations and a significant transformation for the organization.

We also get evaluated as part of our funding cycle. A significant amount of our funding comes from Health Canada, from the federal government through a funding agreement. It's a five-year cycle, so at the end of every five years there's a formal evaluation of our programs. That goes in through the federal government into the Treasury Board – a normal requirement.

With Health Canada, this year we asked if we could do not just an evaluation to meet the needs of Treasury Board; we also wanted to know if we are really making an impact. Health Canada agreed to a bit of a revolutionary type of evaluation for us. They've done it in two phases. The first focused on programs, and phase two began in March and goes until about October. Looking at our major business lines, the second phase is an overall assessment of our performance, of how we transform the organization, whether it's making a difference, our financial efficiency and our effectiveness. It's a good external evaluation of the organization. Health Canada will be talking to people who use our work, to people who maybe don't use our work because they don't see its benefit, and to ministry folks, our liaison staff, our board members and clinicians across the country.

CW: Would it be fair to say that this evaluation – this more recent one you're speaking of – is less an evaluation of surrogate issues of process and efficiency and so on, and more one of actual outcomes in terms of change in the system?

BO: Absolutely. We made that very clear, and Health Canada, through their vision as well, allowed us to move in that direction. To me that's much more important. Certainly we have all kinds of processes in place to meet the appropriate financial stewardship that's required, but what impact are we having and are there things we could be doing differently to really make a difference?

CW: My next question, and I think you've answered in several bits and pieces, but I'll put it bluntly – why should governments continue to support CADTH?

BO: I think they need to look at us as an investment rather than an expense. They're faced with, and will increasingly be faced with, tough decisions on healthcare technologies. Take the trends happening in pharmaceuticals – there are some 2,500 drugs in various stages of clinical trials, and about 70% of those are in specialty areas. In drugs for rare diseases and in cancer – very important unmet needs that they're dealing with –the drugs will come with a different kind of a business model from industry. A lot of the newer biologics are very expensive.

We're talking a lot about personalized medicine and genomics; in some of the new diagnostic tests, we're getting co-dependent technologies. The ministries are facing tough decisions on funding. We can't fund everything that comes to us. I don't think anybody out there says we can fund every want or wish of every patient or citizen in this country, from prevention to rehabilitation to palliative care. The country would go broke. I think there is a need for organizations, groups and individuals to provide good evidence to support better decision making. That's where organizations like CADTH come in. We're independent; we're providing explicit considerations of the relevant knowledge and looking to maximize the benefit across all disease states – not one specific disease state – hoping to potentiate the capacity for providing healthcare. Again, it's about sustainability and outcomes.

To me it comes down to the rapid pace of change and increasingly complex technologies. Tough decisions will have to be made, and there need to be organizations that can support the policy makers in making those decisions.

CW: Thanks, Brian. I think we've covered the waterfront pretty well, but maybe just two more questions. First, are there any changes you would like to see in the whole field of health technology assessment, whether to the input, the process or the output stage?

BO: Yes, there are. At CADTH, we see our role as twofold. First, we are a producer of health technology assessment, but we also have a brokering role in that we can bring information about what other organizations are doing and share it across the country. We can go into the international context and some of the provincial organizations like the Medical Advisory Secretariat and the Ontario Health Technology Advisory Committee through Health Quality Ontario, and ENES in Quebec. We're working with those organizations to collaborate much more closely, perhaps developing a more common agenda of the type of work we do, with better sharing and linkages, so that we can build capacity. Again, we can't do it all, but collectively we could probably do more.

Second, we have to get better with our timeliness. Certainly, when you're looking at evidence, you like to ensure that you're doing the full gold standard methodological review: systematic reviews, economic analysis. But I hear all the time from decision makers that if you're not timely, you're no longer relevant to them. They need to make these decisions in a timely fashion. We need to get better at providing them with good scientific evidence, but sooner. So we need to use those great brains in our scientists and clinicians to develop better methodologies.

I think there's probably an opportunity for us to work more closely with the regulators as well, and there has been great work happening globally and in Canada on this. We've been working very closely with Health Canada to better understand both the proof of concept, or the regulatory aspect, and the proof of value, or the health technology assessment.

I also want to find out whether there's a way that we and the policy makers and regulators can get more upstream in determining the types of technologies that would benefit Canadians as a whole. Perhaps we could be working with industry to drive the innovation agenda or to better understand what drives their needs – their business model. I that a little more upstream work would go a long way.

CW: Well, there's been lots there…

BO: It's transparency; I think transparency in clinical information and better access to better evidence is what we all need. Certainly, we know that clinical trials are extremely expensive to run. We know there've been issues through the years on suppressing information on negative trials. I think it's extremely important that we all have access to all of the information, to make better decisions. Again, look at different methodologies to assess that information. We want to be as transparent as we can; we post everything that we produce on our website. We'd like to work with industry as well to ensure that we have full access to all the information that's out there.

Comments

Hasmik Safar wrote:

Posted 2013/09/01 at 10:37 AM EDT