Introduction: Jurisdictions are increasingly focusing on appropriate use of healthcare services and interventions as a means to improve health system performance. Our objectives were to conduct a scoping review to (a) map Canadian research and related activity on system-level appropriateness of care and (b) create a resource database that could be used to inform evidence-based decision-making and future research priorities in this area.

Methods: We searched Medline, EMBASE and CINAHL databases between 2003–2013 using terms including "appropriate," "inappropriate," "health technology assessment" and "cost-effectiveness." Articles were included if they were Canadian-based and relevant to our definition. The database search was complemented by a website search of relevant Canadian organizations.

Results: 4,979 articles were identified through the literature search, and 103 articles relevant to system-level appropriateness of care across Canada were charted. Of these, 64 contained an evaluation of appropriateness, 30 used a method of cost-effectiveness or total cost impact analysis and 9 involved another methodology. The most common health service categories included drug therapy (n=40) and health service utilization (n=33). Fifty-eight websites were summarized containing material relevant to system-level appropriateness of care.

Conclusion: Our review identifies Canadian research and related activity pertaining to appropriateness of healthcare from a system-level perspective and provides a useful resource both to support evidence-based decision-making and to guide future appropriateness research.

Introduction

In an era of fiscal restraint and rising healthcare costs, most large health systems and third-party payers, including governments, are interested in the evaluation of the appropriateness of healthcare services. That is, are the right services being delivered to the right people, at the right time, by the right complement of healthcare providers and in the right healthcare setting? Although a common consideration in current discussions of healthcare delivery and sustainability, this concept of appropriate healthcare is not new (Brook 2004). More than 15 years ago, an Institute of Medicine national roundtable categorized healthcare quality problems based on appropriateness considerations (i.e., underuse, overuse or misuse), emphasizing the inherent relationship between quality and cost (Chassin et al. 1998). However, despite considerable interest in evaluating the appropriate use of healthcare, a consistent, operational definition of this concept has not emerged.

A 2008 scoping review by Sanmartin and colleagues explored the existing conceptual frameworks of appropriateness of care and summarized several components, including the key elements of the definition, and importantly, noted the perspective taken. In general, the definitions were clinically focused and were only from the patient and provider perspectives. Based on their review, these authors recommended that the concept of appropriate care be expanded to consider societal values and resource availability.

After a decade of steady and substantial annual increases in healthcare funding, health systems in Canada have abruptly shifted to slow healthcare spending increases. In 2003, Canadian total healthcare expenditures, in current dollars, were $124 billion and were projected to increase to $211 billion in 2013. The major cost drivers of this increase were compensation of healthcare providers, increased service utilization and an evolution in the types of services provided and used (CIHI 2013). In an attempt to rein in spending while not compromising the quality of healthcare being delivered, provincial and territorial health systems are now beginning to focus on how to assess and improve the appropriateness of care. Therefore, it is a critical time to ensure that consideration is given to the broader context for what constitutes appropriate care.

Encouragingly, we are seeing more system-level focus on appropriateness of care, particularly in Canada, by several stakeholder groups. The Council of the Federation's Health Care Innovation Working Group has repeatedly emphasized the importance of appropriateness of care in its initial work, while provincial and territorial health ministers have recently discussed appropriateness of care as relating to the proper or correct use of health services, products and resources. Appropriate use is primarily determined by analyses of evidence of clinical effectiveness, safety and economic implications and other health system impacts (Provincial–Territorial Health Ministers' Collaboration 2013). Aligned with this expanded perspective, the Canadian Medical Association also incorporated consideration of the optimal use of resources in a recent definition of appropriateness of care (Shortt 2013).

Although many groups internationally are involved in appropriateness of care research and activity, the emerging interest in the broader contextual aspects of appropriate care necessitates country-specific perspectives and understanding of system features. With the growing appreciation in Canada for considering appropriateness of care from a system-level perspective, there is a corresponding need to understand the extent of research and related activity underway in Canada. The objectives of this scoping review were to (a) identify Canadian research and related activity on system-level appropriateness of care and (b) create a resource database that could be used to inform evidence-based decision-making and future research priorities in this area.

Methods

We conducted a scoping review (Arksey and O'Malley 2005; Brien et al. 2010) to evaluate the following research question: "What is the extent of system-level appropriateness of healthcare research and related activity in Canada between 2003 and 2013?" At the outset, we engaged experts and stakeholders for advice on defining system-level appropriateness of care, review approach and sources. Methodological details of our scoping review approach are presented below.

Stakeholder consultation

For this preliminary phase of the review, we engaged subject matter experts and other stakeholders. We identified an initial set of stakeholders through the research team's background knowledge of appropriateness of care activity in Canada and identified additional stakeholders through snowball sampling. Stakeholders were engaged in an e-mail dialogue, and some participated in telephone interviews to discuss concepts related to our review of system-level appropriateness of healthcare research and related activity in Canada.

Following the consultations, we formulated a guiding definition of system-level appropriateness of healthcare, and outlined a scoping review approach targeting peer-reviewed literature and web-based material that would best capture relevant research and related activity in Canada.

Guiding definition

Drawing on a preliminary review of the literature and our consultations with stakeholders, we defined system-level appropriateness of care as that relating to the use or non-use of a health service or intervention based on the evaluation of (a) evidence of effectiveness (including net individual health benefit and/or population net health benefit); and/or (b) economic implications (e.g., cost-effectiveness, resource availability, sustainability); and/or (c) other health system impacts (e.g., small area variation); and/or (d) consideration of ethical implications and societal values.

Peer-reviewed literature search

We searched Medline, EMBASE and CINAHL databases using variations of the following terms: appropriate, inappropriate, healthcare delivery, health technology assessment and Canada (see Appendix 1 for detailed search strategy.) Results were downloaded to Endnote (Thomson Reuters 2010) and duplicates removed.

We reviewed material for the time period 2003–2013 that aligned with federal, provincial and territorial governments' agreements on a broad set of health policy objectives and funding contributions through the 2003 First Ministers' Accord on Health Care Renewal and the 2004 10-Year Plan to Strengthen Health Care (Health Canada 2003, 2004).

Screening process

A two-step screening process was conducted: (a) title/abstract screening, followed by (b) screening of full-text articles. Inclusion and exclusion criteria (see below) for the title abstract screening and full-text review were similar and based on our guiding definition of system-level appropriateness of care, with additional criteria added for the full-text screening process.

Abstracts were included for a full-text review if one or more of the inclusion criteria were respected: (a) there was mention of the evaluation or impact of appropriate or inappropriate use of a health service or intervention; (b) there was mention of an analysis of effectiveness, cost-effectiveness or economic impact of a health service or intervention; and (c) there was mention of topics relevant to system-level appropriateness, including health technology assessment, small-area variation, ethical and societal implications or the names of relevant organizations. Articles were also included where the scope or focus could not be determined from the title/abstract alone and further information was required.

Studies were excluded if (a) the major jurisdiction under examination was not within Canada, (b) none of the authors were based at a Canadian institution or (c) it was apparent that key terms used in the article or the focus of the article was not related to system-level appropriateness as defined in this review. As is common with scoping reviews, we refined our inclusion and exclusion criteria as we gained familiarity with the material (Arksey and O'Malley 2005; Brien et al. 2010). In certain cases, the inclusion or exclusion criteria were modified as new themes and additional topics emerged during the screening process.

The full text of articles included in the title and abstract screening phase were retrieved by two team members (GG and YT) and reviewed in greater depth according to the inclusion/exclusion criteria. Studies that fell outside the scope of system-level appropriateness as defined in this review, or where the principal focus lay outside of Canada, were excluded. Additionally, commentaries were excluded at this stage. Any articles that could not be retrieved electronically (following a search of two academic library electronic journal databases) were also excluded.

Charting

Information from the full-text screening was charted according to broad categories that were identified through the expert and stakeholder consultation and that evolved as the full-text screening proceeded. We were interested in information related to the type of health service or intervention being evaluated (e.g., was it a drug therapy, use of a health service provider, a particular procedure, a specific healthcare delivery setting such as primary care or emergency department), the type of analysis used in the evaluation (e.g., what type of appropriateness evaluation was used, what type of economic evaluation was used) and the clinical areas represented (e.g., cardiovascular care, chronic disease, mental health) or populations subjected to the intervention (e.g., primary care, paediatrics, seniors). We also wanted to know the type of system that was the setting for the evaluation: was it a hospital system, health region, province or pan-Canadian?

Grey literature search

To complement the search of peer-reviewed literature, a targeted search of websites of relevant organizations (e.g., health technology assessment agencies, provincial quality councils) was conducted. We focused on Canadian-based organizations and did systematic keyword or direct webpage searches for materials pertaining to system-level appropriateness of healthcare.

An initial list of relevant Canadian organizations and government units was drawn from Health Technology Assessment on the Net International: 2013 (Institute of Health Economics, Osteba: Basque Agency for Health Technology Assessment, and Agencias y Unidades de Evaluación de Tecnologías Sanitarias 2013). Additional organizations were searched as they appeared in the results of the initial website review, such as other Canadian organizations and networks or collaborative groups. Furthermore, we included the websites of provincial and federal departments or ministries of health, as well as other organizations recommended during our initial stakeholder consultations. Members of the research team reviewed the list and made suggestions about entities to add or delete, based on alignment of the entity's activities and the guiding definition of system-level appropriateness of healthcare.

Screening process

The website searching took place over the period of August to December 2013, with a minor update made in April 2014. Websites in English and French were reviewed. Two strategies for searching websites were used: (a) utilizing the websites' own search functionality (when available) to search for a standard set of terms and (b) systematically navigating for content on the website.

For the first search strategy, a pilot search was performed in which a subset of websites was reviewed using a set of keyword search terms: "appropriateness," "inappropriate," "inappropriateness," "cost-effective," "health technology assessment," "small-area analysis," "health services utilization" and "disinvestment." The terms were searched using each website's own search functionality, and search results were reviewed for the following material: publications, resource lists, resource types, initiatives, priorities, partnerships and affiliations.

Only websites with existing search functionality were subject to the first search strategy. The second strategy allowed for searching of websites without search functionality. For this approach, main pages of websites were reviewed for sub-pages related to publications, resources, activities and so on. Relevant sub-pages were reviewed for material related to system-level appropriateness of care as described by our guiding definition.

Websites were excluded from the search if they were not deemed to be Canadian organizations or did not have relevant Canadian material.

Charting

For charting, we used similar inclusion criteria as outlined for the peer-reviewed literature search that were based on our guiding definition of appropriateness of healthcare. The relevant material contained within the websites was summarized. Main activities related to system-level appropriateness of healthcare were noted, as were any resources available related to the topic. Demographic information such as organization name and web address was also charted.

Results

Search results

A total of 4,979 peer-reviewed articles identified through the database search went through title/abstract screening; 604 were identified for full-text screening, and 103 articles were charted as relevant Canadian academic literature pertaining to system-level appropriateness of care (Figure 1).

Overall, 73 organizations were included in our initial web-search list. Of these, 15 were excluded: 11 were excluded at the outset as not being Canadian organizations or containing any Canadian activity related to system-level appropriateness of healthcare, while the remaining four were not searched for other reasons (website not available, a duplicate link or organization folded into another organization). In total, 58 websites were searched and charted (Figure 2).

Charting results

Common themes began to emerge as charting of peer-reviewed articles evolved. These included the type of analysis used in the evaluation, the type of health service or intervention being evaluated, the clinical area or population receiving the health service or intervention and the scope of the evaluation. These themes formed the main categories used to chart the data in this scoping review. We also charted demographic information (e.g., primary author, year, title) and a brief description of the rationale for the study. Web Appendix 2 contains the complete database of academic literature charted.

Of the 103 articles charted, 64 contained an evaluation of appropriateness, 30 used a method of cost analysis and the remaining nine articles used other methodologies (Figure 3). Appropriateness of care in these studies was determined using a variety of methods, including comparisons of practice to guidelines (n=40), a set of appropriateness criteria (e.g., Beers criteria, consensus statements) (n=5), comparisons of prescription patterns to drug indications (n=2), use of the RAND/UCLA appropriateness rating method (Fitch et al. 2001; Shekelle 2004) (n=2), peer review (n=2) and a combination of other methodologies such as peer review or consensus recommendations (n=13). Markov modelling was the most common method of cost analysis used among those articles examining cost-effectiveness or cost impacts of health interventions (n=6).

With respect to the health service or intervention being evaluated for system-level appropriateness of care (Figure 4), the two most common categories were drug therapy (n=40) and health service utilization (e.g., use of emergency departments, preventive care) (n=33). There were 14 common clinical areas identified in the included articles (Figure 5), with cardiovascular care being the clinical area with the most articles related to system-level appropriateness (n=21). A 15th category of "other," which includes a variety of clinical areas, was also identified (n=20).

In contrast, there was considerable variation in the amount of material available on the websites searched. We identified information noting considerable activity, interest and awareness in system-level appropriateness of healthcare that extended beyond traditional sources of grey literature (e.g., reports, briefing notes, slide presentations). Therefore, to fully capture the large volume and diversity of information identified during the web search, we created a database of organizations in Canada that are involved with system-level appropriateness research and related activity rather than documenting specific system-level research activities underway, which were inconsistently described. We charted the information according to the type of activity underway at the organization and the types of resources available on the websites searched. In many cases there were multiple activities underway and tools available, and this is captured in the numbers below.Web Appendix 3 contains a complete database of web-search results.

We categorized the activity into the following, noting numbers and definitions: health technology assessment (n=12), reviews (e.g., systematic reviews, meta-analyses or rapid reviews) (n=6), drug reviews (e.g., evaluation of evidence for inclusion or exclusion from a formulary) (n=12), non-scientific evaluations (e.g., commentaries, policy reviews or position papers) (n=14), committees/networks (e.g., groups assembled to produce reviews, or network collaborations) (n=25), methods development and/or training (e.g., for reviewing appropriateness, scientific training) (n=12), funding agency (e.g., provides research funding with targeted funding for appropriateness of healthcare research) (n=3) and quality measurement (e.g., measures healthcare quality and performance, with appropriateness of healthcare as a dimension of quality) (n=8).

Resources were categorized as reports or tools, and instances where limited information was available were noted. Reports included white papers, reports to governments, peer-reviewed publications; these could include recommendations, decisions or new methods. Tools included guidelines, performance measurement or quality frameworks, training tools or searchable databases of material pertaining to appropriateness of healthcare. In some cases the websites contained limited resources, and this was noted. Overall, 15 websites contained reports only, 14 contained tools only, 27 contained both tools and reports, and two had limited information.

Discussion

Based on a novel definition for appropriateness of healthcare that incorporates broader contextual features of the healthcare system, our scoping review mapped research and related activities on system-level appropriateness of healthcare conducted by researchers and health services organizations in Canada. This endeavour has resulted in two detailed databases that can be used to inform research, policy and decision-making for healthcare in Canada. The first database (Web Appendix 2) contains peer-reviewed research completed to date that evaluates aspects of system-level appropriateness of care in Canada. The second database (Web Appendix 3) contains a list of health services organizations involved in funding, evaluating or supporting system-level appropriateness of care research and related activities in Canada.

Some interesting themes emerged. First, the academic literature pertaining to this topic to date has focused mainly on health services utilization or drug therapies, with use in accordance to guidelines and cost-effectiveness being the most commonly used evaluations. Furthermore, cardiovascular services or interventions are currently the most commonly evaluated area from a system-level perspective. In contrast, recent focus of governments and policy makers has been on appropriate use of diagnostic imaging as a means of reducing healthcare costs (Busse et al. 2013; Canadian Foundation for Healthcare Improvement 2013; Medical Imaging Team Day Sponsor Organizations 2012). While we did identify some research related to system-level appropriate use of diagnostic imaging (Alter et al. 2006; Butler and Stolberg 2004; Kent et al. 2004; Landry et al. 2011), many related articles were excluded as they were clinically focused and did not capture the broader elements of system-level appropriateness of care as defined in this study (Butler and Stolberg 2004), may have evaluated variations in use but did not evaluate appropriateness (You et al. 2008) or were commentaries (Laupacis and Evans 2005; You 2009).

Secondly, of the academic literature reviewed that contained evaluations of appropriateness, the methods of determining appropriate care most commonly used were comparison of the care provided versus guidelines for care or other pre-determined criteria (e.g., Beers criteria). This is in contrast to the scoping review published by Sanmartin and colleagues (2008) that found the most commonly used method of determining appropriateness was the RAND/UCLA model (Fitch et al. 2001; Shekelle 2004). This model is based on a type of group consensus, where clinical experts decide on the relative weighting of risk to benefit in a given clinical scenario. Furthermore, several limitations of this model were discussed in the scoping review of Sanmartin and colleagues (2008), including the focus of this method on clinicians' opinions and the lack of consideration of the patient or societal perspective. Given that we were focused on a system-level perspective of appropriateness of care, it is not surprising that we found limited use of this clinically focused method in our review.

Finally, from our review of web-based material, we were able to determine that many health services and related organizations in Canada note appropriateness of care as an activity of interest, whether this be conducting research, funding research, or providing recommendations or summaries. Given the volume and diversity of information identified, it is clear that there is considerable interest but also disparate and often nascent approaches to assessing appropriateness of care in Canada. This finding raises the question of whether there is opportunity for alignment of this work to reduce duplication and focus efforts on common themes that arise across the country. One promising activity that is gaining momentum in Canada is the Choosing Wisely Campaign that launched in April 2014, and which may provide a foundation for alignment.

This scoping review has several limitations that should be taken into consideration. First, scoping reviews, by their nature, do not evaluate the quality of material gathered or the rigour with which evaluations are conducted (Arksey and O'Malley 2005). Therefore, it is beyond the scope of this study to evaluate the quality of the research in Canada related to system-level appropriateness of care. Second, our guiding definition of system-level appropriateness of care focused on a specific level (i.e., the health system) of appropriateness activity, and therefore we excluded those studies that were more clinically focused. Third, our search for grey literature turned up an abundance of information and activity related to appropriateness of healthcare in Canada. However, it is likely that we missed other important sources of information, and we therefore have not fully represented the extent of system-level appropriateness of healthcare research and related activity in Canada. Finally, we chose to focus our review on work that was based in Canada; therefore, we excluded international work that can also be used to inform appropriateness work in Canada. However, aspects such as funding and geography make the Canadian healthcare system distinct from those of other countries. Thus, a focus on appropriateness research and related activity in the Canadian context is warranted.

In conclusion, system-level appropriateness of care research is underway in Canada, and many health services and related organizations are interested in studying this aspect of healthcare. At the same time, governments and policy makers across Canada are increasingly interested in delivering more appropriate healthcare in order to curb rising healthcare costs and improve the overall quality of healthcare. As such, considerable opportunities exist for more focused appropriateness of care research and related activity to meet these needs. The databases created from our scoping review can be used as resources for identifying both the locus of relevant expertise and the key gaps in our knowledge of system-level appropriateness of care. Importantly, the information therein can facilitate the development of more collaborative efforts among groups of Canadian researchers and decision-makers to pursue needed advancements in this area.

Étude de la portée des activités de recherche sur la pertinence des soins au Canada, du point de vue du système

Résumé

Introduction: Les autorités compétentes se penchent de plus en plus sur un usage approprié des services et interventions de santé pour améliorer le rendement du système de santé. Nous avons effectué cette étude afin de (a) cartographier la recherche et les activités connexes sur la pertinence des soins au niveau du système au Canada et (b) créer une base de données des ressources qui pourraient servir à éclairer la prise de décision fondée sur les données probantes et à déterminer les priorités de recherche dans ce domaine.

Méthodes: Nous avons consulté les bases de données Medline, EMBASE et CINAHL, entre 2003 et 2013, à l'aide des termes « approprié », « inapproprié », « évaluation des technologies de la santé » et « coût-avantage ». Nous avons retenu les articles portant sur le Canada et correspondant à notre définition. Pour compléter la consultation des bases de données, nous avons effectué une recherche sur les sites Web d'organisations canadiennes pertinentes.

Résultats: La recherche documentaire a permis de répertorier 4979 articles, dont 103 sur la question de la pertinence des soins au Canada. Parmi ceux-ci, 64 comprenaient une évaluation de la pertinence, 30 faisaient appel à la méthode coût-efficacité ou à l'analyse de l'impact du coût total et 9 employaient une autre méthodologie. Les catégories de services les plus abordées sont la pharmacothérapie (n=40) et l'utilisation des services de santé (n=33). Nous avons regroupé 58 sites Web comprenant du matériel sur la pertinence des soins au niveau du système.

Conclusion: Notre étude a permis de répertorier la recherche et les activités connexes qui portent sur la pertinence des soins du point de vue du système au Canada. Elle fournit également une ressource utile pour appuyer la prise de décision fondée sur les données probantes et pour orienter d'éventuelles recherches sur la pertinence des soins.

Comments

Be the first to comment on this!