Senior decision-makers in the Canadian healthcare system have to continuously make significant, and complex, policy and program decisions. However, it appears that, often, the evidence they have available is fairly simple descriptive information, collected for operational purposes. Trying to solve complex problems with fairly simple data may lead to suboptimal decisions. This article presents a new knowledge development system (KDS) that should allow senior decision-makers and others to manage smarter and take their decision-making to the next level. A KDS represents the integration of information systems, and research and analysis, into one system. It can generate sophisticated, strategic information around complex issues, which should ultimately lead to wiser decisions. This article describes the KDS, provides an example of a current KDS and concludes by presenting a self-diagnostic tool for decision-makers to allow them to determine whether their organization could benefit from a KDS.

Healthcare organizations such as ministries of health, regional health authorities and other organizations collect large amounts of data. However, they appear to struggle with translating these data into strategic knowledge and insights that can be used as inputs into evidence-based decision-making at the clinical, operational, administrative, policy and executive levels. Several reasons seem to account for this difficulty. Information systems are often developed to meet the operational needs of different organizational components. For example, separate systems are developed for finance, human resources and care delivery. Some types of data that exist outside the organization and are critical for certain types of analyses – such as data on the population served (for population health and epidemiological analyses) – may not be readily available or, if available, may not be systematically integrated into the data architecture of the organization. In addition, over the past several years, the focus seems to have been on developing information systems and electronic health records rather than on analyzing data to take maximum advantage of the data that are already available. Thus, organizations may have suboptimal knowledge development, not because of a lack of data but because the data that exist are not fully used to generate new knowledge. Finally, because data may not be used to meet the real needs of organizational actors (e.g., front-line care delivery staff, policy developers, planners etc.), people may not recognize the potential of existing information systems to provide insights into key issues. Thus, a separation often exists between collecting data and using the data to develop new knowledge in healthcare organizations.

In order to obtain more enhanced information, a variety of attempts have been made to develop balanced scorecards, indicators, benchmarks, decision support systems, executive information systems, integrated patient records and so on. However, these efforts appear to have been met with limited success to date.

While we recognize that there is still considerable organizational value in existing information systems that provide data and descriptive information for operational purposes, we argue here for a major paradigm shift in which the priority focus is on strategic knowledge development rather than on operational information. Organizations that want to have more sophisticated and usable information may wish to consider developing a knowledge development system (KDS) to meet their strategic information requirements. Figure 1 shows how we can move from existing reality to data and information (using information systems), to knowledge (through sophisticated analyses of information), to more informed, contextually sophisticated and wise decisions, to the implementation of new policies and programs.

Components of a KDS

The first component of a KDS is what we refer to as an analytical database. This is an electronic space in which data extracts from a variety of existing corporate information systems are maintained on an ongoing basis. It should also include key information from external resources, such as information on the age and sex distribution of the population served. The data collected can be linked, across the data extracts, by using the same unique identifiers for organizations (e.g., hospitals), care providers (e.g., physicians) and patients once appropriate protocols have been put in place to ensure privacy or the data have been rendered anonymous.

The analytical database has many of the benefits of a completely integrated information system – the "Holy Grail" of informatics – but does not operate in real time and is therefore much easier to construct while providing many of the desired benefits of more complex systems. It can be established relatively quickly as it uses information already collected by the organization. Thus, a system of extracts from existing information systems is developed. These extracts can be done annually, quarterly or monthly, depending on the needs of the organization. Analysts then pull the data they need to conduct their analyses from the set of data extracts that have been developed. The analytical database team functions to support the members of the organization in obtaining their information needs quickly and in a manner that is suitable to them.

There are five levels in the KDS. The first three levels of the KDS refer to the levels of comprehensiveness of the analytical database. Level one may not contain client-specific data. Nevertheless, usable, strategic information can still be obtained. An example of a level one analytical database is the Statistics Canada Residential Care Survey. It contains data about care providers and the type and size of facilities (e.g., government-owned with 200 beds, or owned by a for-profit organization with 50 beds). There is also a cost breakdown (based on the budget), data on revenues and data on the number of clients at each of the five federal levels of care need. These data allow for the analysis of residential care services across provinces, across types of ownership and size, by the level of care needs of clients, and by cost per bed. For example, an analysis of long-term care facilities revealed that small facilities typically had a lower per-bed costs than did medium and large facilities, and that the data did not appear to support the argument for economies of scale for larger long-term care facilities (Hollander 1994).

A level two analytical database is a partial database. It is typically the kind of database a service delivery organization or department would have. For example, community care access centres in Ontario have data on clients and providers that allow for a range of useful analyses. Other examples are the data available in hospitals and the hospital abstracts. Thus, level two analytical databases have data on client characteristics (and other client-specific data), presenting conditions, sources of referral, health status and care providers. In some cases, cost data may also be available at this level.

Level three analytical databases are more comprehensive, contain extracts from a wide range of service delivery databases and have all the elements captured in levels one and two. In addition, they build in modules for financial data, population demographics and other types of data. A key issue in developing level three analytical databases is to ensure compliance with privacy legislation. Fortunately, most privacy laws have provisions that allow for the protection of privacy while permitting data to be used for research and analysis purposes.

Level four in the KDS moves from the collection and storage of data to analysis. At this level, a small team of highly trained and experienced analysts can access the level three analytical database to develop new knowledge. Analysts can engage in a wide variety of knowledge development activities including scenario building, simulation and other forms of analysis.

Level five of the KDS includes a range of activities that are complementary to, but do not use or only partially use, administrative data. These activities focus on knowledge development through special, targeted studies on key topics of policy or program interest, surveys, program evaluation activities, quality assurance and so on.

A schematic of how the five levels of the KDS fit together to produce reports, indicators and a new knowledge base for future decision-making is presented in Figure 2. As is illustrated, a KDS incorporates both basic and advanced standard reports. It supplements these reports with more in-depth and sophisticated analysis and special studies, surveys etc. that have policy and/or program relevance. Indicators are developed out of the analytical process. It is our view that this is the best route as it allows a consensus to develop on the relevance and meaning of the indicators through the analytical process and interaction with decision-makers. The key shortcoming of indicators developed on an a priori basis is that they are single items of information (e.g., low unit cost can be "good" and "efficient" or it can be "bad" because the organization is under-resourced or has engaged in cost shifting to patients). Thus, a priori indicators are end points and answer what but not why. For example, they can identify what the per capita admission rate is to a local hospital but cannot explain why this rate may be high or low compared to a provincial average.

Proposed Paradigm Shift (Managing Smarter)

It is our experience that policy makers, planners, clinicians and others appear to have resigned themselves to using suboptimal information for their decision-making. It is understandable that due to fiscal restraints and other factors the primary focus for organizations has been on operational data and day-to-day survival in a fiscally challenging environment. We, however, argue that taking even a modest amount of time to focus on strategic issues and developing new knowledge may pay significant dividends in terms of identifying new approaches to cost avoidance and greater value for money for the organization. The KDS described in this article is relatively inexpensive and can be implemented fairly quickly.

Some of the key issues and challenges related to implementing a KDS are discussed below.

Structure and Costs

In the late 1960s and early 1970s, organizations had in-house analytical units (Hollander and Prince 1993). Such units, focused on high-level strategic analysis, could again be established inside an organization. Another option would be to hire a small team of experts in research, analysis and informatics. To be effective, this team should report to the chief executive officer or to a vice-president. Another option that has been used in some jurisdictions is for ministries of health to contract with a university-based research group. To date, most such approaches have mainly focused on levels one to four of the KDS. The costs of a KDS can be quite modest as only a small number of high-level analysts are required. Experience indicates that a well-functioning KDS usually more than pays for itself.

A Focus on Analysts

There are numerous examples of information systems advocates developing executive or management information systems. These systems contain various sets of aggregated data that can be readily accessed by decision-makers, but they are often quite expensive to develop. While basic operational data should be retained, at the strategic level fundamental differences between the executive information systems approach and the KDS are (1) in our view, senior executives do not wish to be analysts and would rather ask key policy questions and (2) executive information systems generally provide only simple descriptive data to address complex and multi-faceted policy and program issues. In the KDS, organizational actors formulate questions in discussion with the KDS staff, and the KDS staff conduct, as required, comprehensive and sophisticated analyses to answer, or at least shed new light on, the questions posed.

Corporate Priority on Knowledge Development

Organizations may wish to add a strategic knowledge development lens to their decision-making on issues related to data and information. For example, some software vendors develop software that works for operational purposes but from which data cannot be extracted for inclusion into an analytical database. Thus, software purchases and other related issues should be reviewed to ensure they contribute to, rather than impede, knowledge development.

Range of Analyses

It is helpful for an organization to ask the strategic question, to what extent and under what conditions can one type of program substitute for another? These are the kinds of questions that can be addressed by a KDS. For example, an anonymized analytical database has existed at the University of British Columbia since the mid-1980s. An analysis of the administrative data revealed that there was considerable potential for a cost-effective substitution of home care services for residential care services. Furthermore, at a policy and program level, steps were taken to actually substitute home care for residential care services, resulting in significant cost avoidance for at least 10 years (Hollander and Chappell 2007).

Organizational Issues

There may be turf and data access issues, regarding who has control over what data within a given organization. These matters need to be addressed so that analysts can get regular data extracts to do their work, and analytical resources are recognized as valuable corporate resources.

Getting Started

As noted above, there are several potential approaches to developing a KDS. Organizations must determine which approach or combination of approaches best meets their needs, context and constraints. One of the main challenges in a KDS is to engage the right people. They need to be well trained and have a wide range of experience in analysis and informatics. They also must be familiar with several quantitative and qualitative research approaches. Further, it is important to hire analysts with experience in or knowledge of management, policy formulation, program development and operations.

Example of a KDS

The General Practice Services Committee (GPSC) is the main driver of strategic change in primary care in British Columbia (GPSC 2009). In 2007, Hollander Analytical Services Ltd. was retained to conduct an evaluation of the main GPSC initiatives. There were two primary areas to be evaluated: the Full Service Family Practice Incentive Program, which funds incentive payments to family physicians through existing fee schedules; and the Practice Support Program, which provides training on various topics for family physicians.

In order to perform the evaluation, a KDS was developed and implemented. An extensive process was conducted to obtain access to extracts from ministry databases for hospital services, medical services, pharmacy services, and home care and residential long-term care. Extracts from other databases may also be added in the future. A protocol to ensure the privacy and confidentiality of data was developed and approved by the BC privacy commissioner. Thus, a level three analytical database was developed. The structural model for the KDS was one of hiring an external team of experienced experts in research, evaluation and informatics. The team consists of four experts, working part-time. Back-up resources are also available, as required.

Administrative data were obtained for each of the areas for which incentive payments had been introduced – that is, chronic disease management (diabetes, congestive heart failure and hypertension), complex care, mental health and obstetrics. Reports were produced for each topic area and for other topics such as the uptake of incentives and continuity of care. In addition to these reports, a new and groundbreaking article was published on the relationship between attachment to practice and costs (Hollander et al. 2009). This article demonstrated that for patients with high care needs, an overall increase in the level of attachment to their provider of as little as 5% could potentially result in an annual cost avoidance of some $85 million. The article used multivariate statistical analysis to ensure that the inverse relationship between attachment and cost was not due to other factors such as the age distribution of the population. This is an example of the type of analysis conducted at level four of the KDS. The findings have had a major impact on policy and program development in primary care in British Columbia.

In level five of the KDS, a number of surveys were conducted. It was found that the training programs of the Practice Support Program appeared to be quite effective (MacCarthy et al. 2009). A survey was conducted of general practitioners (GPs), which stratified respondents into high, medium and low billers of incentives. This allowed for an analysis, across a wide range of topics, in regard to the perceptions of GPs based on their billing practices (Miller et al. 2009b). In addition, patients were randomly selected from each group of GPs in order to obtain their perceptions of care based on the billing practices of their physicians (Miller et al. 2009a). It was not possible to develop physician-patient dyads due to privacy issues; thus, data were analyzed on a group basis. Nevertheless, these examples show how one can integrate administrative data with survey data to obtain a more complete and nuanced picture of a given topic area.

The overall program of research for GPSC was documented at the beginning of the evaluation in the project work plan. The key results from all the reports produced for the project (up to June 2009) were integrated into a synthesis report that addressed the key research questions in the work plan (Hollander 2009). A knowledge transfer and dissemination framework was also developed for the project.

Could Your Organization Benefit from a KDS?

The following questions are a diagnostic tool for senior executives to determine whether or not their organization could benefit from a KDS:

• Do your analysts (in house or external) have access to patient- and provider-specific data as approved by the appropriate privacy authority, or have access to data that have been rendered anonymous?
• Do your information systems people regularly provide extracts at the patient and provider levels of detail to your analysts, or allow them to access organizational databases to do their own extracts?
• Does your organization regularly conduct cost and utilization analyses (standardizing for levels of care need) across key care delivery components to determine which services can provide good quality care at the lowest costs (e.g., home care and residential care, or preventive home support and hospital care)?
• Do your analysts use more sophisticated statistical software such as SPSS, the statistical package in SAS, or similar software?
• Do your analysts use advanced statistical techniques (e.g., multivariate analysis) in their work, as required?
• Do you receive at least two in-depth reports per year on key strategic issues?
• Does your organization have clear documentation indicating that strategic knowledge development is a top corporate priority?
• Does your organization regularly conduct special studies, surveys etc. to obtain new knowledge on key outcomes or on major strategic initiatives?

If you answered yes to all these questions, you essentially already have a KDS. If you answered no to all or most of the questions, you may wish to consider whether developing such a system would be appropriate for your organization.

Comments

Be the first to comment on this!