Strategic Analytics: Towards Fully Embedding Evidence in Healthcare Decision-Making
Cancer Care Ontario (CCO) has implemented multiple information technology solutions and collected health-system data to support its programs. There is now an opportunity to leverage these data and perform advanced end-to-end analytics that inform decisions around improving health-system performance. In 2014, CCO engaged in an extensive assessment of its current data capacity and capability, with the intent to drive increased use of data for evidence-based decision-making. The breadth and volume of data at CCO uniquely places the organization to contribute to not only system-wide operational reporting, but more advanced modelling of current and future state system management and planning. In 2012, CCO established a strategic analytics practice to assist the agency's programs contextualize and inform key business decisions and to provide support through innovative predictive analytics solutions. This paper describes the organizational structure, services and supporting operations that have enabled progress to date, and discusses the next steps towards the vision of embedding evidence fully into healthcare decision-making.
|LIST OF ACRONYMS|
|CCO||Cancer Care Ontario|
|MOHLTC||Ministry of Health and Long-term Care|
|ISAAC||The Interactive Symptom Assessment and Collection|
|ORN||Ontario Renal Network|
|ORRS||Ontario Renal Reporting System|
|CKD||chronic kidney disease|
|WTIS||Wait Time Information System|
|CSQI||Cancer System Quality Index|
|LHINs||local health integration networks|
|RCPs||Regional Cancer Programs|
|SAAP||Strategic Analytics Advisory Group|
|gFOBT||Guaiac-based Fecal Occult Blood Test|
|FIT||faecal immunochemical test|
|PAO||Privacy and Access Office|
|PHIPA 2004||Personal Health Information Protection Act|
|IPC||Information and Privacy Commissioner of Ontario|
Cancer Care Ontario (CCO) supports quality improvement in prevention, screening and patient care, including patient experience. The agency's work ranges from monitoring and evaluating health-system performance to leading the design and implementation of new healthcare practices and models.
CCO's Information Technology, Tools and Databases
To support quality improvement, CCO has implemented multiple information technology (IT) solutions to collect, manage and report on health system data. Within CCO, there are over 170 data holdings, which include data CCO collects from external partners (e.g., Ministry of Health and Long-Term Care [MOHLTC], Canadian Institute for Health Information and Institute for Clinical Evaluative Sciences), as well as data collected from Ontario health providers and organizations, which is then enriched and linked within the agency. Some examples of these enriched datasets are:
The Interactive Symptom Assessment and Collection (ISAAC) tool is designed to capture patient-reported, cancer-related symptoms primarily through touch-screen kiosks located at hospitals and Regional Cancer Centres. The extensive and timely data collected through ISAAC are communicated to patients' healthcare providers to help them monitor and manage their patients' symptoms. As of February 2014, approximately 285,000 unique patients submitted over 2.1 million symptom screens into the ISAAC database, providing CCO with the largest patient-reported cancer symptom dataset in the world.
Using a web-based application, the Ontario Renal Network (ORN) collects pre-dialysis, acute dialysis and chronic dialysis patient-level data from across the healthcare system for the Ontario Renal Reporting System (ORRS). This dataset is used to report on a range of chronic kidney disease (CKD) indicators to health professionals, policymakers and researchers.
The Wait Time Information System (WTIS) is a web-based tool used to collect and to report wait time information for over 190 procedures in 13 key surgical areas and diagnostic imaging cases. Over 665,700 surgical cases and more than 1.7 million diagnostic imaging cases were entered into the WTIS during fiscal year 2012–2013, making it the most comprehensive wait time information database publicly available in Canada. Approximately 3,100 clinicians use the WTIS to help manage wait times in 96 hospitals across Ontario. Policymakers and planners also use it to inform funding decisions and resource allocation, as well as to identify performance improvement opportunities. Moreover, WTIS is used to collect near real-time data on Alternate Level of Care patients, a designation given by a physician to a patient who is occupying a bed in a hospital, but who does not require the intensity of resources or services provided in that particular care setting.
CCO's Data Strategy
CCO's role in quality improvement has steadily expanded over the past several years. This has been accompanied by a simultaneous growth in both the volume and type of data collected in the organization. As that data expansion continues, there is a need to explore information flow throughout the organization, from its collection and infrastructure to its management and reporting. Additionally, there is an opportunity to link together and leverage this information for program and capacity planning, implementation and evaluation. As such, CCO is engaging in the first comprehensive data-focused organizational and stakeholder review to gauge current and future data needs, both from an institutional and patient-focused perspective. The application of data for operational planning and reporting has always been a clearly identified priority. However, the application of data to predict and model future state is, to a large degree, in its infancy. This will lead to the development of CCO's first enterprise-wide data strategy. The organization will place a significant focus on utilizing current data and driving future acquisition, to significantly increase capacity for predictive modelling and strategic analytics (SA) for system planning and evaluation.
CCO's Strategic Analytics Practice
In recent years, the resource constraints experienced by health systems across Canada have increased the need for innovations that improve the efficiency and effectiveness of the management and delivery of healthcare. At CCO, the intersection of these trends with the agency's data infrastructure has led its programs to an increasing pursuit of data-driven insights, such as the Cancer System Quality Index (CSQI), iPort and iPort Access.
The first of its kind in North America, CSQI helps drive CCO's mission of improving services and outcomes related to cancer within Ontario. The tool is produced with and on behalf of the Cancer Quality Council of Ontario and tracks Ontario's progress in a number of evidence-based quality measures and informs policymakers, physicians and researchers where quality improvements are needed. For example, the CSQI has highlighted improvements in access to care through decreased wait times for cancer services and in the effectiveness of care through improved surgical outcomes by consolidating complex care for cancers of the lung, esophagus, pancreas and liver (Anas et al. 2012). Now in its 11th year, CSQI continues to shine a light on areas for improvement in Ontario's cancer system.
iPort and iPort Access are secure, web-based tools developed to help provide accessible information about cancer surveillance and activity at the provincial, regional and Local Health Integration Network (LHIN) level, as well as wait time and surgical efficiency data collected through the WTIS. These tools are helpful for health system planning using analytical reports encompassing a number of key metrics. Individuals with access to these tools can generate detailed analytic reports that include information with respect to geography, time, age and diagnosis time. Additionally, users can access comprehensive dashboards for up-to-date information on health system performance. An example of a dashboard is the Cancer Regional Performance scorecard, which is a performance management tool that monitors and measures the performance of the Regional Cancer Programs (RCPs) against provincial priority indicators and targets. The purpose of the scorecard is to help drive system improvement and address poor performance, as part of CCO's quarterly performance review process. The electronic Scorecard enables users to view the underlying data supporting each metric, allowing for more detailed analysis to support performance improvement. Users are able to visualize trends in performance, make regional comparisons against targets and gain an overall picture of provincial performance where RCPs are ranked 1 through 14.
Strategic Analytics Advisory Group
To oversee the development of the SA practice, an external advisory panel was instituted in 2012. The Strategic Analytics Advisory Group (SAAP) is composed of nine thought leaders from across the healthcare sector. The panel was established to have leaders in the healthcare information space, external to CCO, act in an advisory capacity to offer strategic insights, thus propelling CCO's achievement of its goal to enhance healthcare decision-making by generating richer evidence. The SAAP meets with the SA team and members of the CCO leadership three times a year to help identify opportunities for advanced analytics based on their understanding of the nuances of analytics and the challenges facing the healthcare system.
Strategic Analytics Projects
Since its inception, SA has led a number of projects. One recent example involved working with Access to Care (one of CCO's main programs) on the development of a model to support the province's LHINs in predicting the required number of hip and knee replacement surgeries to meet and sustain wait time targets during fiscal years 2013–2014 and 2014–2015. The model relied on WTIS data for up-to-date wait list information, and used time-series forecasting to predict wait list arrivals, seasonal variations to estimate monthly surgical volume and queuing theory to estimate the average and 90th percentile wait times.
Another example is a current project with CCO's cancer program in support of the Ontario Cancer Plan III's commitment to develop patient-centred models of care (Cancer Care Ontario, 2011). The project involves developing an interactive decision support tool that is heavily based in CCO's data holdings. The resulting model enables users to assess the impacts of followup care devolvement, as well as repatriation or centralization of specific types of care, to determine the required number of medical, gynaecological and radiation oncologists.
CCO's Cancer Screening team is also engaged in a SA project to predict the impact on the Ontario healthcare system associated with switching from the current colorectal cancer screening test (Guaiac-based Fecal Occult Blood Test, or gFOBT) to the more sensitive Fecal Immunochemical Test (FIT). A multi-component system impact model is being built to support the planning activities around the implementation of FIT. One component of the model will predict volume shifts in cancer cases based on important Ontario-specific cancer screening parameters as identified by CCO. The other component of the model uses these results to build a final demand model using multiple CCO data sources, which focuses on predicting changes in the patterns of health system use associated with cancer screening.
In 2013–2014, SA developed a strategic regional capacity planning tool that encompasses a mathematical model of the CKD health system. The tool projects future patient net population growth over the next 12 years using time series forecasting at the sub-LHIN level, and translates it into capacity requirements at each dialysis facility. This is accomplished through a series of allocation algorithms that determine the care modality and location of all patients receiving dialysis. These algorithms leverage spatial analysis and optimization routines to ensure that they are calibrated to the population and service characteristics of each dialysis regional program, and to the individual facilities that provide the care.
Much of the data from ORRS is used throughout the model to project future capacity requirements. This data is supplemented with data from the Ontario Health Insurance Plan claims payment system to provide robust forecasts based on 14 years of data. The model parameters can be easily refreshed as new data becomes available.
In addition to population growth, the model accounts for changes in the patient journey. Specifically, target independent dialysis (ID) rates obtained from each regional CKD program are used to estimate the number of patients receiving dialysis at home, thereby lessening the burden on in-facility capacity requirements. By accounting for the transition and retention characteristics of ID patients in each program, the model is able to translate annual ID targets to program-specific quarterly referral goals. The model also accounts for the return of home patients to facility-based dialysis temporarily or permanently, often as a result of medical complications requiring close attention. As such, it can provide guidance on how much capacity each regional program must reserve within their hospitals to accommodate home patients who unexpectedly return.
The model creates a multi-level, system-wide view of the CKD healthcare system, informing a variety of decisions and allowing them to be made in a consistent, evidence-based manner. The primary output is the predicted surplus or shortage of dialysis stations at each treatment facility, which is used to support capacity expansion proposals, and as evidence for need of new treatment facilities. However, other model outputs are also relevant. For example, the model also projects the required patient-based funding to support the annual request for MOHLTC funding.
The model's interface has allowed it to be used as a flexible tool in planning processes, rather than as a static source of information. It allows users to manipulate input parameters, which has facilitated ongoing conversation between ORN and local planning stakeholders. In 2013, ORN provided baseline results from the model to regional planning stakeholders, who responded with input of certain model parameters and assumptions, including projected growth rates and patient travel patterns. ORN planners are now able to take the feedback from the regional planning stakeholders and make the necessary changes via the interface to re-run the model and produce an updated set of results. ORN can also use the model to test hypothetical planning scenarios, such as the effect of opening new facilities on capacity requirements at nearby facilities.
The ORN model is an excellent example of the capacity that CCO is committed to developing across the organization and within the healthcare system. Through the development of a robust data infrastructure and enterprise data strategy, acquisition of new data sources and exploration of automated operational reporting, CCO will direct greater focus to developing this expanded planning capacity.
Establishing the SA team at CCO is a journey that has merely begun. In this journey, we are guided by three principles: relevance, agility and positive user experience. We strive to support relevant clinical, management and policy decisions that have a real impact on healthcare performance; to provide the solutions in an agile manner to empower timely decision-making; and to enable a positive user experience that allows decision-makers to interact with and adopt solutions to support their decision-making.
The backbone of these solutions is robust analytics that go beyond historical descriptive analytics towards predictive and prescriptive analytics. While the general perception of such advanced analytics is that they require a lengthy process to develop, we are constantly learning and adopting agile approaches, such as machine learning, that expedite the process. Furthermore, we are currently working on optimizing the backbone of our analytical processes through standardization of methodologies and analytic steps, as well as our infrastructure set-up, in order to reduce project delivery times further.
CCO has come to understand the importance of supportive and entrenched policies and analytic processes, joint governance of the projects with CCO programs and mutual collaborative efforts with the Privacy and Access Office (PAO). It is by maintaining these success factors that we can move towards our vision of fully embedding evidence into healthcare decision-making.
Insert: The Future of Privacy: Privacy by Design
In leveraging CCO's rich datasets and technological and software assets to bring advanced insights to clinical programs, CCO must meet its privacy law-related mandate, including obligations pursuant to the Personal Health Information Protection Act (PHIPA 2004), as well as those set by its regulator, the Information and Privacy Commissioner of Ontario (IPC). Close collaboration with the Privacy and Access Office at CCO has enabled CCO to ensure its datasets are used in a way that minimizes the chance for any privacy breaches.
Moreover, CCO is currently moving towards a Privacy by Design (Information and Privacy Commissioner 2011) model, whereby all future privacy controls will be embedded seamlessly into every analytics approach. Privacy by Design is a concept that was developed by the IPC to address the growing and systemic effects of IT and of large networked data systems. To accomplish Privacy by Design, the following seven principles must be applied:
- It is proactive not reactive such that it anticipates privacy issues before they occur.
- It sets privacy as the default setting, ensuring that personal data are automatically built into the IT system.
- It is embedded into the design and architecture of systems without diminishing functionality.
- It aims to accommodate all interests such that it results in positive sum, not zero sum.
- It extends throughout the life cycle of the data (from start to finish), including destruction.
- It seeks to ensure that all operations remain visible and transparent.
- It requires architects, operators and users of the data to keep the interests of the individual first and foremost.
CCO's PAO at CCO is continually working with the SA – and all the agency's programs – to embed these principles throughout the entire data life cycle.
About the Author(s)
Jason Garay is the Vice-President of Analytics and Informatics at CCO. He is active in the academic community, holding a faculty appointment as Adjunct Professor at the Dalla Lana School of Public Health at the University of Toronto. He has also been a member of several healthcare committees including: President, Association of Public Health Epidemiologists in Ontario; Provincial Co-Chair, Integrated Public Health Information System (iPHIS); National Chair, Pan-Canadian Public Health Epidemiologists' Network; and Executive Board Member, Communicable Diseases Surveillance Network. Jason holds a Master of Health Science in Community Health and Epidemiology from the University of Toronto as well as a BASc in Occupational and Public Health from Ryerson University and an Honours BSc in Statistics and Psychology from York University.
Rosario Cartagena is a Manager working in CCO's Privacy and Access Office (PAO). She is a lawyer by training and most of her work consists of drafting legal agreements with respect to the sharing of data between institutions, leading negotiations between parties, advising CCO business programs with respect to risk management and providing strategic advice to CCO on issues related to privacy law.
Ali Vahit Esensoy is a Senior Methodologist and the Acting Senior Manager in the Strategic Analytics Team at CCO. He has helped establish the Strategic Analytics portfolio, and the development of the orthopaedic wait list management tool for LHINs, dialysis capacity planning model for the ORN, and the oncologist planning model for the cancer models of care initiative.
Kiren Handa is a Acting Director within CCO's Analytics Department. She manages the Strategic Analytics practice, leveraging her experience in project management, client relations and analytics.
Eli Kane is the Senior Manager, Evidence and Information in the Evaluation and Reporting team of Cancer Screening at CCO. Eli has care and leadership over a dynamic team that turns big data into meaningful information for Cancer Screening program performance evaluation and risk based decision-making.
Neal Kaw is a Student Analyst at CCO. Neal led the implementation of the capacity planning tool for chronic kidney disease. He is currently pursuing a BASc degree in Industrial Engineering at the University of Toronto.
Somayeh Sadat, PhD, is a Senior Methodologist within the Strategic Analytics team at CCO and has played a key role in establishing the practice. She specializes in advising programs on best defining key business questions that can be supported by analytics, leading analytic projects that involve developing data-driven decision-making tools and building analytic capacity organization-wide.
Anas, R., R. Bell, A. Brown, W. Evans and C. Sawka. 2012. "A Ten-year History: The Cancer Quality Council of Ontario." Healthcare Quarterly 15: 24–27.
Cancer Care Ontario. 2011. "Ontario Cancer Plan 2011–2015". Retrieved January 22, 2014. http://ocp.cancercare.on.ca.
Information and Privacy Commissioner. 2011. "Privacy By Design". Retrieved January 22, 2014. <http://www.privacybydesign.ca/index.php/about-pbd/>. Personal Health Information Protection Act. 2004. S.O. 2004, Chapter 3. Schedule A and its Regulations.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed