Healthcare Quarterly

Healthcare Quarterly 21(2) July 2018 : 6-9.doi:10.12927/hcq.2018.25630
ICES Report

Integrating Population-Wide Laboratory Testing Data with Audit and Feedback Reports for Ontario Physicians

Michael A. Campitelli, Matthew Kumar and Anna Greenberg on behalf of the ICES/HQO Ontario Laboratory Information System Working Group

Abstract

Audit and feedback reports, distributed by Health Quality Ontario to consenting primary care physicians, provide doctors with a confidential summary of how they manage patients with diabetes; these reports currently lack clinical information. We examined the feasibility of linking the Ontario Laboratories Information System (OLIS), a large provincial database of laboratory test results, with the existing provincial audit and feedback reporting structure to integrate measures of glycemic and cholesterol control among patients with diabetes. We found that we could ascertain glycated hemoglobin (69.9%) and low-density lipoprotein cholesterol (64.1%) test results in the previous year for most patients and that there was wide variation among physicians in the proportion of patients who exceeded clinical thresholds for these measures. Our study highlights the potential value of reporting more clinically rich information to physicians to improve diabetes care and management and demonstrates the feasibility of using OLIS data at the population level to enhance ongoing research and quality improvement.

The Issue

Nearly 1.6 million Ontarians were living with diabetes in 2016, costing the province an estimated $1.5 billion in direct healthcare expenditures, including inpatient hospitalizations, physician visits and medications (Canadian Diabetes Association 2016). In Ontario's framework for preventing and managing chronic diseases, a key component requires the use of information systems to track the performance of guideline-informed care and to produce feedback on performance for evaluation and continuous quality improvement (Ontario Ministry of Health and Long-Term Care 2007).

Audit and feedback (A&F) reporting involves providing recipients with a summary of their performance over a specified period of time and is widely used by healthcare stakeholders to monitor and change health professionals' behaviour (median 4.3% absolute improvement in provider compliance with desired practice), both to increase accountability and improve quality of care (Ivers et al. 2014). Currently, the Institute for Clinical Evaluative Sciences (ICES) and Health Quality Ontario (HQO) collaboratively produce A&F reports, which are distributed biannually to consenting primary care physicians across the province. Among other items, the reports contain indicators of diabetes management, such as the proportion of patients with diabetes who are up to date with testing for glycated hemoglobin (HbA1c), low-density lipoprotein cholesterol (LDL-C) and retinal eye exams, and include provincial- and regional-level benchmarks. However, the reports lack more detailed clinical information on the patients with diabetes in a physician's practice.

Recently, ICES began linking laboratory test results from the Ontario Laboratories Information System (OLIS) database with the other routinely collected health administrative data held by ICES. The OLIS database is maintained by eHealth Ontario and networks together multiple community, hospital and public health laboratories. The database contains information on laboratory test orders and results for over 68,000 unique test types from the fields of biochemistry, hematology, immunology, microbiology and pathology (eHealth Ontario 2018). In an effort to enhance the A&F reports with more clinical information, we examined the feasibility of using data captured in OLIS to create aggregated, physician-level measures of glycemic and cholesterol control among Ontario residents with diabetes.

The Study

We used an existing cohort of all Ontario residents, alive and eligible for healthcare services under the Ontario Health Insurance Plan (OHIP) as of March 31, 2014, who were assigned to a single most responsible primary care physician using methods that have been described elsewhere (approximately 12.4 million patients) (Kiran et al. 2015). For our analysis, we included cohort members who had been diagnosed with diabetes for at least two years, using the Ontario Diabetes Dataset (Hux et al. 2002).

The patients with diabetes were linked to the OLIS database using the appropriate Logical Observation Identifiers Names and Codes (LOINC) to identify HbA1c and LDL-C laboratory tests between April 1, 2013, and March 31, 2014. We quantified the proportion of patients with diabetes who had at least one HbA1c test, at least one LDL-C test and at least one of each test during this period. In addition, we assessed the proportion of tested patients with diabetes who exceeded clinically defined thresholds for high HbA1c (>9%) and high LDL-C (>4 mmol/L).

Funnel plots were used to provide a graphical representation of the variation observed in the physician-level proportion of patients tested for HbA1c who exceeded the 9% threshold (Spiegelhalter 2005). Multilevel logistic regression modelling, with a random intercept at the physician level, was used to test if the observed variation was statistically significant (Austin and Merlo 2017). These analyses were restricted to physicians treating more than 20 patients with diabetes tested for HbA1c. The above process was repeated to examine variation in the physician-level proportion of patients tested for LDL-C who exceeded a threshold of 4 mmol/L.

Key Findings

We identified 1,108,350 Ontario residents, alive and eligible for OHIP services on March 31, 2014, who had diabetes for at least the previous two years and were attached to 11,024 physicians in the province. Among these patients with diabetes, 69.9% received at least one HbA1c test, 64.1% received at least one LDL-C test and 58.5% received both tests between April 1, 2013 and March 31, 2014. Among patients with at least one HbA1c test, 12.9% exceeded the high HbA1c threshold. Similarly, 4.6% of patients with at least one LDL-C test exceeded the high LDL-C threshold.

Wide variability was observed in the physician-level proportion of tested patients who exceeded the 9% threshold for HbA1c (Figure 1). Among the included physicians, 11.4% were above the upper 95% confidence limit on the funnel plot, whereas only 2.5% would be expected by chance. This between-physician variability was statistically significant (p < 0.001). Similarly, significant variation (p < 0.001) was observed among the physician-level proportion of patients tested who exceeded a threshold of 4 mmol/L for LDL-C (Figure 2).


Click to Enlarge
 


Click to Enlarge
 

Key Messages

By linking OLIS data to the health administrative data housed at ICES, we were able to determine an HbA1c and LDL-C laboratory test value in the past year for approximately 70% and 65% of a population-based cohort of Ontario residents with diabetes, respectively. These proportions are likely to increase over time as OLIS incorporates additional laboratories into its network. This represents a large proportion of patients with diabetes in the province for whom we now have enhanced clinical data to supplement the other measures of diabetes care and management we are able to generate using traditional health administrative data sets. Previous studies have noted significant variations in both HbA1c/LDL-C testing and HbA1c/LDL-C control across geographic regions (Gamble and Butalia 2016), and such variations in clinical practice typically represent evidence-based care not being consistently applied throughout a healthcare system (Wennberg 2002). For this reason, numerous jurisdictions have published diabetes atlases that display key process and outcome indicators over specific areas to promote best practices and quality improvement initiatives for diabetes care (Gamble and Butalia 2016). However, the significant physician-level variation we observed in the proportion of tested patients who exceeded a particular clinical threshold for HbA1c or LDL-C highlights the value of reporting this information directly to physicians via A&F reports. Notifying these physicians concerning the disease severity of their patients with diabetes may lead them to alter aspects of their clinical practice to better manage these individuals or seek external resources for quality improvement purposes.

More broadly for ICES, we were able to successfully link a large, population-based cohort of patients with diabetes to OLIS information, demonstrating an organizational capacity to receive, process and clean "biomedical big data" and make it available for analysis (Stukel et al. 2017). Although prior ICES studies have incorporated laboratory test results for selected patient populations, to our knowledge, this is the first such linkage between health administrative data and laboratory test results at the general population level in Ontario. A commonly cited limitation of health administrative data is the lack of clinical details that can be ascertained within the records (Garland et al. 2015). Physiologic and laboratory measures are frequently required to measure the severity of acute and chronic diseases but are difficult to capture at the population level. The linkage of OLIS data will allow for the inclusion of individuals who have recent clinical values from laboratory tests into population-based research. For example, in addition to having test results for HbA1c that estimate the magnitude of blood glucose control in patients with diabetes, there has also been work completed at ICES to process OLIS test results for serum creatinine and urine albumin-creatinine ratio, which can help measure the extent of kidney disease. Ongoing work in this area will expand the breadth of laboratory tests available for analysis at ICES and represents a promising area of enhancement to our future research.

About the Author

Michael A. Campitelli, MPH, is a staff scientist in the Strategic Partnerships and External Services department at ICES. He can be contacted at michael.campitelli@ices.on.ca.

Matthew Kumar, MSc, is a senior research analyst in the Data and Analytic Services department at ICES.

Anna Greenberg, MPP, is the vice-president of Health System Performance at HQO.

The ICES/HQO Ontario Laboratory Information System Working Group was composed of senior management representatives (Mahmoud Azimaee, P. Alison Paprica, Michael J. Schull and J. Charles Victor) and senior scientists (J. Michael Paterson, Baiju R. Shah, Jack V. Tu and Richard H. Glazier) at ICES who supported this work and provided technical, clinical and methodological expertise.

Acknowledgment

This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported are those of the authors and are independent from the funding sources. No endorsement by ICES or the MOHLTC is intended or should be inferred. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information (CIHI). However, the analyses, conclusions, opinions and statements expressed herein are those of the author and not necessarily those of CIHI. No funding bodies had any role in the study design, data collection, analysis, decision to publish or preparation of the manuscript. The data sets used in this study were linked using encoded identifiers and analyzed at ICES.

References

Austin, P.C. and J. Merlo. 2017. "Intermediate and Advanced Topics in Multilevel Logistic Regression Analysis." Statistics in Medicine 36(20): 3257–77. doi:10.1002/sim.7336.

Canadian Diabetes Association. 2016. 2016 Report on Diabetes in Ontario. Toronto, ON: Canadian Diabetes Association. <https://www.diabetes.ca/getmedia/a45fe16a-3967-416c-bef5-d810f00ddb65/Diabetes-in-Ontario_FINAL.pdf.aspx>.

eHealth Ontario. 2018. "Ontario Laboratories Information System." Retrieved May 29, 2018. <https://www.ehealthontario.on.ca/for-healthcare-professionals/ontario-laboratories-information-system-olis>.

Gamble, J.-M. and S. Butalia. 2016. "Medical Practice Variations in Diabetes Mellitus." In A. Johnson and T. Stukel, eds. Medical Practice Variations. New York, NY: Springer.

Garland, A., H.B. Gershengorn, R.A. Marrie, N. Reider and M.E. Wilcox. 2015. "A Practical, Global Perspective on Using Administrative Data to Conduct Intensive Care Unit Research." Annals of the American Thoracic Society 12(9): 1373–86. doi:10.1513/AnnalsATS.201503-136FR.

Hux, J.E., F. Ivis, V. Flintoft and A. Bica. 2002. "Diabetes in Ontario: Determination of Prevalence and Incidence Using a Validated Administrative Data Algorithm." Diabetes Care 25(3): 512–16.

Ivers, N.M., A. Sales, H. Colquhoun, S. Michie, R. Foy, J.J. Francis et al. 2014. "No More ‘Business as Usual’ with Audit and Feedback Interventions: Towards an Agenda for a Reinvigorated Intervention." Implementation Science 9: 14. doi:10.1186/1748-5908-9-14.

Kiran, T., A. Kopp, R. Moineddin and R.H. Glazier. 2015. "Longitudinal Evaluation of Physician Payment Reform and Team-Based Care for Chronic Disease Management and Prevention." CMAJ 187(17): E494–502. doi:10.1503/cmaj.150579.

Ontario Ministry of Health and Long-Term Care. 2007. Preventing and Managing Chronic Disease: Ontario's Framework. Retrieved May 29, 2018. <http://www.health.gov.on.ca/en/pro/programs/cdpm/pdf/framework_full.pdf>.

Spiegelhalter, D.J. 2005. "Funnel Plots for Comparing Institutional Performance." Statistics in Medicine 24(8): 1185–202.

Stukel, T.A., P.C. Austin, M. Azimaee, S.E. Bronskill, A. Guttmann, J.M. Paterson et al. 2017. Envisioning a Data Science Strategy for ICES. Toronto, ON: Institute for Clinical Evaluative Sciences. Retrieved May 29, 2018. <https://www.ices.on.ca/Publications/Atlases-and-Reports/2017/Data-science-strategy>.

Wennberg, J.E. 2002. "Unwarranted Variations in Healthcare Delivery: Implications for Academic Medical Centres." BMJ 325(7370): 961–64.

About the Authors

Michael A. Campitelli, MPH, is a staff scientist in the Strategic Partnerships and External Services department at ICES. He can be contacted at michael.campitelli@ices.on.ca.

Matthew Kumar, MSc, is a senior research analyst in the Data and Analytic Services department at ICES.

Anna Greenberg, MPP, is the vice-president of Health System Performance at HQO.

The ICES/HQO Ontario Laboratory Information System Working Group was composed of senior management representatives (Mahmoud Azimaee, P. Alison Paprica, Michael J. Schull and J. Charles Victor) and senior scientists (J. Michael Paterson, Baiju R. Shah, Jack V. Tu and Richard H. Glazier) at ICES who supported this work and provided technical, clinical and methodological expertise.

Comments

Be the first to comment on this!

Related Articles

Note: Please enter a display name. Your email address will not be publically displayed