Healthcare Quarterly

Healthcare Quarterly 12(Sp) May 2009 : 76-77.doi:10.12927/hcq.2009.20756

Turning Data into Meaningful Information

Julian Martalog and Shalu Bains


The Ontario Ministry of Health and Long-Term Care (MOHLTC) launched the Wait Time Strategy in 2004 to improve access to healthcare by reducing the wait times for procedures and treatments. A fundamental component of the strategy was the development of the Wait Time Information System (WTIS). On behalf of the MOHLTC, Cancer Care Ontario (CCO) delivered the first electronic application used by hospitals province-wide to collect essential wait time data. Until then, clinicians had been maintaining wait lists within their own offices (usually on paper), but had no effective way to manage waits that were getting too long. Patients also wanted faster treatment, but had no concrete information to hold the health system accountable for inappropriate waits or to help in managing their own care. Lastly, hospitals and health system planners knew that a more comprehensive view of wait times could help them make objective decisions around how to allocate resources. The WTIS was introduced to solve this information challenge.

Having better information, however, is only one side of the equation. Arguably, it's how you use the data that will provide the benefit. The Wait Time Strategy (the strategy) used a "pay for performance" approach requiring hospital leaders to be accountable for using the data captured through the WTIS to achieve defined wait time targets in return for funding for more procedures and programs. Hospital accountability for improving performance was further driven through the reporting of wait times on a public website (

Here we examine the steps that CCO took to support the collection of necessary data and turn it into meaningful information to drive improvements. This experience is now being used to shape performance management activities for the broader access to care agenda across the province.

Setting the Standards for Success

For the Wait Time Strategy, success hinged on being able to secure accurate, reliable and timely wait time data. While a long-term solution was being developed, an interim manual tool was put in place to begin the process of data collection and reporting for the initial five service areas of the Wait Time Strategy. This preliminary information was published online every two months, giving Ontarians and healthcare providers their first opportunity to view wait times for key procedures, by hospital.

The interim process also served to define what was needed to effectively measure performance in the long run. As an example, to ensure timeliness and accuracy of data, it was determined that patients scheduled for treatment should be entered into the system within two business days of the decision to treat, and that these cases should be closed within two business days of the completion of the procedure. Also, any external circumstances that delayed treatment (such as vacations) should be tracked and deducted in order to calculate the "true" waiting period. These measures ultimately formed the criteria for wait time reporting as stipulated in accountability agreements between the MOHLTC and hospitals as a condition for funding under the strategy.

In addition, with a better understanding of current performance levels gained through the preliminary data, Clinical Expert Panels advising the Wait Time Strategy now had evidence on which to establish performance targets. The province now has targets based on reasonable maximum wait times according to the urgency of a patient's condition, using a priority scale of 1 (most urgent) through 4 (least urgent). Priorities - and the target wait times associated with each level - provide standards for treatment across Ontario and serve as a method of accountability for physicians, hospitals and the government.

Getting to Wait List Management

With standards defined, the WTIS was built and implemented to accelerate and automate the collection of wait time data from hospitals and clinician offices across Ontario using a common and consistent approach. Most importantly, the electronic solution allowed data to be captured and reported in near real-time, tracking delays in treatment and flagging cases that approached wait time targets - all of which was critical to actively manage wait lists.

Arriving at a point where clinicians, hospitals and health system planners had accurate and comprehensive wait list information was no easy task. From a data management perspective, the process involved four sequential stages:

  1. Collect existing data. To get a full picture of the backlog, clinicians were required to enter not only new wait list cases into the WTIS, but also all pre-existing cases that were being maintained manually. Clinicians also needed to apply the new provincial priority ratings to all new cases.

  2. Clean up the wait list. Next, the list needed to be purged of cases that did not aptly constitute a "wait" - for example, patients entered as "placeholders" for OR time or who appeared on multiple physician wait lists (in an attempt to get treatment faster). These "non-waits" may have accounted for anywhere between 10 and 30% of entries.

  3. Assess long-waiters and priorities. The sanitized list could now be reviewed for patients whose waits were excessive (2 years or more) or whose level of urgency for treatment was greater according to the priority guidelines.

  4. Manage the wait list. With a complete and standardized inventory of patients waiting for a procedure, along with each patient's acuity and length of wait, clinicians now had the ability to make more informed decisions on how best to manage their patients.

Staying on Track through Regular Reporting

Today, wait times for the province are reported publicly each month through the Ontario Wait Times website, enabling healthcare providers and patients alike to compare results by urgency of care, hospital and region. This has created a new level of transparency around access issues within Ontario, which in turn has led to a greater sense of public accountability for making improvements.

The ability to meet data reporting requirements is closely monitored by the Wait Time Information Program, which plays an important role in determining what may be hindering hospitals in complying with data quality standards or performance targets. With feedback from the field, rigorous data quality improvement methods have been implemented to continually refine data-capture practices within hospitals and by clinicians, ensuring the highest quality data. Hospitals that fail to submit their wait time data according to the guidelines are notified that they will be reported as "non-compliant" on the website and that incremental funding may be recovered.

CCO has also developed a number of tools for hospital leaders to monitor performance levels against key criteria and better identify problem areas. For example, CCO prepares a quarterly "scorecard" for the provincial government and is planning to prepare one for Local Health Integration Network (LHIN), offering a snapshot of key performance indicators along with supporting analysis and interpretation of results (Figure 1). The standardized format allows the MOHLTC and LHINs to consistently track their performance in these areas. Scorecard data highlights areas where subsequent analysis is required.


[Figure 1]

Through CCO's web-based iPort Access™ reporting tool, leaders have the ability to generate more detailed, drilled-down reports by specific criteria. This tool not only puts information at the fingertips of hospital leaders, but also gives them the ability to further investigate potential inequities in access (gender or age variances, for example) or analyze utilization patterns.

With the level of reporting provided through the WTIS, hospital boards now have the necessary information to govern their organization's access management strategy and compare their hospital's performance with others in the province. With timely data, hospitals can better manage access, waits and patient flow within their organizations and improve efficiencies in the delivery of care in line with funding targets. Clinicians have the ability to provide the necessary patient information to hospitals so that surgeries can be booked, wait times tracked and potential problem areas identified. In addition, patients now have information to manage their own care and, through provincial targets, a sense of how quickly they can and should receive treatment.

The importance of accountability-driven public reporting is underscored in the context of sustainability of the Wait Time Information Program. The website receives an average of about 8,700 hits per day - a clear indication from the public that access issues are important to them. With or without incremental funding applied through the pay for performance model, now that patients and the public have a way to track progress and hold their government to account, hospitals and government will continue to feel pressure to keep wait times down.

Analysis shows that the accountability driven through public reporting is paying off with a marked improvement for wait times. Figure 2 reports the comparison of February 2009 wait time information against baseline for the areas of care originally targeted in the Wait Time Strategy. (CT and MRI scans, and knee and hip replacements, are broken out separately.)


[Figure 2]

Moving from Information to Action

Once problem areas are identified, CCO provides additional support and resources to the MOHLTC and LHINs in using wait time data to make strategic and operational decisions to drive performance improvements and inform future intervention strategies. With baseline information and standard performance measures, changes can be planned and started with expectations on the extent to which their success will move the measures in the desired direc-tion. This is a shift from what histori-cally has been intuition-based decision-making in healthcare to an evidence-based process (Devit et al., 2005)

CCO conducts in-depth analyses on specific areas to help the MOHLTC and LHINs gain a better understanding of regional differences and root causes of poor performance. This information is used to facilitate discussions between the MOHLTC, LHINs and hospitals around where to target intervention or allocate additional resources. When required, wait time experts and advisors are sent out to hospitals to provide one-on-one advice, share best practices and discuss solutions.

This was the case in late 2007 when CCO conducted a special analysis on cataract surgery wait times. Although substantial improvements had been made in this area, variations in wait times still existed across LHINs. An analysis by region revealed two LHINs with wait times significantly above the provincial target. Together, they accounted for 44% of the total "over-the-target cases" in the province. In an attempt to find the root cause, a further breakdown of data from these LHINs identified two specific facilities where patients were waiting significantly longer than in the rest of the province.

For the first hospital, an impact analysis against a number of variables concluded that the high ratio of cataract surgeries performed at this facility was driving down the overall performance of the LHIN, but that it was a systemic problem within the region rather than something this facility was or was not doing. In this case the Access to Services and Wait Times Lead for the province, Dr. Alan Hudson, and the chair of the Ophthalmology Clinical Expert Panel, Dr. Phil Hooper, were asked to work with the LHIN to better understand the regional challenges and determine appropriate solutions so that any potential negative impacts on either the LHIN's public reputation or funding allocation could be avoided. Today, the LHIN is using the various resources available to play a more proactive role in reviewing wait lists and managing accordingly.

A comprehensive analysis of the second facility concluded that the LHIN's performance was being impacted by one surgeon, who had a significantly high proportion of lengthy waits. On the basis of these findings, the WTIP recommended that the LHIN allocate additional OR time to the surgeon and when appropriate provide patients with the option to choose treatment with another clinician or facility. Based on the analysis of wait time information and recommendations made by the WTIP, the surgeon was able to treat more of his urgent patients and move them through the system, reducing his wait list by half and the number of patients with prolonged waits (more than a year) by 45%.

Annually, the MOHLTC also assesses hospitals' success in meeting volume targets and the conditions of funding. With reliable wait list information extracted through the WTIS, the province can make more informed, unbiased decisions about allocation of future funding to reward performance improvements and enforce consequences of under-performance. For example, in 2006-2007, the MOHLTC's decisions for the allocation of $109 million incremental in-year funding were driven, in part, through an in-depth trending analysis and forecasting completed by CCO.

Supporting Future Planning and Continuous Improvement

The translation of strategic direction into measurable results has been an ongoing challenge within the Canadian healthcare system. A lack of consistent measures, ever-changing priori-ties and a reactionary focus on short-term priorities make the conversion of strategy into measured outcomes particularly difficult (Devit et al., 2005). CCO is setting out to change that. As the strategy matures, the organization's focus on performance management continues to grow, particularly in ensuring all surgical and diagnostic imaging services achieve their access targets. Today, CCO is looking at leveraging its information management capability and data assets to further support business intelligence - an evolving area of excellence in which comprehensive information is used to forecast trends and predict future needs and costs. Moving into this level of data-driven decision-making is increasingly important for performance management within healthcare - where planning for quality care can be effectively tackled at the provincial, regional and facility levels and the resulting efficiencies appropriately rewarded. With the results seen to date, there is tremendous opportunity for using wait time information to continue developing new performance indicators and targets as a way to incent continuous improvement and raise the bar for access to healthcare.

About the Author(s)

Julian Martalog is the Director of the Access to Care Informatics department at Cancer Care Ontario. Julian has experience in information management with local, provincial and national organizations. He holds a Masters in Health Administration from the University of Ottawa and is a certified Health Executive with the Canadian College of Health Service Executives.

Shalu Bains is the Manager of Reporting and Analysis in Cancer Care Ontario's Access to Care Informatics department. She has experience in health information management and decision support through her previous positions in both hospital and national organizations. She holds a Masters in Health Administration degree from the University of Ottawa and is a certified Health Executive with the Canadian College of Health Service Executives.


Devit R., W. Klassen and J. Martalog. 2005. "Strategic Management System in a Healthcare Setting - Moving from Strategy to Results." Healthcare Quarterly 8(4): 58-65.


Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed