Healthcare Quarterly

Healthcare Quarterly 15(4) October 2012 : 19-21.doi:10.12927/hcq.2012.23194
ICES Report

The Institute for Clinical Evaluative Sciences: 20 Years and Counting

Doug Dolan, James Grainger, Nancy MacCallum, Deborah Creatura, Jacqueline Forrester and Susan Shiller

The Beginning

In the early 1990s, as the quality and efficiency of healthcare in Ontario were coming under increased scrutiny, provincial officials saw the value of a new type of research institute that would provide evidence-based insights on system performance. But before the government could give its full support to the Institute for Clinical Evaluative Sciences (ICES), there were two hurdles to overcome: First, ICES scientists needed to explain precisely what kind of data they were seeking, and why. Second, the new organization had to formulate a set of privacy protective ground rules – at a time when jurisdictions around the globe were taking the first steps toward creating regulatory frameworks to safeguard personal data.

The proposal from the original ICES team – led by Dr. David Naylor and his colleague (and successor) Dr. Jack Williams – was that the Ontario Ministry of Health could gain these kinds of population-based findings from the administrative data routinely collected in providing health services to Ontarians. Strengthening their case was the fact that the Ontario Health Insurance Plan (OHIP) had recently begun issuing individual cards to every eligible resident, replacing the earlier family cards. This would vastly enhance researchers' ability to study specific types of patients – pinpointing, for instance, users of cholesterol medications or children in a particular age range for a pediatric study.

After considering the implications of redirecting personal health information for a secondary purpose, government officials agreed that the benefits to Ontarians far outweighed the potential risks – a tremendous achievement for Dr. Naylor, and a great show of faith by the ministry, which was understandably concerned about disclosing data outside its walls.

In April 1992, ICES opened its doors in A-Wing on the campus of Sunnybrook Hospital in North Toronto. Dr. Naylor was the institute's first CEO. Dr. John Evans chaired a four-member board of directors. The following year, ICES moved into the newly renovated G-Wing—where it remains today.

The Data Privilege

Today, ICES is privileged to have one of the world's largest holdings of individual-level, de-identified and linkable health and health-related data. Encompassing the health services records of nearly 13 million people (as of 2012), this vast data resource provides the foundational evidence for vital research in support of more effective health system planning, evaluation and policy. ICES is designated as a "prescribed entity" in Ontario under the Personal Health Information Protection Act (PHIPA). This allows ICES to collect and use administrative data for the purposes of monitoring and evaluating the provincial health system. All ICES polices, practices and procedures for using data are reviewed and approved on a regular basis by the Office of the Information and Privacy Commissioner of Ontario.

Over the past two decades, ICES has steadily expanded its data resource by collecting additional data sets from organizations such as the Cardiac Care Network of Ontario and Cancer Care Ontario, as well as from hospitals, physicians, laboratories, pharmacies and long-term care facilities. Data-sharing agreements are developed on an individual basis with other provider organizations, whose new data are de-identified before being integrated and made linkable within the ICES resource. In some cases, new data are collected from scratch or from sources outside the health sector, which requires extra vigilance to ensure that the data integration is compliant with Ontario privacy legislation. Expert teams of data quality and data management specialists at ICES work with partner agency staff to securely transfer data into the ICES environment.

A New Way to Share Information

Getting evidence-based knowledge into the hands of decision-makers has been central to the mandate of ICES since its inception. By providing unbiased scientific findings to key stakeholders in the Ontario health system – including policy makers, administrators, planners and practitioners – the earliest ICES researchers believed they could have a positive influence on clinical practice and public policy affecting all aspects of healthcare delivery. In its first full year of operation (1992–1993), ICES fulfilled that objective by publishing approximately 30 research reports, including many that illuminated important issues and stimulated debate across the health sector and in the media. But ICES researchers were considering alternative ways to broaden the reach and impact of their work. A diverse team began to explore how they might reform their findings into more accessible documents providing quickly digestible overviews of province-wide trends. The result was a new genre of evidence-based reporting: the research atlas.

The first ICES publication to adopt the new model was Patterns of Health Care in Ontario, released in 1994 (Goel et al.). This pioneering research atlas covered a wide range of system-related and disease-specific topics, including cancer surgery and hysterectomy rates, variations in physician-billing patterns and access to services based on patients' location and socio-economic status. The atlas highlighted regional patterns in healthcare delivery and provided recommendations to help guide quality improvement, resource allocation and future decision-making.

The second edition of Patterns of Health Care in Ontario, published in 1996 (Goel et al.), updated regional analyses and hospital performance metrics while providing new research on community health indicators, mental health services, pediatrics and provincial prescribing trends. Both editions were widely cited during this period, as the government of Premier Mike Harris undertook a hospital restructuring program aimed at meeting the demands of a healthcare system undergoing profound change.

In 1997, ICES released its first topic-specific atlas, Cancer Surgery in Ontario (Iscoe et al.) – an in-depth look at surgical practices and one of the first steps in a broader provincial initiative to reorganize and improve cancer care. The atlas provided crucial data and guidance for the regional integration and rationalization of cancer services.

Subsequent atlases reported on the use of health services across the continuum of care, investigating emergency department challenges, disease-specific health service use (diabetes, cardiovascular disease, arthritis, etc.), access to care and more. In recent years, ICES researchers have continued to publish a series of thematic and topic-specific atlases and investigative reports. All are aimed at providing pragmatic, accessible research for a healthcare system challenged by resource allocation issues and a growing, rapidly aging population base.

Research with Impact

Most ICES research finds its way into peer-reviewed journals. A number of these studies have contributed to the development of public policies aimed at protecting and promoting the health of Ontarians, while reducing inequities in service delivery. A few examples follow:

  • A pivotal study on the links between cell-phone use and motor vehicle collisions helped lay the groundwork for legislation banning driver cell-phone use in Ontario and many other jurisdictions (Redelmeier and Tibshirani 1997).
  • A study of mortality rates among patients requiring cardiac surgery showed that patients waiting for valve operations have a higher risk of death than those waiting for isolated coronary artery bypass graft (CABG) surgery (Morgan et al. 1998). This finding led to the creation of provincial guidelines to promote safer queuing policies for non-CABG cardiac procedures.
  • A study of positron emission tomography (PET) scanning raised doubts about the technology's value and led to the drafting of a published report for the provincial health ministry's Committee on Technical Fees (Laupacis et al. 2004). This report influenced the government's subsequent decision to collect appropriate data on clinical effectiveness before deciding whether a new technology should be implemented.
  • A 2010 ICES study by Kelly et al. (2010) confirmed that paroxetine (brand name Paxil) – a commonly prescribed antidepressant – could nullify the effects of the cancer drug tamoxifen if taken by women with breast cancer. An editorial published in the same issue of the BMJ recommended that physicians avoid co-prescribing the two drugs to women with breast cancer and that regulators insist on stronger warnings on drug labels (Andersohn and Willich 2010).
  • A series of papers by ICES researchers measured the impact of heart failure on Ontario's healthcare system and explored factors that contribute to poor patient outcomes following discharge from the emergency department. These studies influenced the development of practice protocols to help reduce unnecessary hospital admissions for heart conditions (Alter et al. 2012; Lee et al. 2010a, 2010c).
  • ICES researchers have conducted extensive analyses on the use of implantable cardiac defibrillators (ICD), drawing on the Ontario ICD Registry, which is maintained at ICES (Krahn et al. 2011; Lee et al. 2010b; McFadden et al. 2012). These studies have shown higher rates of post-implementation complications than were reported in clinical trials.

Sharing a Valuable Resource

In the early 2000s, an initiative to expand data access geographically, spearheaded by then President and CEO Andreas Laupacis, proposed the establishment of secure ICES satellite sites at strategically selected locations across Ontario. In this new model, researchers at remote sites would be able to launch unique studies and collaborate with colleagues at the main ICES facility in Toronto and with other satellites. The expanded structure would build greater capacity for policy-relevant health services research throughout the province. It would also support the development and mentoring of more research fellows and students in the field.

In the spring of 2006, as part of its contract-renewal discussions with the provincial government, ICES proposed to establish satellite sites at each of Ontario's six medical schools. The Ministry of Health and Long-Term Care endorsed the proposal in principle, and planning got under way to establish the inaugural ICES satellite at Queen's University. When the new satellite (ICES@Queen's) opened in 2007, researchers at one of Canada's leading universities welcomed the immediate access to a rich reserve of data that they previously could use only by visiting Toronto. Two more satellites have since been added to the ICES network: one at the University of Ottawa (ICES@uOttawa) in 2010 and another at the University of Toronto (ICES@UofT) two years later. A fourth site at Western University (ICES@Western) opened in December 2012, and discussions are well under way to add a fifth at McMaster University in 2013. Dr. David Henry, who became president and CEO of ICES just as the first satellite got up and running, emphasizes the broader impact of moving to a multi-site model: "The implementation of satellite sites, by increasing the number of scientists and the breadth of scientific expertise, greatly extends the breadth of research opportunities and areas of specialty across the ICES network."

20 Years of Progress

On October 1, 2012, ICES marked its 20th anniversary with a public symposium, an opportunity to celebrate past accomplishments with stakeholders and to share an ambitious research agenda that spans a wide spectrum of health service issues – all aimed at furthering the institute's strategic vision: to expand access to data, engage new partners and enrich organizational capacity through enhanced funding and operational capacity.

If we had to distill two decades of remarkable effort down to a single word, it would be impact. (As of 2012, more than two dozen journal publications cite ICES findings each week.) In countless ways, ICES research has changed how, where and why health services are delivered – across Ontario and well beyond its borders. Everyone who has been part of the ICES story can feel proud of that legacy.

The goals of the visionary founders of ICES remain as relevant as ever: to create knowledge that will make Ontario's health system more effective, and to promote better health for all Ontarians.

About the Author(s)

Doug Dolan is a communications consultant and writer in Toronto, Ontario.

James Grainger is a writer and editor in Toronto, Ontario.

Nancy MacCallum, MLIS, is a communications and policy officer at the Institute for Clinical Evaluative Sciences (ICES), in Toronto, Ontario.

Deborah Creatura, MA, is a communications officer at ICES. She can be contacted at

Jacqueline Forrester, BA (Hons), MPH (c), is a research assistant at ICES.

Susan Shiller, MSc, is director of communications at ICES.


Alter, D.A., D.T. Ko, J.V. Tu, T.A. Stukel, D.S. Lee, A. Laupacis et al. 2012. "The Average Lifespan of Patients Discharged from Hospital with Heart Failure." Journal of General Internal Medicine 27(9): 1171–79.

Andersohn, F. and S.N. Willich. 2010. "Interaction of Serotonin Reuptake Inhibitors with Tamoxifen." BMJ 340: c783.

Goel, V., C.D. Naylor and G.M. Anderson, eds. 1994. Patterns of Health Care in Ontario (1st ed.). Ottawa, ON: Canadian Medical Association.

Goel, V., J.I. Williams, G.M. Anderson, P. Blackstien-Hirsch, C. Fooks and C.D. Naylor, eds. 1996. Patterns of Health Care in Ontario (2nd ed.). Ottawa, ON: Canadian Medical Association.

Iscoe, N.A., T. To, E.H. Gort and M. Tran. 1997. Cancer Surgery in Ontario. Toronto, ON: Institute for Clinical Evaluative Sciences.

Kelly, C.M., D.N. Juurlink, T. Gomes, M. Duong-Hua, K.I. Pritchard, P. C. Austin et al. 2010. "Selective Serotonin Reuptake Inhibitors and Breast Cancer Mortality in Women Receiving Tamoxifen: A Population Based Cohort Study." BMJ 340: c693.

Krahn, A.D., D.S. Lee, D. Birnie, J.S. Healey, E. Crystal, P. Dorian et al. 2011. "Predictors of Short-Term Complications after ICD Replacement: Results from the Ontario ICD Database." Circulation: Arrhythmia and Electrophysiology 4(2): 136–42.

Laupacis, A., L.F. Paszat, D. Hodgson and V. Benk. 2004. Health Technology Assessment of Positron Emission Tomography (PET) in Oncology: A Systematic Review. Quarterly Update. Toronto, ON: Institute for Clinical Evaluative Sciences.

Lee, D.S., M.J. Schull, D.A. Alter, P.C. Austin, A. Laupacis, A. Chong et al. 2010a. "Early Deaths in Patients with Heart Failure Discharged from the Emergency Department: A Population-Based Analysis." Circulation: Heart Failure 3(2): 228–35.

Lee, D.S., A.D. Krahn, J.S. Healey, D. Birnie, E. Crystal, P. Dorian et al. 2010b. "Evaluation of Early Complications Related to De Novo Cardioverter Defibrillator Implantation: Insights from the Ontario ICD Database." Journal of the American College of Cardiology 55(8): 774–82.

Lee, D.S., T.A. Stukel, P.C. Austin, D.A. Alter, M.J. Schull, J.J. You et al. 2010c. "Improved Outcomes with Early Collaborative Care of Ambulatory Heart Failure Patients Discharged from the Emergency Department." Circulation 122(18): 1806–14.

MacFadden, D.R., E. Crystal, A.D. Krahn, I. Mangat, J.S. Healey, P. Dorian et al. 2012. "Sex Differences in Implantable Cardioverter-Defibrillator Outcomes: Findings from a Prospective Defibrillator Database." Annals of Internal Medicine 156(3): 195–203.

Morgan, C.D., K. Sykora and C.D. Naylor. 1998. "Analysis of Deaths While Waiting for Cardiac Surgery among 29,293 Consecutive Patients in Ontario, Canada." Heart 79(4): 345–49.

Redelmeier, D.A. and R.J. Tibshirani. 1997. "Association between Cellular-Telephone Calls and Motor Vehicle Collisions." New England Journal of Medicine 336(7): 453–58.


Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed