Baker, G.R., A. MacIntosh-Murray, C. Porcellato, L. Dionne, K. Stelmacovich and K. Born. 2008. "Intermountain Healthcare." High Performing Healthcare Systems: Delivering Quality by Design. 151-178. Toronto: Longwoods Publishing.
• View the individual chapters for free in the Table of Contents.
• Order the printed paperback version or the PDF version here.
Highlights of recent achievements
In the 30th-anniversary Intermountain Healthcare (IHC) annual report, Chairman Merrill Gappmayer and Chief Executive Officer (CEO) Bill Nelson reflected on their organization's evolution and the constancy of its mission over the past three decades:
We began as a multi-hospital system and today we are an integrated system combining doctors, hospitals and health plans. ... Early in IHC's history, our trustees committed our organization to becoming better, not bigger. That is still our philosophy today. IHC's hospitals and clinics have grown to provide access to a growing population but still serve about the same proportion of inpatients they did in 1975 ... due to the influx of competitors, IHC's share of outpatient surgeries has actually decreased. ... We continually seek to improve our services and to become "ever better." (Intermountain Healthcare 2005a: 4-6)
IHC strives to be "an organization driven by excellence" (Intermountain Healthcare nd a). For years IHC and its organizations have been recognized for excellence in several areas, spanning integration, information systems, clinical care and financial performance (Intermountain Healthcare 2006a):
- Since 1999 IHC has been ranked the top integrated health system in the United States (US) by Modern Healthcare Magazine and Verispan research firm. Based on a survey and Verispan's Integrated Health Network (IHN) rating system, top regional, non-specialty healthcare systems are chosen for their ability to operate as a unified organization in each of eight categories: integration, integrated technology, contractual capabilities, out-patient utilization, financial stability, services and access, hospital utilization and physicians (Health First 2005).
- IHC's Latter-Day Saints (LDS) Hospital was selected four times as one of America's best hospitals by US News & World Report in four different medical specialties: respiratory disorders, pulmonary medicine, endocrinology and diabetes care. US News & World Report rates hospitals in 16 specialties and overall by surveying physicians on reputation and by collecting data on mortality and a mix of care-related factors such as nursing and patient services. The 50 hospitals in each specialty with the highest scores are listed (Intermountain Healthcare 2006b).
- The American Nursing Association designated LDS Hospital as Utah's first Magnet Hospital. Magnet status is a designation of nursing excellence based on an appraisal of quality indicators and standards of nursing practice as defined in the American Nurses Association's Scope and Standards for Nurse Administrators (American Nurses Credentialing Center 2007).
- For the seventh time in eight years, in 2006 Hospitals & Health Networks and the American Hospital Association named IHC one of the most technologically advanced and savvy hospital systems in the US in the 100 Most Wired Hospitals list. This recognition is based on a survey representing 1,255 hospitals nationwide that asks hospitals to report on their use of information technology to address five key areas: safety and quality, customer service, business processes, workforce and public health and safety (Hospitals and Health Networks 2006).
- In 2004 IHC's LDS Hospital and McKay Dee Hospital Center received a National Consumer Choice Award for Medical Excellence. This award is based on a nationwide consumer survey by the National Research Corporation representing more than 400,000 individuals in Utah and other areas throughout the US.
- IHC continues to have the highest bond rating for a US non-profit healthcare system. As a result of its adequate financial reserves and prudent management, IHC is able to borrow at lower rates and to pass savings on to consumers in the form of lower rates for services (Intermountain Healthcare nd b).
IHC's reputation for clinical excellence is based on a strong foundation of evidence-based medicine and clinical process management that has resulted in dramatic improvements in patient outcomes and costs. There are several examples of such improvement:
- In 1999 IHC standardized care processes for the prescription of appropriate medications for cardiac patients at discharge, including beta-blockers, statins, ACE/ARB inhibitors, antiplatelet medications and warfarin. Over a two-year period the proportion of cardiac patients receiving appropriate medications at discharge increased by 50% to proportions of more than 90%, far exceeding the US average. These process improvements at IHC have been associated with significant improvements in clinical outcomes for this group of patients, including significant reductions in mortality and readmission rates of congestive heart failure and ischemic heart disease patients (James 2005b; Lappé et al. 2004).
- In 2001 IHC standardized processes for initiating appropriate elective inductions for women waiting to deliver ( < 39 weeks' gestation). This resulted in a decrease in the rate of induction from 29% to 5% over two years, thereby exceeding the goal of reducing the rate of inappropriate inductions by 50%. This reduction led to substantial decreases in the rate of unplanned Caesarean sections and the average number of hours in labour and delivery (in electively induced patients). More appropriate elective inductions have resulted in a decrease in costs to the system of $400 per birth and $10 million per year (James 2005b).
- In 2002 IHC clinicians standardized bilirubin screening in babies. Screening for bilirubin increased by more than 80%, from 12% in 2002 to 99% in 2005. This has resulted in a significant decrease in the number of babies born at IHC that develop hyperbilirubinemia and associated complications (James 2005b).
- In 2003 clinicians adopted measures for maintaining tighter glucose control of patients in intensive care units (ICUs). These changes led to a statistically significant reduction in the rates of mortality in this patient group. This improvement is now being spread - with similar success - to patients undergoing cardiovascular surgery at IHC (Intermountain Healthcare 2005b).
- In 2004 clinicians at IHC standardized care for patients on ventilators, thereby reducing the average number of days for each patient on a ventilator by more than a day. These improvements were associated with a 10% reduction in the rate of ventilator-associated pneumonia over two years. More appropriate ventilator use at IHC is also associated with a shorter length of stay in ICU and a reduction in cost by more than $3,000 per ICU patient.
These examples of clinical improvements are underpinned by a rigorous review of evidence as well as by standardization and redesign carried out through data collection and analysis of protocols and complex care processes. A key champion at the centre of this evidence-directed, clinical quality improvement at IHC is Dr. Brent C. James, IHC's vice-president of medical research and continuing medical education and the executive director of the IHC Institute for Health Care Delivery Research. James is widely recognized in North America and internationally for his work on clinical process management and for encouraging a transformative shift from what he terms autonomous "craft-based medicine" to evidence-directed, team-based medicine focused on patients and clinical processes.
While evidence-based protocols, data collection and analysis of clinical processes have been used elsewhere as strategies for improvement, few healthcare systems have successfully translated these efforts into sustained system-wide improvements in healthcare. How have James and his IHC colleagues used such strategies to create a system capable of improvement?
Method: Exploring a system capable of improvement
To address this question, in October 2006 a team of researchers from the University of Toronto's Department of Health Policy, Management and Evaluation visited IHC. This site visit was part of an initiative called Quality by Design, which aims to identify and define elements of healthcare systems capable of improvement with a view to helping to inform strategic investments in improvement capability in Ontario. Quality by Design is funded primarily by the Ontario Ministry of Health and Long-Term Care in partnership with the University of Toronto's Department of Health Policy, Management and Evaluation.
IHC was one of five healthcare systems selected from a short list of high-performing systems nominated by a panel of international leaders and experts. In Salt Lake City the team met with and interviewed James, other administrative and clinical leaders and support staff working to make improvements. This case study highlights the findings of that site visit.
The System and its environment
IHC is a non-profit healthcare system and the largest provider of healthcare services in Utah, one of the fastest growing states in the US. With annual revenue of $3 billion, IHC employs over 26,000 people across more than 200 facilities throughout Utah and Idaho. IHC provides care across the continuum (except for long-term care) in 21 hospitals, over 80 out-patient clinics, counselling centres, home health agencies and over 100 medical group practices.
The central administrative offices for IHC and its flagship LDS Hospital are located in Salt Lake City. IHC's 2,200 hospital beds are clustered in nine main hospitals, which account for 95% of the beds and volume (see Table 1 for a summary of IHC's clinical statistics). Twelve of IHC's hospitals are located in urban areas (Salt Lake City-Ogden and Provo-Orem), where they have a strong market presence; the nine remaining hospitals are in rural areas of Utah and southern Idaho. The construction of three new hospital sites is under way and will largely replace current sites (with 300 additional beds). Most notably, in late 2007 the Intermountain Medical Center - a $400 million facility - will replace LDS Hospital (Intermountain Healthcare 2006a).
IHC has over 3,200 affiliated physicians, who account for over 60% of all physicians working in the area. A third of these physicians are employed by the IHC system. Approximately 600 of IHC's employed physicians belong to the Intermountain Medical Group, a network of multi-disciplinary physicians working in IHC's physician, urgent care and occupational health clinics (Intermountain Healthcare nd c). It is estimated that 1,250 physicians of those affiliated provide 90% of the care in the system.
|Table 1. IHC annual clinical statistics for 2005|
|Acute admissions: 123,011
Acute patient days: 497,151
In-patient surgeries: 37,700
Emergency department visits: 429,124
Home care visits: 170,720
Ambulatory surgeries: 96,435
Out-patient visits: 5,559,053
|Source: Intermountain Healthcare (2006a). Used by permission.|
Under the leadership of Dr. David Burton, IHC began to develop its own health plans in 1983. Today, IHC's health plans and its five provider networks have 460,000 members, making up 40% of Utah's commercial market. The development of IHC's health plans has been an important strategy for the health system, protecting it from the excesses and fluctuations experienced by other plans. IHC's primary competitor is Columbia Hospital Corporation of America (HCA). Overall, IHC has managed to keep costs affordable for patients, with competitive health plan premiums and in-patient charges that are 27% lower than the national average and out-patient surgical charges that are 35% lower than those in other Utah facilities. IHC continues to realize enough cost savings to give plan members a rebate (Intermountain Healthcare 2005a).
IHC was established on April 1, 1975, when the Latter-Day Saints Church donated 15 hospitals it owned and operated as a community partnership to this new organization. The Latter-Day Saints hospitals were among the first in the Intermountain area - IHC's flagship LDS Hospital was established in 1905. The donation was made on the condition that a non-profit organization would be formed to operate the hospitals on behalf of the communities they served. IHC was created as a charitable, non-profit, secular organization that owned and managed its hospitals on behalf of the people of Intermountain West. Community leaders governed the newly created organization as unpaid volunteer trustees and named Scott Parker as inaugural CEO. Within a few years several hospitals asked to join IHC and communities invited IHC to build facilities. By 1982 IHC had begun to provide non-hospital services (Intermountain Healthcare 2005a).
Although there is no formal relationship between the Latter-Day Saints Church and IHC, the Church maintains a strong presence in the organization's culture and values. James attributes the constancy of IHC's mission, its commitment to the community and its collaborative culture to the organization's historical roots. For example, he says, "Acting for one's own benefit, at the expense of the collective good, violates the principles on which IHC was founded and is still strongly frowned upon by leaders and staff at the organization." In addition, an important part of IHC's mission is to provide residents "with access to health services, regardless of ability to pay." IHC provides more than $85 million in uncompensated care annually (excluding bad debts, which amount to more than $75 million per year) (Intermountain Healthcare nd d).
IHC's journey to a system capable of quality improvement
In its early days leaders described IHC as a multi-hospital system - "a loose confederation of competing hospitals." By 1986, in the context of escalating healthcare needs and costs and declining reimbursements, Parker - IHC's CEO - saw the integration of hospital operations as a necessary step to improve efficiency. Eliminating a layer of management, the IHC hospital system was restructured into regions with a single parent board of trustees (and advisory boards for facilities), a centralized management structure with a shared general ledger and coordinated budget planning. In addition to realizing operational efficiencies, leaders hoped to improve quality of care while reducing costs and focusing on prevention. A major barrier to achieving this goal was the conflicting aspirations of different parts of the system, notably providers, hospitals and insurers. Parker articulated a vision emphasizing synergy, and physicians, hospitals and health plans began to communicate and work more closely together in order to align their efforts toward achieving higher quality, lower cost and prevention.
In 1993 leaders took further steps to more formally integrate physicians, hospitals and health plans in an effort to "improve the total process of care." A task force explored and recommended options for physician relationships with IHC. The system's leadership hoped to give physicians an organizational voice and to bring clinical sciences into prominence. The task force endorsed a new relationship for physicians that would enhance their integration into the IHC system, including a medical division for physicians who chose to be employed by IHC. As a result, the Intermountain Medical Group was created and physicians were invited to play a more significant role in IHC's management and operations (Intermountain Healthcare 2005a). The Intermountain Medical Group proved to be strategically important for engaging physicians in improvement by providing a laboratory for clinical innovation and holding physicians accountable for results. By the mid 1990s IHC had become a structurally integrated health system.
As early as 1985 CEO Parker required that leaders and managers at IHC demonstrate a return on investment (ROI) for all new initiatives. In order to find more robust ways of calculating ROI, IHC's Department of Finance, led by Steven Busboom, began working on an idea that would separate a hospital's contribution from a physician's contribution to the care of a patient. This work led to the development of one of the first activity-based costing systems in US healthcare (James 1989). According to many observers IHC leadership's early understanding of costs and the information systems needed to track them was rare in the healthcare arena.
Linking the study of variation to leadership of improvement
With physicians becoming increasingly integrated into the system, IHC leaders needed to find ways to support their work and interests, especially for emerging clinical champions. In 1986 Dr. Steven Lewis, IHC's senior vice-president of medical affairs, recruited Dr. Brent C. James from the Harvard School of Public Health to "reach out to physicians in the system, and do research." James, who had initially completed his surgery residency at IHC in the late 1970s, was returning to the Utah-based system with strong statistical and clinical expertise. James began his new role by working on studies of variation in access to care. He quickly became intrigued by variation in clinical practice and quality of care, especially through his work on transurethral prostatectomies (TURP) that showed wide variations among physicians on the grams of tissue removed by surgical time (James 2001a). James noted that issues of practice variation had several causes, including complexity, poor judgment/uncertainty and human error. He felt that despite healthcare's rapidly improving science, these causes highlighted performance and execution problems (2001b). Expanding his work on variation, James began to lead a series of quality, utilization and efficiency (QUE) studies at IHC.
In 1987 James shared his work on variation with Dr. Paul Batalden, a physician peer and clinical leader then at Park Nicollet Clinic in Minneapolis, Minnesota. Batalden - who shared James' interests in clinical quality - introduced James to the work and principles of Dr. W. Edwards Deming, an American statistician and teacher who pioneered quality improvement and who is widely credited with having trained Japanese leaders on methods for improving manufacturing quality. In 1987 James participated in Deming's four-day seminar on quality improvement. Deming's teachings, and particularly his notion that "clinical and cost outcomes are two sides of the same coin," had a profound impact on James, especially given the growing emphasis on ROI among IHC leaders. James returned from the seminar with a deeper appreciation of the principles and methods that could improve variation and the "production" of healthcare: the knowledge that quality drives down cost and that both should be quantified simultaneously.
Given James' medical background, he recognized early on the importance of actively engaging physicians and other clinicians in systematically examining data and mapping clinical processes to identify causes of variation and to test improvement. James created clinical oversight teams composed of physicians and nurses to participate in this work. One of the most notable examples of James' early leadership was his work on reducing post-operative wound infections and tracking associated outcomes and costs at LDS Hospital. James involved clinicians in closely studying and testing all steps of the operative process to determine the point at which patients were at greatest risk of developing wound infections. This process led the team to identify variations in the practice of pre-operative delivery of prophylactic antibiotics as the critical opportunity for improvement (Classen et al. 1992). Eventually, the standardization of surgeons' ordering practices and timing of delivery dramatically reduced the risk of post-operative infection by 50%. Applying the knowledge he had gained from Deming and using the activity-based costing system, James estimated cost savings of $30,000 per (prevented) infection.
Building on the ideas and results that emerged from the QUE studies, in 1989 James published Quality Management for Health Care Delivery, which "presents evidence to support the proposition that an organized system to achieve high quality care can lead to lower health care costs. In the present national environment a highly structured approach to the pursuit of quality is essential" (James 1989: 1). In this book James articulated the ideas that were to inform the journey on which IHC was about to embark.
By 1989, as James and clinical teams began to improve clinical processes and demonstrate better clinical outcomes and costs using a structured approach, CEO Parker and his leadership team gave quality improvement a more prominent role in IHC's core strategy. Creating an infrastructure to support a strategic focus on improvement, the senior leaders created the Quality Mission Statement and formed and led IHC's Quality Council and, eventually, the Core Quality Committee. They also placed responsibility for organization and management of quality improvement with the vice-president of professional services (Intermountain Healthcare 2005a). To create more formalized scientific support for the improvement work that James was leading with clinical teams, IHC's leadership established the Institute for Health Care Delivery Research (hereafter referred to as the Institute) under James' leadership (Intermountain Healthcare nd e). Shifting from a focus on quality assurance, in 1990 IHC explicitly adopted the principles of total quality management (TQM) as an important part of its culture. In 1991 IHC received the Healthcare Forum/Witt Award: Commitment to Quality (Intermountain Healthcare 2005a).
Enabling the improvement of quality and cost with information systems
In addition to the leadership's commitment to quality improvement as a core strategy, a critical accelerator to improvement across the IHC system was the evolution of an integrated information system. As James notes, "An early start created the strongest medical informatics system globally ... and much of Intermountain's success in integrating patient care is attributed to this strong clinical informatics system."1
The use of electronic data has a long history at IHC. In 1964 physician Homer Warner initiated research into clinical information systems and decision support tools for cardiology at LDS Hospital. Warner and his colleagues eventually created HELP, one of the first hospital clinical information systems based on the episodic medical record (HCT Project 2005; Warner et al. 1968). As a result IHC became the first hospital system in the US with an electronic medical record (EMR) system. HELP is still the backbone of IHC's clinical informatics system and is considered by James and others as "one of the finest and most complete electronic medical management systems." Important add-on components of the HELP system include the use of a single, unique identifier for each patient, which makes it possible to link data and for providers to access comprehensive information from any location, as well as built-in case-mix risk adjustment (Narus and Clayton 2002).
Building components of HELP internally created significant advantages for IHC: most critically, it afforded the freedom and flexibility to test new approaches and to customize a system that was congruent with the organization's core strategy. For example, in the late 1980s HELP provided appropriately detailed data for James' studies on variation and tests of process improvement with clinical teams and the ability to link these hospital data to enable activity-based costing. Today IHC's leadership anticipates future system-level priorities and goals by providing resources for the planning and integration of new data elements and analyses into their system one to two years in advance.
After several years of building the initial components of HELP internally as a hospital clinical information system, IHC altered its information architecture to develop HELP2 - a longitudinal electronic patient record linked to a clinical data repository, health data dictionary and decision-support applications called the Enterprise Data Warehouse (see Figure 1). After eight years of development this repository now integrates clinical data with other information, including existing administrative data (hospital, clinic, health plan, clinical laboratory); state birth reports; tumour, cardiovascular and neonatal registries from external sources; and even manually entered discharge documents (Narus and Clayton 2002). The Enterprise Data Warehouse contains 25 years' worth of medical records and provides IHC staff with an extraordinary knowledge base about their patients, the care that they and others provide and its clinical outcomes and costs - all at various levels of aggregation and analysis. Using this infrastructure IHC developed an ambulatory medical care record system used by nearly all its employed physicians (HCT Project 2005). The in-patient system is still largely the old HELP system (attempts to develop applications using the new infrastructure were not fully successful). To complete the in-patient portion of its EMR system, IHC is now partnering with General Electric (GE). The system, which is called Centricity, will be built in Salt Lake City in partnership with GE (HCT Project 2005).
Overall, the ability to build on and link key clinical process, outcome and finance measures enables the information system to develop and track a balanced and relevant set of measures for accountability and system performance as well as day-to-day clinical process improvement. Systems elsewhere are often limited by the availability of data in administrative databases; as a result they frequently maintain a disproportionate focus on accountability for finances and facilities management.
In addition to developing its first-class information system, IHC has invested in the use of its information for clinical decision-making and inter-professional collaboration. The interface between the EMR and care providers is at the point of care; in some of IHC's facilities, providers can enter and view data using clinical workstations in patient rooms (Narus and Clayton 2002). For decades clinicians working in LDS Hospital's ICU and other areas have conducted patient rounds at computer terminals where they can view all patient information, including laboratory and diagnostic imaging reports. In addition, an outcomes tracking system is available to staff on IHC's intranet, displaying standard reports customized for clinical programs that include control charts, comparative benchmarks and other tools for monitoring and flagging significant changes over time. Although a few clinical programs operate completely electronically, an IHC analyst estimates that still only 10%-15% of data is exclusively electronic at the point of care across the IHC system: "There is still a lot of paper to data entry going on." The GE partnership is intended to help expedite a more complete shift to electronic documentation.
Although physicians and other clinicians at IHC were initially reluctant to use the EMR system, increased productivity and quality helped convince many of them of its benefit. In the spirit of IHC's evidence-based approach to care and quality improvement and James' Institute, a number of empirical studies were conducted to illustrate for clinicians that using the EMR system would decrease waste, improve reimbursement and aid in decision-making (Narus and Clayton 2002). A recent report suggests this evolution of business intelligence tools at IHC now enables analysts to spend 90% of their time - which they used to devote to data collection - working with clinicians and managers to improve processes (Sanders 2002). The integrated data systems are also helping to demonstrate that quality and cost can be jointly pursued as two important imperatives for the system, rather than be seen as competing goals.
Developing improvement knowledge and skills - the Advanced Training Program
In order to facilitate the study and improvement of clinical processes using data from IHC's information systems to achieve better performance for the system, clinicians and leaders had to develop their own knowledge and skills in the use of improvement methods and tools. According to James, "You have to have training in quality improvement to change the culture."
Having furthered his own learning at Deming's seminar, James developed educational programs for IHC clinical staff and leaders and offered them through the Institute. By 1990 this education had evolved into the Advanced Training Program (ATP), which addresses quality improvement theory, measurement and tools, healthcare policy and systems and leadership. Since its inception the ATP has gained recognition from sector leaders such as Donald M. Berwick, president and CEO of the Institute for Healthcare Improvement (IHI), who has said, "The ATP is the finest training program we know of for bringing front-line clinicians, health care leaders, and internal change agents to a deeper understanding of what it means to make quality the core strategy for an organization" (Intermountain Healthcare nd f).
The ATP does not endorse one specific approach or method for improvement (e.g., Plan-Do-Study-Act, Model for Improvement, Lean, Six Sigma); instead, it teaches a core set of improvement principles and introduces tools from a variety of approaches. A key principle of the ATP is that it is action-based and participants are required to apply their learning to an improvement project. Participants are paired with Institute staff members who provide mentorship and coaching support as they work on their projects between sessions. Participants in turn share their project results at the conclusion of their training. In addition to teaching more than half of the ATP, James brings in some well-recognized external quality improvement experts as faculty, including Ken Kizer, David Eddy and James Reinertsen. The Institute offers two versions of the program: an 11-day mini-ATP course largely geared to clinicians and a full 20-day ATP course (Intermountain Healthcare nd g).
In the early 1990s ATP participants were predominantly drawn from within the IHC system. Bringing seasoned clinicians, managers and administrators together to learn the same core theory, methods and tools was a revolutionary concept at the time. According to James,
The course has helped to create a palpable cultural change at Intermountain. ... The group learning and mentored project work fostered an environment of interdisciplinary collaboration ... and it created more widespread support for quality improvement ... so that it wasn't just me. Participants could bear personal testimony to successful quality improvement.
Since 1991, graduation from the full ATP course has been a requirement for all IHC senior managers and leaders. James notes that over 30 to 40 of IHC's physician leaders have become champions of quality improvement following ATP participation.
To date 2,000 people have graduated from the ATP (including external participants), generating more than 1,000 quality improvement projects. Quantifying results of the ATP projects from IHC participants, James estimates that the program has yielded a 4-to-1 ROI for the IHC system. Today only 20% of participants are from within IHC. Because the Institute is required to break even in order to pay staff and not to generate profit, James has been able to continue to offer the ATP to external participants at an affordable rate. Participants' backgrounds are varied: 40% physician executives, 20% nurse/other clinician executives, 15% administrative staff, 10% senior executives and 10% academic researchers. Participants come from a broad range of international and national jurisdictions, and an explicit goal of the program is the continued networking of students and alumni. According to James there are more than 15 ATP-type programs around the world and he hopes to expand this number.
Clinical integration strategy
Under Parker's continued leadership, by the mid 1990s IHC had made important strategic investments in improvement capabilities. These included giving physicians and clinical scientists more prominence in managing the system, developing leadership-level quality oversight infrastructure, creating an integrated data repository and launching a training program for improvement knowledge and skills. After 20 years under Parker, its inaugural CEO, IHC named a new CEO, Bill Nelson.
Shortly after his arrival in 1995 Nelson noted some key challenges facing IHC's improvement strategy. Most notably, clinical quality improvements and parallel cost improvements were largely focused on only one area of the system. Furthermore, managers and clinical leaders were still successfully using advocacy (rather than evidence) to dictate priorities for improvement. Nelson asked James and Dr. David Burton, a senior clinical leader and IHC's senior strategic planner, to lead the development of a new strategic quality plan that would transform improvement at IHC from an emphasis on projects in parts of the system to a core business strategy that could add value across the system. Nelson envisioned a strategy focused on designing an improvement system that would systematize and deploy better-quality outcomes and costs throughout IHC and that could "cut through the advocacy to create a sense of discipline and purposeful focus at IHC." The system's new leader hoped to build on the investments already made to make improvement a system imperative rather than an organizational or individual responsibility.
Responding to Nelson's strategic vision, Burton and James developed a strategy they called "clinical integration." Based in part on the Baldrige model and Deming's work, this strategy focuses on a data-driven analysis and prioritization of key value-added processes and on the reorganization and realignment of the system around a set of high-priority clinical areas. Using data on service delivery, priority clinical areas were identified. Within these priority clinical areas clinicians and other leaders were to map and develop evidence-based care models for vital processes and then build them into workflows. This approach to process management also involves aligning management structures and systems (e.g., information systems, management infrastructure, accountability mechanisms, incentives) to the redesign and standardization of care. According to James (2001c: 34), "Clinical integration shifts the management paradigm further from a financial/administrative model to a clinical process model - the core way we actually accomplish work and add value."
Selection of priority clinical programs
In the first phase of the strategy development Burton and James used a set of criteria and data to redesign and focus the system around specific programs across IHC regions. The prioritization criteria included patient volumes, cost per case (in-patient and out-patient), variation in clinical quality, team-based (or microsystem) processes of care and social equity (see Table 2 for a sample of the data used in this exercise). As a result of this process nine clinical programs were identified as priorities: Cardiovascular, Neuromusculoskeletal, Surgical, Women and Newborn, Intensive Medicine, Pediatrics, Behavioral Health, Oncology and Primary Care. Across these clinical programs, the Cardiovascular and Women and Newborn programs were ranked as the top two priorities and further development of the strategy began in these two areas.
|Table 2. Sample of cost data used to prioritize IHC's clinical programs|
|Clinical Program||Hospital In-Patient and Out-Patient Cost ($US)||% of Total Cost|
|Women and Newborn||$114,984,231||16.4%|
|Total: Clinical Programs||$646,260,602||92.4%|
|ICU & Trauma||$31,079,870||4.4%|
|Total: All Hospital Cases||$646,260,602||100%|
|Source: James (2001c). Used by permission.|
Development of evidence-based clinical process models
In the second phase of the strategy development, clinicians in each priority clinical area worked in collaborative groups called "development teams" to map key processes. Development teams include a team leader who is a specialist and expert in the clinical field and clinicians who represent regions across the IHC system. Team members work together to create and adapt care process models, including evidence-based practice guidelines and clinical protocols using conceptual flow diagrams and other methods and tools (James 2001c). The goal of each team's work is to "reflect the inherent hierarchical process/outcomes structure of every clinical process." As shown in Figure 2, teams start with a high-level clinical process model (e.g., labour and delivery), and use that as a starting point for developing increasingly more detailed models of evidence-based care for each process.
There is one development team for each major clinical process. Within each team core, work groups are assigned to specific processes. Development teams are charged with working on four types of care processes: clinical conditions, non-condition specific clinical processes, service quality and administrative processes. A member of each team also has the task of staying current with the best available evidence and IHC's current performance in order to identify ongoing opportunities to adapt current process models and to develop new process models. The teams hold regional clinical learning days to engage clinicians in defining strategic improvement goals, and they share the clinical process models more broadly within their clinical priority areas, seeking feedback prior to proceeding with further phases of the strategy.
For each priority area development teams identify a family of processes that account for a large proportion of volume and cost. Burton and James note that 10% of clinical processes accounted for 90% of volumes. As shown in Table 3, the Women and Newborn team determined that labour and delivery had a family of eight key processes.
|Table 3. Sample of data on women and newborn family of processes|
|Women and Newborn Family of Processes|
|Processes||DRGs||In-patient and Out-patient Costs ($US)||%||Cumulative %|
|Pregnancy and Delivery|
|Complicated term newborn||385, 389-390||$11,037,703||10.3%||61.4%|
|Post-partum and other||376-377, 384||$1,343,002||1.3%||84.2%|
|Total: Pregnancy and Delivery||$90,430,298||84.2%|
|Benign gyn conditions||356, 358-360, 364-365, 368-369||$14,973,415||14.0%||98.2%|
|Malignant gyn conditions||353-357, 363, 366-367||$1,922,540||1.8%||100.0%|
|Total: Women and Newborn||$107,326,253||100%|
|Source: James (2001c). Used by permission.|
James estimates that the clinical process models continue to be adapted every six months. Based on IHC's evaluation of the implementation of this strategy, clinical teams work on a maximum of four clinical process models in a clinical setting at one time. According to James, "Any more than four and the paper shuffling overwhelms us."
Integration of process models into information systems
In addition to leading the creation of clinical models for improvement through process management, the development teams are instrumental in subsequent phases of the strategy. To help build the clinical process models into workflow, the teams generate a list of desired outcomes for tracking reports using flow diagrams to identify key measures for monthly reporting and the desired level of aggregation of those measures. For labour and delivery, for example, teams identified a series of outcome measures for tracking, including Caesarean delivery/Vaginal birth after Caesarean (VBAC) rates, measures of clinical process failure, satisfaction and cost (see Table 4). The teams work with support staff to develop reports using existing computerized data, standardized manual systems or newly created data systems.
|Table 4. Outcomes for tracking - labour and delivery|
|Caesarean and VBAC delivery rates
|Clinical process failures
|Cost outcomes - relative resource utilization (RRU)
|Source: James (2001c).
Used by permission.
James reports that when comparing the availability of data in IHC's system in the mid 1990s to the requirements generated throughout the development of the clinical integration strategy, IHC's information system had only 50%-70% of the data needed. The clinical integration strategy was a key driver for the development of the clinical repository described earlier in this report. According to James (2001c), "In the long term ... the primary purpose of an electronic medical record is protocol support. ... We were not able to show a return on investment for our electronic medical record systems until we combined them with our clinical improvement. ... Informatics builds the tools; clinical quality improvement builds the content."
To facilitate use of data for decision-making, worksheets were developed that automatically retrieve all the necessary clinical information, including standing order sets customized to patients, automated advisories and alerts and monthly reports posted on the intranet. As an example, the clinical integration strategy supports general practitioners (GPs) and their practices in key high-volume, high-cost conditions such as diabetes, chronic heart failure and asthma. GPs receive ongoing electronic updates with specific indicators for their patients' management in the form of patient worksheets. These include patients' general status as well as disease-specific information. Patients' problems and chronic conditions are detailed, and the worksheets include medication profiles, preventive care summaries, pertinent laboratory results and pertinent exams as well as passive reminders to physicians, organized by illness (e.g., regular eye checks for diabetes patients).
Clinicians develop and test prototypes of these tools prior to integrating them formally into the system. James describes the integration of clinical process models into decision support systems as "codifying knowledge through expert system support." Although this strategy enhances the standardization of clinical care based on evidence, it is meant to "provide a shared baseline" that allows physicians to vary decisions based on individual patient needs. The strategy supports evidence-based decision-making by building it into workflow, but does not dictate it.
Development of educational materials
Following the integration of data elements from the clinical process models into IHC's information systems, assigned members of the development teams participate in the design of educational materials both for clinical teams in the priority areas and for patients. IHC has a department of eight staff members who develop educational materials as part of the clinical integration strategy. These materials are aimed at helping guide clinicians as they put evidence into practice.
The ATP and other educational and training programs offered at the Institute include a focus on clinical integration. The patient-oriented materials assist patients to understand what they should expect and how they can participate in their care. James estimates that it takes nine months to develop and integrate a clinical process model into workflow.
Integration into management and support structure
IHC's senior leadership redesigned its management and accountability structure (see Figure 3) to support its clinical integration strategy (James 2001c). The theory behind this redesign is that each clinical priority program, such as Women and Newborn, will become a "centre of excellence" with resources to support ongoing improvements in care.
In each clinical program, regional management teams comprising a medical director (0.25 full-time equivalent [FTE]) and a nurse administrator (1 FTE) provide leadership within IHC's Urban North, Urban Central and Urban South regions. Medical directors are carefully selected based on credibility with their peers in the specific clinical area, demonstrated management and leadership skills and ongoing clinical practice and improvement knowledge. A medical director's role is to work with front-line clinical staff; he or she identifies issues in the implementation of clinical process management, sets clinical goals and holds clinical teams accountable for performance. Nurse administrators oversee each clinical program's operations and support the use of rapid cycle improvement in the implementation of clinical process models.
In addition to the regional structure, each priority clinical program is led at the system level by a central program management team that is also made up of a medical director (0.25 FTE) and a nurse administrator (1 FTE), as well as a statistician, a data manager and a support team (e.g., finance, human resources). The central program management teams oversee priority- and goal-setting for the system, budget-planning for the program area, coordination for process management and support across regions. The statisticians and data managers play a critical role in managing the data flow within the clinical programs, supporting teams as they implement new clinical process models and reviewing system outcomes.
Regional and central clinical program management teams as well as development teams meet regularly as a guidance council to review clinical and financial outcomes, set improvement goals, identify resource and support needs, pinpoint opportunities for improvement and share ideas and practices. Each priority clinical program's guidance council reports to a clinical integration executive team that provides system-level leadership for the deployment of the strategy and that ultimately reports to IHC's board.
Every year IHC's board members - half of whom have a medical or nursing background - work with the executive team to set between two and four board goals for each clinical program. These goals typically include at least one each of clinical, financial and service-quality goals, and they drive budget and planning for the clinical programs. In addition to these board goals the guidance council and program management teams select and monitor a series of additional goals in order to gauge implementation of the evidence-based clinical process models. Statisticians draw on the data repository to create registries for the clinical programs and to prepare customized regional, system-level and board performance reports showing progress on goals; this work is in addition to the outcomes tracking available to staff on the intranet. The data in the performance reports and on the intranet are as timely as one month from patient care and, in some cases, one week. The resulting organizational structure focuses on care delivery and links information systems designed to provide data on key organizational processes with a structure that manages and continually seeks to improve core clinical processes.
Aligning incentives for clinical integration
In addition to the redesigned management structure, IHC's leadership has used information and incentives to support the clinical integration strategy. Within each clinical program physicians are given a set of core indicators that measure their own results against peer, regional and system-level results and goals. According to James, these reports encourage a form of "healthy competition that pushes people ahead and has been one strategic lever used to shift physicians towards acceptance of quality as a standard business process and operation." Regional-level medical directors use physician-level data as evidence of contributions to system-level improvement and to guide performance evaluations of employed physicians. Practice groups may be financially rewarded for improvement. Recently, some groups have received up to $25,000 in recognition of their contributions to achieving system-level goals. Senior leaders at regional and system levels have 25% of their salary contingent on the achievement of board goals.
Value for investment in clinical integration
After more than a decade of implementing this strategy across several priority programs, has the clinical integration strategy accomplished the vision Nelson set out in 1995?
As evidence of the spread of improvement across IHC, James estimates that the system is now regularly achieving 80% of its goals (higher targets are set every year). Several program-specific improvements in clinical and financial outcomes that have emerged as a result of this strategy are described in the introductory section of this case study and demonstrate measurable clinical and financial value. The centre-of-excellence concept has created an infrastructure and discipline for focused, evidence-based improvement that moves beyond projects to create system responsibilities and everyday work processes. The clinical integration strategy and its structured, phased approach of selecting key processes, mapping evidence-based models, automating information systems and workflow, developing competencies and integrating itself into management structure and incentives are now being applied to patient safety and the prevention of adverse drug events (James 2005a). James estimates that the variable cost savings resulting from the clinical integration strategy is $15 million US, on a $4 million US investment.
James notes that harvesting quality savings can be accomplished using three strategies:
- Targeting specific improvement projects to achieve internal savings
- Using evidence of improved results in contract negotiations with payers
- Partnering with purchasers in "shared risk"
IHC leaders pay careful attention to all three of these strategies. Because clinical process redesign can reduce hospital and other service utilization revenues, it is important to take a broad system view and to use these improvements to negotiate with payers. According to James (2001c), "You must align contracting strategies to harvest savings back to your organizations; otherwise, clinical quality improvement is a fast way to destroy your organization financially." In the past, several external factors affected IHC's return on investment. In particular, costs at IHC were decreasing as a result of improvement (e.g., community-acquired pneumonia); however, there was no parallel improvement in revenue because the Diagnosis-Related Group (DRG) coding system "was cheating" them. Improvements in the system over time have changed this.
Overall (and inclusive of the effects of clinical integration), James estimates IHC has saved approximately $100 million US through clinical quality improvement. Although IHC does not have a formal reinvestment savings plan, the savings are evident through excess system capacity and are directed toward capital investment. Purchasers are also beginning to benefit from quality improvement. Drawing on a recent surplus, IHC refunded 6% of annual insurance premiums to the 460,000 members of its health plans.
Internal and external supports for quality improvement at IHC
IHC has additional internal support resources beyond the management and infrastructure developed to support clinical integration as the core strategy. Nine statisticians working in the priority clinical programs are employed at James' Institute. In addition to supporting these programs, these statisticians support the development of the data repository, mentor ATP participants and help individual teams and clinicians with research. At a corporate level IHC employs analysts dedicated to the management of the clinical repository, a hospital analytics team responsible for reporting standards and providing support to clinical programs and a tactical team focused on specific business intelligence tools. Furthermore, IHC has a corporate quality management department and a team of nurses responsible for managing the external reporting and regulatory requirements of national and accreditation bodies (e.g., Centers for Medicare and Medicaid Services [CMS], The Joint Commission on Accreditation of Healthcare Organizations [JCAHO], Healthcare Effectiveness Data and Information Set [HEDIS]), thereby insulating the priority clinical programs from these demands. Although some of IHC's core goals and measures align with externally driven measures, many do not. James notes that many of these externally driven measures are "not designed for front-line use and their definitions are not functional." Individual organizations across IHC also have their own quality management departments.
To external observers the linkage and interaction between these support resources and the clinical integration strategy is not always clear, and there appears to be some overlap among these roles as well as potential redundancies. The extent to which clinical programs use specific types of support seems dependent on factors such as relationships and preferences. In addition, although a vital resource for most in the execution of the clinical integration strategy, the Institute has recently been perceived by some as an entity that is increasingly external to IHC.
Unlike some other high-performing systems, IHC appears to have developed and executed its improvement strategy by focusing and building its improvement capability internally. In addition to its information system, the development of training and skills has been homegrown. Unlike the case with several other US organizations, IHI is not a critical support resource for IHC. IHC participated in earlier collaboratives, and James and other IHC clinicians provide expertise and advisory support for IHI. However, IHC's priorities and targets differ from IHI's. According to James, "We didn't participate in 100,000 lives because we had been working on the same interventions and achieving targets for years ... and we had moved on."
Challenges and future directions
IHC's strategy for improvement is based on evidence-directed clinical process management in carefully selected priority areas as well as on the alignment and integration of decision support, training and education, management and accountability. It also depends on incentives for this work. This strategy is widely recognized as having been successful in creating system-level transformation and is being implemented at the Mayo Clinic and a number of other health systems elsewhere. As one group of researchers has observed, "One of the best examples of an acute care provider that has used outcome information for strategic advantage is IHC" (Thompson et al. 2000: 579).
Like all strategies, however, IHC's has its challenges, and these raise several questions about future directions at the organization. IHC leaders note that although it was initially intended to have a broader focus, the strategy has not yet been used to improve administrative processes; rather, some separate support resources appear to be emerging for this work. In addition, certain non-clinical processes (e.g., patient flow) now require a more nimble organizational strategy. To date, focused work within priority clinical programs has had limited cross-program collaboration on such critical issues and, according to a senior-level staff member, "There is a need to shake up the silos."
Furthermore, at a corporate level IHC has recently developed a new set of extraordinary goals for clinical excellence, service excellence, operational effectiveness, physician engagement and community engagement. Will a new management structure and model be required for cross-program work and the achievement of new corporate goals, or can these be built on top of what has already been developed? What will these innovations do to the momentum created so far as leaders engaged in the clinical integration strategy continue to spread the concept to other priority program areas?
Led by clinicians, the IHC strategy has been extraordinarily successful at initiating a major cultural shift throughout the system by engaging front-line clinicians in defining excellence in clinical care and by working actively in teams on improvement. However, despite some of the innovations mentioned earlier in this report, creating automated tools that physicians and other clinicians feel comfortable using is an ongoing challenge for IHC. How will the new in-patient EMR that will be developed in partnership with GE help, or will it potentially pose further challenges?
The extent to which non-employed physicians have engaged in strategy deployment also varies. Despite the flexibility built into the system and its bottom-up development, an ongoing challenge is the perception of physicians outside the IHC medical group that the clinical integration strategy comes at the cost of local and professional autonomy and micromanagement from high-level decision-makers. IHC has assumed that achieving improvements among those physicians closely aligned with the system will help to win over the rest. But will IHC again need to change its relationship options to attract more physicians as employees or articulate strategies to develop more clinical champions among the non-employed group? Will it require an enhanced relationship between IHC and medical schools so that the principles of clinical integration and improvement are introduced to physicians earlier in their education?
A critical success factor for IHC has been its investment in the development of improvement capability through decision support and training in improvement skills and knowledge. The tension and potential overlap between the various sources of support at corporate and regional levels and within the Institute is an ongoing challenge. Will IHC's leaders streamline these resources and initiate clearer mandates and expectations for their use? Current support for clinical programs implementing the clinical integration strategy appears to emphasize analytical support more than support in the use of improvement methods, tools and change management. Clinical leaders are expected to have this knowledge and these skills as a result of their participation in the ATP, and this is also part of the nurse administrators' role. Is this a reasonable strategy and do clinical leaders actually have the time needed to actively support teams in improvement and change management? Will the Institute continue to grow to support external organizations and participants, and how will this affect its identity as a critical resource for internal support at IHC? What more will be needed?
After more than a decade as CEO, Nelson and his team - including James - continue to lead IHC. But beyond the Institute's ATP, how is IHC developing its leaders? Leadership continuity and mission constancy have been critical to the success of IHC's plan. Will eventual changes in leadership change the current direction from evidence-based clinical improvement to one of operations and growth? How much of IHC's success is supported by its social and cultural environment? And that begs the question, would IHC have been as successful using this strategy in vastly different healthcare environments such as California or New York?
Finally, how will IHC continue in its quest to be "ever better"?
American Nurses Credentialing Center. 2007. "ANCC Magnet Recognition Program® - Recognizing Excellence in Nursing Services." Silver Spring, MD: Author. Retrieved January 21, 2007. < http://nursecredentialing.org/magnet/ >
Classen, D.C., R.S. Evans, S.L. Pestotnik, S.D. Horn, R.L. Menlove and J.P. Burke. 1992. "The Timing of Prophylactic Administration of Antibiotics and the Risk of Surgical-Wound Infection." New England Journal of Medicine 326(5): 281-286.
HCT Project. 2005. Local Health Information Infrastructure Initiatives. Q&A with CIO Marc Probst. San Francisco: Montgomery Research. Retrieved January 21, 2007. < http://www.HCTProject.com >
Health First. 2005. "Health First Ranked One of the Top 100 Integrated Healthcare Networks in the US." Rockledge, FL: Author. Retrieved January 10, 2007. < http://www.health-first.org/news_and_events/ hf_verispan_top_100_2005.cfm >
Hospitals and Health Networks. 2006. "The 100 Most Wired Hospitals." Chicago: Health Forum Inc. Retrieved January 21, 2007. < http://www.hhnmag.com >
Intermountain Healthcare. nd a. Mission, Vision and Values. Salt Lake City: Author. Retrieved January 4, 2007. < http://intermountainhealthcare.org/ xp/public/aboutihc/mission.xml >
Intermountain Healthcare. nd b. Intermountain - Affordability Reducing Costs. Salt Lake City: Author. Retrieved January 21, 2007. < http://intermountainhealthcare.org/xp/public/ about-intermountain/affordability/reducingcosts.xml >
Intermountain Healthcare. nd c. Intermountain Medical Group. Salt Lake City: Author. Retrieved January 11, 2007. < http://www.intermountainhealthcare.org/xp/ public/doctors/medicalgroup/ >
Intermountain Healthcare. nd d. Intermountain Healthcare - Our Nonprofit Mission. Salt Lake City: Author. Retrieved January 4, 2007. < http://www.intermountainhealthcare.org/xp/ public.aboutihc/nonprofit/ >
Intermountain Healthcare. nd e. Institute History. Salt Lake City: Author. Retrieved January 11, 2007. < http://intermountainhealthcare.org/xp/public/ institute/about/history.xml >
Intermountain Healthcare. nd f. Advanced Training Program. Salt Lake City: Author. Retrieved January 11, 2007. < http://www.ihi.org/IHI/Programs/ConferencesAnd Training/AdvancedTrainingProgram.htm >
Intermountain Healthcare. nd g. Institute of Healthcare Delivery Research - Education Programs. Salt Lake City: Author. Retrieved January 11, 2007. < http://intermountainhealthcare.org/ xp/public/institute/education/ >
Intermountain Healthcare. nd h. IHC Primary Care Clinical Programs: Adult Diabetes Patients in Your Practice. Salt Lake City: Author.
Intermountain Healthcare. 2005a. Annual Report 2004. Salt Lake City: Author.
Intermountain Healthcare. 2005b. Healing for Life - Annual Nursing Update. Salt Lake City: Author. Retrieved January 21, 2007. < http://www.intermountainhealthcare.org/ xp/public/documents/corp/nursingupdate.pdf >
Intermountain Healthcare. 2006a. Annual Report 2005. Salt Lake City: Author. Retrieved January 4, 2007. < http://intermountainhealthcare.org/xp/public/ documents/corp/annualreport.pdf >
Intermountain Healthcare. 2006b. "LDS Hospital Again Ranked One of America's Best Hospitals in U.S. News & World Report Magazine's 2006 Rankings of Nation's Top Health Care Centers." Salt Lake City: Author. Retrieved January 10, 2007. < http://intermountainhealthcare.org/xp/ public/lds/aboutus/news/article21.xml >
James, B.C. 1989. Quality Management for Health Care Delivery. Chicago: The Hospital Research and Education Trust of the American Hospital Association. Retrieved January 4, 2007. < http://intermountainhealthcare.org/xp/ public/documents/institute/articles_qmmp.pdf >
James, B.C. 2001a. Applying Research to Affect Clinicians and Clinical Governance. Presentation at the 4th International Conference on the Scientific Basis of Health Services, Sydney, Australia. Retrieved January 11, 2007. < http://cme.ihc.com/xp/ihc/documents/ institute/conferencepresentations.pdf >
James, B.C. 2001b. The Scientific Basis of Quality Improvement. Presentation at the 4th International Conference on the Scientific Basis of Health Sciences, Sydney, Australia. Retrieved January 4, 2007. < http://cme.ihc.com/xp/ihc/documents/ institute/conferencepresentations.pdf >
James, B.C. 2001c. You Manage What You Measure: Tracking Clinical Outcomes. Presentation at the Sharp HealthCare Quality Leadership Conference, San Diego. Retrieved January 21, 2007. < http://intermountainhealthcare.org/xp/ public/documents/institute/sharp530.pdf >
James, B.C. 2005a. Creating an Integrated, High Quality Health Care Delivery System that Supports Medication Safety. Presentation to the Institute of Medicine Committee on Identifying and Preventing Medication Injuries, Washington DC.
James, B.C. 2005b. Quality and Cost - What You Can't Afford to Ignore. Presentation at the Introduction to State Health Policy - A Seminar for New State Legislators (Agency for Healthcare Research and Quality), Rockville, MD. Retrieved January 21, 2007. < http://www.ahrq.gov/news/ulp/statepolicy/james.ppt >
Lappé, J.M., J.B. Muhlestein, D.L. Lappé, R.S. Badger, T.L. Bair, R. Brockman, T.K. French, L.C. Hofmann, B.D Horne, S. Kralick-Goldberg, N. Nicponski, J.A. Orton, R.R. Pearson, D.G. Renlund, H. Rimmasch, C. Roberts and J.L. Anderson. 2004. "Improvements in 1-Year Cardiovascular Clinical Outcomes Associated with a Hospital-Based Discharge Medication Program." Annals of Internal Medicine 141: 446-453.
Narus, S.D. and P.D. Clayton. 2002. "Clinical Information Systems at Intermountain Healthcare." Imaging Economics (February). Retrieved January 4, 2007. < http://www.imagingeconomics.com/ issues/articles/2002-02_12.asp >
Sanders, D. 2002. "Intermountain Healthcare: Designing, Developing and Supporting an Enterprise Data Warehouse (EDW) in Healthcare." In Protti, D.J., ed., How Business Intelligence Is Making Healthcare Smarter: World View Report 10. 2005. London: NHS Connecting for Health. Retrieved January 4, 2007. < http://www.connectingforhealth.nhs.uk/ newsroom/worldview/protti10/ >
Thompson, D.I., C. Sirio and P. Holt. 2000. "The Strategic Use of Outcome Information." Joint Commission Journal on Quality Improvement 26(10): 576-586.
Warner, H.R., R.M. Gardner, T.A. Pryor and W.M. Stauffer. 1968. Computer Automated Heart Catheterization Laboratory. UCLA Forum in Medical Science #10. Berkeley: University of California.
1 All unattributed quotations by James and others were gathered by the Quality by Design team in conversations with James undertaken during our preparation of this case study.
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed