Abstract

Public reporting of comparative quality performance measurements of healthcare providers is a recent U.S. and world trend: not a fad. While there are significant benefits to be derived from such an initiative, there are staggering hurdles that must be overcome as well. For 10 years, I have been privileged to view public reporting at the very interface of purchaser-provider interaction in the Cleveland market. As Canada and its provinces consider public reporting of performance measurements, I believe there are lessons to be learned from the Cleveland experience.

The Cleveland Health Quality Choice Program

Frustrated by high costs, annual double-digit inflation, excess capacity, fee-for-service payment systems, uncertainty about quality and increasing competition in an expanding global market, Cleveland employers initiated a collaborative program with area hospitals and physicians to address these issues in 1989.

The coalition's charter provides an opportunity for area hospitals to voluntarily submit data that would provide a comparison of performance to be shared with employers to enable them to make informed purchasing decisions. The Cleveland Health Quality Choice (CHQC) program set out to measure and compare risk-adjusted outcome performance in 27 area hospitals and report on these findings. The first semiannual report was released in April 1993. Out of the fear that a high-performing provider or others might release private data to the media to its advantage, it was decided that a consumer report should be prepared and released to the public.

The CHQC program uses quality indicators selected by consensus to report on intensive care, general medical, general surgical and obstetrical inpatient services. The outcomes reported are mortality rates, length of stay, cesarean section rates, vaginal birth following a prior cesarean section (VBAC) and patient satisfaction. The program employs state-of-the-art clinical systems (e.g., Apache III System, CHOICE System, NRC/HRS Healthcare) to control for patient risk factors that could influence the outcome but are not controlled by the provider.

Cost and Quality as Monitored by CHQC

By one account, Cleveland improved from the fourth most expensive central U.S. city for healthcare in 1985 to somewhere in the third quartile of 100 cities by the mid-1990s. Annual double-digit inflation subsided in the early 1990's, as it did in many U.S. cities. By 1997, Cleveland experienced the lowest annual increase in overall healthcare costs of the largest 100 U.S. cities. Cleveland also experienced extraordinary improvement in quality outcomes as measured by CHQC. Mortality rates declined by almost 25% and length of stay by 30%. Cesarean section rates declined 15%, and VBAC rates increased 20%.

Benefits of Public Reporting

Many factors contributed to the changes monitored and reported by Cleveland Health Quality Choice and others. Contributing factors include advancing medical technology, process-of-care improvement initiatives undertaken by hospitals and physicians, aggressive managed care practices designed to lower costs and employer-purchaser initiatives. We believe that CHQC and the public report contributed as well.

Few dispute that public reporting of valid and reliable performance data serves as a positive influence to drive quality improvement. Scientists and industry observers have reported on this influence in regions where performance information is in the public domain (e.g. Cleveland Health Quality Choice, the New York Cardiac Registry, the Pennsylvania Health Care Cost Containment Commission).

Public reporting heightens the awareness of healthcare providers. The benefactor is the community. It is not unusual for portions of the Cleveland public report to appear in the local newspapers or on the evening news. Hospital trustees and executive officers now give equal time and weight to financial and quality reports. Results suggest that hospitals and physicians have learned how to use comparative performance reports to identify opportunities for improvement and have implemented process-of-care changes that have resulted in improved outcomes. Public accountability enhances the urgency to do so!

Employer-purchasers and their managed-care intermediaries are obligated to ensure that network providers offer healthcare to employees and their families that is acceptable, if not commendable. Public reporting of performance measures maintains pressure on employers and managed-care organizations to invoke value purchasing decisions on behalf of their employees: to make available the highest quality healthcare at reasonable costs.

Purchasers use public data to reassure themselves and their employees that healthcare available is of acceptable quality; to serve as a basis for dialogue with providers to encourage quality improvement; or, in extreme and rare circumstances, to include or exclude a provider from the network.

The Downside of Public Reporting

There are two major adverse aspects to public reporting to consider. Public reporting heightens the level of anxiety among providers and markedly increases their demand that data reliability and measurement methodologies achieve near perfection, as market share may be impacted. It also raises the spectre that those who use the information to make choices may not be fully informed of the limitations and interpretations of the data provided, leading to errant conclusions.

Individual provider performance is normally compared to a standard. This standard is usually a national, regional or local performance mean score or average. If some providers perform above the average (mean), other providers will perform below average. Public reporting of performance heightens the anxiety for those who are below average or for those in which the outcome reported does not agree with their public quality persona.

Upon reviewing public reports, it is not unusual for some providers to demonstrate symptoms of the process described by Elizabeth Kiebler Ross in her description of death and dying. We have witnessed expressions of denial, anger, guilt and, in many cases, eventual acceptance of the data as meaningful and indicative of performance improvement opportunities by physicians and hospitals.

Cleveland Health Quality Choice employs costly, state-of-the-art performance measurement systems selected by area physicians and described by Lisa Iezonni, M.D. of Harvard as "the most rigorous used anywhere." Yet, much of our time is spent fighting rearguard actions against allegations from some hospitals that the "systems are flawed," another hospital is "gaming the system," or that "my patients are sicker and your methods do not account for that." No measurement methodology available is perfect or foolproof. These confrontations are costly and demeaning of important information that is useful.

The practice of medicine is imperfect, but it is advancing at the speed of light. The science of adjusting data for patient risk factors or comparing quality performance is far from perfected and is in its infancy. Even more primitive is the use of comparative performance information by purchasers and consumers to make informed decisions.

Little is known about purchaser use of information to make informed decisions. There is discouraging news regarding consumer use of public reports to make informed decisions. Where data are publicly available, studies indicate that the majority of U.S. consumers are not aware of their presence. When they are aware, studies indicate few bother to access the information. Almost no individual interviewed is known to have used available information to make a decision on where to receive healthcare. With the public release of performance data goes a responsibility to launch a massive education campaign to ensure that users understand the use of data and the interpretation caveats necessary to make a truly informed decision.

The Media

Public reports invite media participation. Our program has experienced mostly positive treatment by the local media, and we attribute this to our policy. Our staff makes every effort to assist print media reporters. They receive a personal briefing and the report 48 hours in advance of release, allowing adequate time for them to write their stories. However, media treatment of performance information is subject to editorial policy and reporters' understanding of statistics. Editorial policy will determine if the information is used to promote community understanding of healthcare choices, or if the information will be used to sensationalize stories about poor performance. Electronic media often provide 30 seconds of coverage focused on ranking the best or worst performers.

Conclusions

The public release of information comparing the quality of care delivered by healthcare providers is a major undertaking. Hospitals and physicians traditionally use clinical information to identify opportunities for improvement. However, the public disclosure of comparative performance information spotlights these provider efforts and increases the responsibility of purchasers. Successful introduction of this information will depend upon how well the benefits and challenges are managed.

Public Report Benefits

  • Achieves public accountability.
  • Results in universally heightened awareness of quality and refocuses attention to the balance between quality and cost: not just costs.
  • Ignites and drives quality improvement at a more rapid rate.
  • Provides information to enable purchasers and consumers to make more informed healthcare decisions.

Public Report Challenges

  • Heightens the level of anxiety among providers since market share will be impacted.
  • Requires that data are highly reliable.
  • Requires that data be adjusted for patient risk factors (out of the control of the provider) that could impact the outcome of care reported.
  • Challenges purchasers and consumers to appropriately interpret and use the information.
  • Requires that purchasers demonstrate responsible use of the information to make decisions and influence quality.
  • Requires a massive effort to educate and inform consumers regarding the use of information.
  • Requires supportive editorial policy and informed interpretation of information by the media.

There are valuable lessons to be learned from those who have pioneered the field of public disclosure of comparative performance reports. The next millennium will evolve new pioneers who have succeeded in applying the lessons of the past to develop the next generation of public reports.

About the Author

Dwain L. Harper is President of the Quality Information Management Corporation and Executive Director of the Cleveland Health Quality Choice Program.