Abstract

Objective: The Minimizing Errors Maximizing Outcomes Study is designed to examine the effect of workplace conditions on quality of care and medical errors. In the first phase of the study, patients were asked to "tell their stories" via focus groups.

Design: Moderators used a standard question guide. Researchers read the transcripts independently and reached consensus on major themes. Two coders independently assigned transcript statement to themes.

Setting: Three focus groups were conducted in three cities, including 21 patients from three clinics.

Patients: Patients with previously scheduled appointments at participating clinics were invited to join the focus groups.

Measurements and Main Results: Agreement between the two coders was 77.5% (kappa value 0.66). All but 2% of 187 distinct comments could be grouped into four categories: (1) Systems Issues (44% of comments). Long waits for providers and lack of access were the most common frustrations. Understaffing, underfunding and lack of health insurance were perceived as contributing to poor quality of care; (2) Interpersonal Skills (37%). Physician listening skills were valued. Participants felt patient attitudes affected care. (3) Knowledge and Technical Skills (9%). (4) Errors (7%). Medication errors, errors of inattention and technical errors were discussed.

Conclusions: Patients provide important insights into complex systems issues, which can guide planners in improving quality and reducing errors. According to focus group participants, healthcare could be improved and made safer by increasing timely access to patients' own physicians, decreasing the time patients spend in waiting rooms, and adding staff to double-check prescriptions.

Introduction

Medical errors kill more people in the United States than automotive accidents, breast cancer, or AIDS (Centers for Disease Control and Prevention 1998). According to the Institute of Medicine report, at least 44,000 (and perhaps as many as 98,000) unnecessary deaths and 1,000,000 excess injuries occur each year due to medical errors (Kohn et al. 2000). A national survey conducted by Blendon and colleagues found that 35% of physicians and 42% of adults have experienced a medical error, either personally or through a friend or relative (Blendon et al. 2002). Improving healthcare quality, including reduction of errors, has become an urgent national priority (Kohn et al. 2000; National Patient Safety Foundation 1997; President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry 1998; US Department of Health and Human Services 2000; Hurtado et al. 2001).

Traditionally, research on medical errors and their impact on care has focused on the inpatient setting where injuries, such as medication and surgical errors, are easy to define and count (Weingart et al. 2000). Errors, however, are also common in outpatient settings (Gandhi et al. 2003; Gurwitz et al. 2003; Fischer et al. 1997). These errors are often difficult to define and their impact on care is hard to measure. Examples include delayed diagnoses, lack of timely laboratory test follow-up, inattention to risk, non-adherence to published guidelines and inadequate pain management.

Patient feedback may be one method to identify the type, frequency and impact of errors in the outpatient setting and their relationship to perceived quality of care. The use of open-ended questions invites participants to share experiences, opinions and suggestions, which can uncover previously unconsidered concerns (Schwarz et al. 2000). This paper reports qualitative results from the first phase of MEMO (Minimizing Error, Maximizing Outcome), a project funded by the Agency for Healthcare Research and Quality to examine the effect of healthcare work conditions on quality of care and medical errors. In this paper, outpatients identify the workplace and personnel characteristics they believe contribute to high quality care and those they associate with substandard care and errors.

Methods

Subject Recruitment

The participants were drawn from three urban research sites participating in MEMO. These practices have a diverse patient base and feature a range of payers including commercial feefor- service plans, managed care plans, Medicare, Medicaid and significant numbers of indigent or uninsured patients. Using a combination of telephone and waiting room recruitment, all patients with a scheduled appointment were invited to participate in a 60-minute discussion of healthcare quality. Patients were invited sequentially as they appeared in person or on the clinic schedule. All participants provided informed consent and were paid $10.

Focus Group Protocol

The MEMO regional Site Director and the region's research assistant moderated each focus group. Each site director possessed experience in the conduct of focus groups. The moderators followed a standard interview guide centred on features of good care developed from a thorough review of the literature and the conceptual model underlying the MEMO study (see Williams et al. 2002). The Institutional Review Board at each site approved all consent forms and interview guide. The sessions were audiotaped and transcribed. Speakers' identities and references to specific clinics were deleted prior to analysis.

Transcript Analysis

The analysis is based on the grounded theory method suggested in Krueger (1994). In this process, analysts read and reread the transcripts until a comprehensive set of themes (and subthemes) emerges. Four of the authors independently analyzed the transcripts using this methodology. After considerable discussion, a consensus on four major themes and 13 sub-themes (see Table 2) emerged. The analysis continued with two of the authors (an internist and epidemiologist) independently assigning each of the transcript statements to one of the themes and sub-themes. After this initial coding, the two authors compared their assessments, identified and reconciled differences and completed a final coding of all transcript statements. Using this coding, frequencies were calculated for each of the major themes. Inter-rater reliability was assessed by evaluating the percent agreement between the two raters for each theme. Between-site differences were assessed using a chi-square test.

Results

A total of 21 patients participated in the three focus groups (seven in each group), including 14 women and 7 men. Twelve subjects were African American, five Latino and four Caucasian. Ages ranged from 25 to 75 with a median age of 55 years.

Together, the three focus groups generated 187 discrete non-process statements. The overall agreement between the two raters assigning statements to themes was 77.% (kappa value 0.66).

All but 2% of the comments could be grouped within the four themes: Systems Issues (83 comments, 44%), Interpersonal Skills (70 comments, 37%), Knowledge and Technical Skills (17 comments, 9%) and Errors (13 comments, 7%). There were no significant differences in the proportion of comments on each theme across the three groups (chi-square = 7.9; df = 8, p = 0.44).

Systems Issues

Time

Multiple comments revealed frustration with waiting in several settings: physicians' offices, pharmacies, emergency rooms and appointment scheduling. Patients speculated waits were due to understaffing. Despite their frustration, patients displayed an awareness of the complex factors involved in scheduling and providing services on time. One patient expressed the problems inherent in scheduling different patients with unique problems and concerns for uniform amounts of time, stating, "I know the doctors can get crowded and they take a little more time with one patient than they do the other." Another patient recognized the difficulties physicians face under time pressure:
It's double standard, because…we want to get out of here quick and fast. But if…they want to get out of here quick and try to speed it along, we get upset because they are trying to push us along. It is a double standard type of thing. But I'm always in a rush to get in and out of here.

Access

Patients described substantial unmet needs for appointment access. Patients described long waits for regular visits, reporting, "I called in November and couldn't get in until January" and "…if something happens you are lost….you have to wait six months or three months. You just don't know what to do." Patients also reported difficulty accessing providers for urgent problems. Some were told to go to the emergency room if they needed to be seen that day. Patient loyalty to primary care physicians and rigid appointment scheduling can sometimes lead to "near misses" as one patient reported:
One time I came in because I was so sick. They said you couldn't see her because she has an appointment with another patient. I was so sick I didn't know my head from my toe. I said, 'Well look, I'm not leaving until I see her.' They said, 'We'll see what we can do.' So then they must have went and told her and she came in. She just looked at me, and she said, 'Oh, oh, you had a stroke.' I feel that I should see my primary care doctor because she knows me, and everything would be all right.

Some patients described difficulty contacting physicians between appointments. One reported

I went for a mammogram. I got a letter telling me to come back or call ASAP. I called and never could get her. I was nervous….I think she should have told me in person. I was about to die from nervousness and waiting.

Others were satisfied with contact, stating

Every time that I call my doctor I may not talk to her right at that time, but the message does get to her. If she doesn't call me then her nurse will call me, and I will get some feedback one way or the other. That I like.

Patients speculated that access was difficult because doctors were "under staffed" and "overloaded."

Coordination of Care

Patients appreciated communication between staff, providers and patients in arranging regular visits, coordinating specialty care and managing outpatient follow-up. Common frustrations included lack of communication about scheduling changes. One patient wondered, "sometimes he has emergencies and when I get there to see him they say he got called to the hospital or whatever. Why didn't they call me as soon as they knew he wouldn't be in?"

System Resources

Patients perceived problems related to understaffing and underfunding in clinics, outpatient pharmacies and services such as health education and translation. They noted pharmacies would benefit from more personnel to distribute medications.

Insurance and Payment Issues

Patients reported differences in quality of care based on insurance coverage and ability to pay, stating, "if people can pay you get a lot better care" and "a lot of time people still have to go without medications that would really help them because they can't pay for them. If you can't pay for it, it will not help you."

Interpersonal Skills

Physicians

Physician interpersonal skills received more comments (38) than any other sub-theme. Patients appreciated a "positive attitude" and valued good listening skills. They thought asking questions about what a patient had said, demonstrating knowledge of the patient, attending to complaints and simply being willing to hear patients' problems were markers of a physician's ability to listen. Patients recognized physicians do not have unlimited time for visits and appreciated any indication that a physician had expended extra time or effort for them. Patients also emphasized the importance of education. They identified use of specific skills such as partnering, goalsetting and encouragement.
I'm a smoker. I used to smoke a pack and a half a day. I'm down to four cigarettes now. It took me two years to get to this stage. Every time I come in, they say well that's very good. Let's try for one less cigarette next time. She's always encouraging me.

Other Staff

Most patients recognized nurses as an integral part of healthcare teams and appreciated their work. Subjects were less satisfied with other personnel, complaining about staff who don't answer questions when asked or one who
didn't even ask me what my name was. I told her who I was. She didn't even talk to me. I just stopped going. I thought they were supposed to greet someone with a warm greeting. That is not the kind of attitude that I would expect from anyone giving care.

The most important issues were recognition, personal assistance and respect. Several patients were pleased when greeted by name. Another reported that assistance locating a clinic made her feel like family. Subjects regarded mutual respect between staff and patients as part of good care, reporting, "I love the hospital. I feel very good with the treatment that they give us. …We respect each other." Some patients were aware of the daily issues facing staff and expressed sympathy, wondering, "how up would I be at the end of the day if I was faced with these things? It takes a lot. There are patients who are cranky and mad. They think they are the only ones to be contended with."

Patients

In all groups, patients made references to their own use of interpersonal skills in managing healthcare encounters. They believed their attitudes affect physicians and staff, reporting, "if you go into the exam room with a good attitude, then that will rub off on them….I come in full of smiles." Patients also felt their attitudes affected their health outcomes, advising, "it pays to be good to your doctor. It helps your healing." Patients felt empowered to shape their healthcare environments, stating
We make the doctors and the doctors make us. If I come in late every time, or don't come in every time, the doctor says you're sick why don't you come? If you're sick why don't you get here on time? If we respect the doctor, then they will respect us.

Patients described other actions they take to improve their care. One patient whose spouse doesn't speak English said, "we schedule our appointments together to help each other understand what is going on." Others thought bringing family members to a visit improved care.

Seems like you get better care if people in your family, like maybe your husband, wife or kids come with you to the doctor. They should talk to the doctor too. It seems like the doctor does more for you if you have a family member with you.

Knowledge and Technical Skills

Physicians

Patients valued thorough, efficient doctors who go "straight to the problem." They believed physicians should have knowledge of procedures and medical specialties. Patients were concerned that physicians' multiple responsibilities could interfere with competence, stating, "some. …wear too many hats. When you wear too many hats, you cannot become proficient."

Other Staff

When discussing nurses and staff, patients focused on skills rather than knowledge. Patients wanted continuous monitoring, careful blood pressure measurement and skillful phlebotomy. Patients indicated they evaluated skills to a greater extent than staff realized:
I have seen incidences where they've taken blood pressure and just let it go back down to zero. … what I know about taking blood pressure is you have to do it at a reasonable speed. So they're guessing at what the actual blood pressure is perhaps. That may seem insignificant to them, but to the patient, it's what's going on here? I guess they imagine that you don't know anything about blood pressure, so you wouldn't notice it.

Errors

Patients discussed three general categories of medical errors: errors of inattention, medication errors and technical errors. Patient reactions to errors included tolerance, believing, "Well everybody does mistakes. If you go to the other hospitals you might find the same mistake. In general they do a good job. They try their best." Patients suggested changes to catch errors, such as "if they would doublecheck before they leave the [pharmacy] window it would solve a lot of problems."

Medication Errors

Five comments described medication mistakes. Some received the wrong medications or incomplete medications, for example: "It just wasn't mine. Someone else's medicine was mixed in with mine" and "I have gotten home and noticed that all of my medicine is not in my bag. I went the next day and they didn't have any to give me." One patient reported not being told a medication had been taken off the market.

Errors of Inattention

Six patients reported errors related to lapses in attention. These included repeatedly attempting to take blood pressure on the right arm of a chemotherapy patient who was not supposed to have her right arm used and discharging a patient who had developed an undiagnosed infection in the hospital.

Technical Errors

Two comments described errors involving procedures, including a patient who was told a clamp was lost during her c-section.

Discussion

The publication of Through the Patient's Eyes (Gerteis et al. 1993) signalled the growing influence of the patient in the patient-physician relationship. Patient satisfaction and now patient safety are topics of discussion and concern from the board room to the operating room. One finding of this study - that patients are astute observers of healthcare systems and sophisticated participants in healthcare interactions - reminds us that patients are far from being passive recipients of medical care. Patients spontaneously identified several complex systems issues including time issues, coordination of care, system resources and the effects of insurance and financial status on services received. Some of these issues, notably waiting time (Concato and Feinstein 1997; Kravitz et al. 1996; Dansky and Miles 1997; Huang 1994; Probst et al. 1997) and access to care (Murphy et al. 2001; Flocke 1997; Harris et al. 1999; O'Malley et al. 2000) have previously been linked to patient satisfaction. Insurance status has been shown to affect quality of care: insured patients are able to obtain better primary care than the uninsured, and privately insured patients are able to obtain better care than the publicly insured (Shi 2000). While patients expressed frustration with systems issues, they also showed understanding of the complex factors that create frustrating conditions.

A second finding confirmed the importance of physician and staff interpersonal interactions with patients. Participants identified use of specific skills in these areas such as partnering, encouragement and recognition of patients' emotional states. These concerns have been explored extensively in the medical literature (Murphy et al. 2001; Flocke 1997; Flocke et al. 2002; Wensing et al. 1998; Concato and Feinstein 1997; O'Malley et al. 2000). A new concept that emerged from our focus groups was patients' awareness of their own use of interpersonal techniques in managing healthcare encounters. Many patients described how statements expressing respect for healthcare workers or actions, such as bringing family members to an appointment, could improve their care. Future research might explore if patients' use of specific relationship skills affects the quality of care they receive.

The third finding was that comments regarding medical errors and technical mistakes emerged without specific prompting. This supports our earlier finding that patients are sophisticated observers. Interestingly, these observers most frequently attributed errors to inattention and not incompetence. This suggests a substantial role for patient queries in the drive to increase patient safety. Further research should investigate how patients might contribute to the safety of their own care.

An obvious limitation of our study is the small number of patient participants. However, our sample did consist of a diverse group of African- American, Latino and Caucasian patients. Another limitation is that patients were recruited from among those attending appointments and may not represent those who obtain care from emergency rooms or other settings or who have limited access to care. Finally, the focus groups were conducted in English and may not represent the concerns of patients with limited English proficiency. Our findings may be viewed as a starting point for further research and confirmation with other populations.

About the Author

Deborah Dowell, MD, Division of General Inernal Medicine, Department of Medicine, New York University School of Medicine.

Linda Baier Manwell, MS, Section of General Internal Medicine, Department of Medicine, University of Wisconsin- Madison Medical School.

Ann Maguire, MD, MPH, Division of General Internal Medicine, Department of Medicine.

Perry G. An, BA, New York University School of Medicine .

Laura Paluch, BA, Center for Urban Population Health, Aurora Sinai Medical Center.

Karla Felix, BS, Department of Medicine, VA New York Harbor Health Care System, Medical Service (111).

Eric Williams, PhD, Department of Management and Marketing, Culverhouse College of Commerce and Business Administration, University of Alabama.

Acknowledgment

Funding for this project was provided by the Agency for Healthcare Research and Quality, grant no. RO1 HS11955. This paper was presented in abstract form at both the Society of General Internal Medicine 2003 Midwest Regional Meeting and the Society of General Internal Medicine 2004 Annual Meeting. The authors would like to thank the focus group participants for their insights and the site directors, research assistants and MEMO collaborators who worked so hard to design and facilitate the focus group sessions.

Deborah Dowell, Linda Baier Manwell, Ann Maguire, Perry G. An, Laura Paluch, Karla Felix and Eric Williams for the MEMO Investigators*

Please send correspondence to: Deborah Dowell, MD, Division of General Internal Medicine, Department of Medicine, New York University School of Medicine, 227 Madison Street, New York, NY 10002; Telephone: 212-238-7478; Fax: 212-238- 7668; email: drd1@nyu.edu

* MEMO Investigators: Mark Linzer, M.D., Chief, Section of General Internal Medicine, University of Wisconsin; Elizabeth Arce, Division of General Medicine & Primary Care, Cook County Hospital; Stewart Babbott, M.D., Baystate Medical Center; JudyAnn Bigby, M.D., Brigham & Women's Hospital; James Bobula, Ph.D., University of Wisconsin, Department of Family Medicine; John Frey, M.D., Department of Family Medicine, University of Wisconsin; Jessica Grettie, Department of Family Medicine, University of Wisconsin; Barbara Horner-Ibler, M.D., Department of Internal Medicine, Milwaukee Clinical Campus, University of Wisconsin; Robert Konrad, Ph.D., Sheps Center for Health Services Research, University of North Carolina; Peggy Leatt, Ph.D. Department of Health Policy & Administration, University of North Carolina; Bernice Man, M.D., Division of General Medicine & Primary Care, Cook County Hospital; Julia McMurray, M.D., Section of General Internal Medicine, University of Wisconsin; Marlon Mundt, M.A., M.S., Department of Family Medicine, University of Wisconsin; Mary Beth Plane, Ph.D., Department of Family Medicine, University of Wisconsin; Joseph Rabatin, M.D., Bellevue Hospital; Elianne Riska, Ph.D., Department of Sociology, Abo Akademi University, Finland; William Scheckler, M.D., Department of Family Medicine, University of Wisconsin; Mark Schwartz, M.D., Department of Medicine, VA New York Harbor Health Care System; Jessica Sherrieb, Section of General Internal Medicine, University of Wisconsin; Anita Varkey, M.D., Division of General Medicine & Primary Care, Cook County Hospital.

References

Blankfield, R.P., M.Goodwin, C.R. Joen and K.C Stange, 2002. "Addressing the Unique Challenges of Inner-City Practice: A Direct Observation Study of Inner-City, Rural, and Suburban Family Practices." Journal of Urban Health 79(2): 173-85.

Blendon, R.J., C.M. DesRoches, M. Brodie, J.M. Benson, A.B. Rosen, E. Schneider, D.E. Altman, K. Zapert, M.J. Herrmann and A.E. Steffenson. 2002. "Views of Practicing Physicians and the Public on Medical Errors." New England Journal of Medicine 347(24): 1933-40.

Centers for Disease Control and Prevention. 1999. "Births and Deaths: Preliminary Data for 1998." National Vital Statistics Reports 47(25): 6.

Concato, J. and A.R. Feinstein. 1997. "Asking Patients What They Like: Overlooked Attributes of Patient Satisfaction with Primary Care." American Journal of Medicine 102(4): 399-406.

Dansky, K.H. and J. Miles. 1997. "Patient Satisfaction with Ambulatory Healthcare Services: Waiting Time and Filling Time." Hospital & Health Services Administration 42(2): 165-77.

Fischer, G., M.D. Fetters, A.P. Munro and E.B. Goldman. 1997. "Adverse Events in Primary Care Identified from a Risk-Management Database." Journal of Family Practice 45(1): 40-46.

Flocke, S.A. 1997. "Measuring Attributes of Primary Care: Development of a New Instrument." Journal of Family Practice 45(1): 64-74.

Flocke, S.A., W.L. Miller and B.F. Crabtree. 2002. "Relationships Between Physician Practice Style, Patient Satisfaction, and Attributes of Primary Care." Journal of Family Practice 51(10): 835-40.

Gandhi, T.K., S.N. Weingart, J. Borus, A.C. Seger, J. Peterson, E. Burdick, D.L. Seger, K. Shu, F. Federico, L.L. Leape and D.W. Bates. 2003. "Adverse Drug Events in Ambulatory Care." New England Journal of Medicine 348(16): 1556-64.

Gerteis, M.S., S. Edgman-Levitan, J. Daley, T.L. Delbanco. 1993. Through the Patient's Eyes. San Francisco, CA: Jossey-Bass Publishers.

Gurwitz, J.H., T.S. Field, L.R. Harrold, J. Rothschild, K. Debellis, A.C. Seger, C. Cadoret, L.S. Fish, L. Garber, M. Kelleher and D.W. Bates. 2003. "Incidence and Preventability of Adverse Drug Events Among Older Persons in the Ambulatory Setting." JAMA 289(9): 1107-16.

Harris, L.E., R.W. Swindle, S.M. Mungai, M. Weinberger and W.M. Tierney. 1999. "Measuring Patient Satisfaction for Quality Improvement." Medical Care 37(12): 1207-13.

Huang, X.M. 1994. "Patient Attitude Towards Waiting in an Outpatient Clinic and Its Applications." Health Services Management Research 7(1): 2-8.

Hurtado, M., E. Swift and J.M. Corrigan, eds. 2001. Envisioning National Health Care Quality Report. Washington, DC: National Academy of Sciences.

Kohn, L.T., J.M. Corrigan and M.S. Donaldson, eds. 2000. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press.

Kravitz, R.L., E.J. Callahan, D. Paterniti, D. Antonius, M. Dunham and C.E. Lewis. 1996. "Prevalence and Sources of Patients' Unmet Expectations for Care." Annals of Internal Medicine 125(9): 730-37.

Krueger, R.A. 1994. Focus Groups: A Practical Guide for Applied Research (2nd ed.). Thousands Oaks, CA: Sage.

Murphy, J., H. Chang, J.E. Montgomery, W.H. Rogers and D.G. Safran. 2001. "The Quality of Physician-Patient Relationships. Patients' Experiences 1996-1999." Journal of Family Practice 50(2): 123-29.

National Patient Safety Foundation at the AMA. Prepared by Louis Harris & Associates. Public Opinion of Patient Safety Issues Research Findings. September 1997. McLean, VA: Author.

O'Malley, A.S., C.B. Forrest and P.G. O'Malley. 2000. "Low-Income Women's Priorities for Primary Care: A Qualitative Study." Journal of Family Practice 49(2): 141-46.

President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry. 1998. Quality First: Better Health Care for All Americans-Final Report to the President of the United States. Washington, DC: US Government Printing Office.

Probst, J.C., D.L. Greenhouse and A.W. Selassie. 1997. "Patient and Physician Satisfaction with an Outpatient Care Visit." Journal of Family Practice 45(5): 418-25.

Schwarz, M., S.E. Landis, J.E. Rowe, C.L. Janes and N. Pullman. 2000. "Using Focus Groups to Assess Primary Care Patients' Satisfaction." Evaluation & the Health Professions 23(1): 58-71.

Shi, L. 2000. "Type of Health Insurance and the Quality of Primary Care Experience." American Journal of Public Health 90(12): 1848-55.

Street, R.L., E. Krupat, R.A. Bell, R.L. Kravitz and P. Haidet. 2003. "Beliefs about Control in the Patient-Physician Relationship: Effect on Communication in Medical Encounters." Journal of General Internal Medicine 18(8): 609-16.

US Department of Health and Human Services. 2000. The Challenge and Potential for Assuring Quality of Health Care for the 21st Century. Washington, DC: Author.

Weingart, S.N., R.M. Wilson, R.W. Gibberd and B. Harrison. 2000. "Epidemiology of Medical Error." British Medical Journal 320(7237): 774-77.

Wensing, M., H.P. Jung, J. Mainz, F. Olesen and R. Grol. 1998. "A Systematic Review of the Literature on Patient Priorities for General Practice Care. Part 1: Description of the Research Domain." Social Science & Medicine 47(10): 1573-88.

Williams, E.S., T.R. Konrad, M. Linzer, J.E. McMurray, D.E. Pathman, M. Gerrity, M.D. Schwartz, W.E. Scheckler and J. Douglas for the SGIM Career Satisfaction Study Group. 2002. "Physician, Practice, and Patient Characteristics Related to Primary Care Physician Physical and Mental Health: Results from the Physician Worklife Survey." Health Services Research 37(1): 121-43.

Yarnall, K.S., K.I. Pollak, T. Ostbye, K.M. Krause and J.L. Michener. 2003. "Primary Care: Is There Enough Time for Prevention?" American Journal of Public Health 93(4): 635-41.