We describe our experience with a Nursing Usability Laboratory, where human factors design principles were applied to common nursing procedures. Our first step was to develop a human factors usability checklist. We then used this checklist while observing 11 nurses completing two standardized tasks on a simulated patient: (1) programming an insulin infusion and (2) programming a heparin infusion. We found that a usability checklist can help to uncover systematic error-provoking conditions in nursing tasks, that immediate improvements can be made in nursing training and practice and that participant nurses found the process useful. This paper will be of interest to any hospital seeking to enhance safety by applying human factors design principles.

Background

A central concept in patient safety is to build a system of care that reduces the possibility for human error and prevents human error from causing patient harm. "Human factors" is the scientific discipline concerned with designing systems to meet the needs, limits and capabilities of the people who work in them. The application of human factors science is a logical step in the patient safety movement.

One important aspect of human factors science is the usability of medical devices. Human factors principles can be used in the design and development of medical devices, and in the ongoing evaluation of devices while in use. Usability testing can identify unanticipated design problems with a relatively small sample of users (Kushniruk et al. 2004).

There are relatively few published examples of usability testing of medical devices (Making Health Care Safer 2001; Chiu et al. 2004; Nunnally et al. 2004). Human factors principles reduce programming errors with patient-controlled analgesia pumps, and improve efficiency (Lin et al. 1998; Gosbee 2004). Although usability should be a prime consideration in medical device selection, the process of device procurement may overlook vital usability considerations (Keselman et al. 2003). Fortunately, a growing number of institutions have adopted usability principles for device procurement and training (Gosbee 2004).

At Sunnybrook, our safety program is focused on three core areas: culture, reliable design and measurement. Our work on reliable design began by inviting two human factors consultants to provide advice. We developed a Human Factors course for staff that has been very popular and highly rated. This paper describes our initial experience in applying human factors usability principles to medical device use in a "Usability Laboratory." Our overarching goals were

1. To develop a usability assessment method
2. To apply this usability assessment method to existing clinical tasks
3. To build expertise and teamwork in the assessment of device usability

Our specific objectives for this initial pilot study were

1. To develop a human factors usability checklist
2. To develop a procedure checklist for two common nursing tasks: programming a heparin infusion, and programming an insulin infusion
3. To identify usability problems with our current infusion device while carrying out these nursing tasks

Project Description

We selected two common nursing tasks for evaluation: programming a heparin infusion and programming an insulin infusion. Nurses regularly carry out these tasks in any acute care ward, critical care area or emergency room. Heparin infusions are based on preprinted standard orders and protocols, whereas insulin infusions are less standardized. On the acute care wards, and in the emergency room, there is no standardized insulin infusion protocol. In the critical care units, there is a preprinted insulin infusion protocol, but the physician can still write any range of infusion parameters.

We also developed a generic usability checklist based on human factors principles (Gosbee 2004; Gosbee and Lin 2001) and advice from our human factors consultants (see acknowledgments). We used the checklist during the observation and debriefing sessions. (See Table 1 for selected features of the checklist.)

The nurse educators developed a checklist for each procedure. An excerpt from one of these checklists is found in Table 2. We focused solely on the programming steps. We did not address issues related to drug selection, intravenous admixture, labelling of infusions, or infusion line setup.

Each nurse educator recruited a convenience sample of volunteer nurse participants from their units, and each participant received a briefing by the Patient Safety Service team. We emphasized that our goals were to identify and understand problems related to infusion pump use. We explained that the session was not a performance evaluation, and that all information was to be used to improve the system of care. We also emphasized that the nurses' experiences using the devices were essential to the success of the project, so that we would debrief after the session.

The volunteer nurse then entered the simulation lab (Figure 1). Two intravenous infusions were already set up on the simulated patient, and there were two infusion devices available. Two observers took notes: a patient safety specialist and a nurse educator; they both used the procedure checklist while observing the nurse. The patient safety specialists also had the usability checklist available, but they both found it more useful during the debriefing session. Each participant completed the heparin programming task, followed by the insulin programming task. After the first few volunteers, we gave participants the option of calling a "time out." Participants said that, in usual practice, if they ran into difficulties they would take a "time out" and seek advice from a colleague.

After the two tasks were completed, the observers debriefed with the nurse participant, guided by the questions on the usability checklist. The biomedical engineer downloaded all of the programming steps onto a laptop for subsequent review. Later, the patient safety specialist (EE) and the biomedical engineering technician (RB) reviewed the programming steps and the field notes to identify recurrent patterns of error. The programming steps combined with the field notes were used to draw preliminary conclusions. The study team then reviewed and endorsed the conclusions.

Results

We recruited a total of 11 nurses, including eight from acute care and two from critical care or coronary care units (one was not recorded). Seven nurses reported greater than two years' experience in their current role.

[Table 3]

We identified several usability problems (Table 3), including incorrect programming of infusion parameters, difficulty recovering from these programming errors, inability to clear previous program settings and inconsistencies between infusion rate displayed on correctly programmed device and the preprinted order sheet.

Many nurses experienced difficulty with the programming steps. First, there was an inconsistency between the order of parameters on the preprinted orders and the order of parameters requested by the machine. The machine requested in this sequence: desired rate, total dose, total volume, and remaining volume in the bag. By contrast, written orders usually listed them in this sequence: total dose, total volume, and desired rate.

In addition, the messages displayed during the programming sequence were confusing or ambiguous. For example, the machine displayed a message "CONC" twice, but each time a different parameter was needed. With the first CONC prompt, the nurse had to enter the total drug dose. With the second, the nurse had to enter the total original volume on the infusion bag. There were differences between the two prompts, but these were easily overlooked. The next prompt was for volume, but it was asking for the remaining volume in the bag, not the initial volume in the bag.

Programming errors were not visible until the final infusion rate was calculated and displayed by the device. In most cases, the nurse recognized that an error had been made somewhere, because the rate was unusually high or low. In one case, the patient would have received a significant overdose.

We observed difficulty in recovering from these programming errors. First, it was not clear where the error occurred, because there was no easy method for reviewing the programmed settings. Nurses tended to restart the entire programming sequence, because it was not possible to easily detect which step was incorrect. Some nurses attempted to clear all previous settings, but encountered difficulty. The "Clear" button did not actually clear previous settings. Similarly, turning off the device did not clear the settings. The necessary step was to choose the option code, then enter 99, but not all nurses were aware of this.

Finally, we identified problems caused by inconsistencies between the device display and the written orders. Specifically, the machine displayed the calculated rates to one decimal place, whereas the preprinted orders round up or down to the nearest unit. The nurse might expect a rate of 24 units per hour, yet see a programmed rate of 23.6 units per hour. This occasionally led to delays while the nurse decided whether the programming was correct.

Some errors were caused by the study itself. In two cases, the participants chose an unusual programming option that calculated the rate in units per kilogram per hour. This led to additional programming steps and unusual rate displays. The participants said that they would not normally choose this option, but did so because they were being observed.

We gained additional usability problems and insights during the debriefing sessions. The major usability problem was recognition and recovery from errors. Many nurses felt that the machine did not help them recognize errors; one said "You have to rely on yourself."

Some nurses stated that the programming options were so confusing that they never used them. Instead, they used the simplest set of steps to program the rate (displayed in millilitres per hour). This workaround, while it eliminated the need to program the dose of medication or the original volume in the intravenous solution bag, might lead to problems in the clinical setting. If a nurse were expecting the machine to be programmed in units per hour, the rate might be adjusted incorrectly (e.g., the rate is increased by 10 millilitres per hour instead of 10 units per hour).

Many nurses expressed particular difficulty with certain error signals ("FIX 21" and "FIX 24"). Although none of these signals arose during the study, it was clear that they were a source of great concern to the nurses. We learned that one of these signals led to immediate disabling of the machine, and only the biomedical engineering department could enable it again for clinical use. Participants described significant frustration trying to get such machines working again, without knowing that nothing could be done to override the signal.

The participants all stated that the experience was informative and positive. The nurse educators also found the experience very instructive. They said it would change the way they carried out their orientation and training in the future.

Discussion

In this small pilot study, we identified several usability problems with our current infusion procedures. In the short term, we will use this information to modify our training procedures and the design of our preprinted orders. In the long term, we will provide feedback to the manufacturer to improve future designs, and we will use usability testing to guide procurement of new medical devices.

Prior studies have identified important usability problems with infusion devices. Lin et al. (1998) identified several usability problems with a patient-controlled analgesia device, including difficulty in viewing settings, difficulty recognizing and recovering from error, misleading and confusing displays and misleading or confusing labels. Simple redesigns of the user interface reduced errors, reduced mental workload and increased programming efficiency. Automation can paradoxically lead to new errors. For example, an automated patient identification bar coding device may create opportunities for error in patient identification and medication administration (Patterson et al. 2002). Similarly, we found that using the "advanced" programming options paradoxically increased the likelihood of programming error.

Our small pilot study has several limitations. We studied a small number of volunteers from our institution, so the generalizability of our results is limited. We have begun to address this limitation by presenting our results to local hospital nursing committees. So far, the response has generally been endorsement and enthusiasm. Another limitation is that some nurses said they changed their usual pattern of work because they were being observed, which led them to make mistakes they would not otherwise have made.

We conducted our project in a high-fidelity simulator using a simulated patient. However, the key learning was generated from observing the participants' use of the device. We believe that any institution can conduct a similar project in a low-fidelity setting (i.e., any room); the only necessary components would be volunteer nurses, observers, an infusion device and some checklists.

Our preliminary study suggests an exciting role for human factors in the development of hospital procedures and training programs. Nursing staff and educators found the method to be instructive and constructive. The project generated recommendations and changes in infusion device training, infusion device procurement and standardized order design.

One of the new CCHSA Required Organizational Practices (Canadian Council on Health Services Accreditation 2005) is to provide ongoing, effective training for service providers on all infusion pumps. Our approach may be useful to other Canadian institutions as they address this required practice. Our results suggest that observation of simulated or actual use, and attention to human factors usability principles, will be needed to maximize the safety benefit of infusion device training.

In summary, we applied human factors usability principles to two common infusion device tasks and identified several usability problems. We have begun to modify our training programs and preprinted orders to address these problems, and we plan to provide feedback to the device manufacturer. This method can be easily adopted by any healthcare organization to enhance the safety of intravenous infusions.

Comments

Be the first to comment on this!