The Alberta Health Services Human Factors (HF) team completes many quality improvement projects involving human participants and requires a robust and efficient ethics process. The team has developed an ethics process utilizing ARECCI (A pRoject Ethics Community Consensus Initiative), wherein HF specialists review their project for alignment with a reference project. The reference project captures a broad range of work that the HF team may lead or support in some way, and it has a corresponding series of countermeasures that have been created to address ethical risks. While some challenges remain, the process has largely allowed the team to meet its ethics goals.

The Alberta Health Services Human Factors Team

Alberta Health Services (AHS) is a province-wide health authority in Alberta. The AHS Human Factors (HF¹) team, which is part of the Provincial Patient Safety Department, works to create a safer healthcare system through the application of HF principles and methods. HF, as described by Sanders and McCormick (1993),

focuses on human beings and their interaction with products, equipment, facilities, procedures, and environments used in work and everyday living ... . Human factors, then, seeks to change the things people use and the environments in which they use these things to better match the capabilities, limitations, and needs of people. (p. 4)

The HF team receives requests to complete or support quality improvement (QI) projects from across AHS. The projects undertaken by the team typically fall into one or more of six domains. Domains include medication safety, workspace design, medical device evaluation, information design, human error prevention and electronic systems safety. Projects can vary considerably in scope and, relatedly, in the evaluation methods employed. While some work entails providing guidance on best practices or conducting expert reviews, other situations require collecting and analyzing human participant data.

As with healthcare QI projects more generally, collecting data from staff, physicians or patients poses ethical risks, which must be managed. In HF evaluations in particular, participants can be put into (simulated) situations where they may make errors, they may be interviewed about past experiences where adverse events or close calls have occurred or they may be observed while they do their work. Participants may feel that their knowledge and abilities are being assessed and, consequently, there is a risk that they may feel embarrassed about their performance and even fear retribution from management (Wickens et al. 2004).

QI Ethics

There is discussion in the literature about the distinction between QI and research as it pertains to ethics (e.g., Flaming et al. 2009; Stiegler and Tung 2017). Fiscella et al. (2015) contend that QI projects cannot always be easily distinguished, and suggest that the degrees of study rigour and generalizability are not effective for disentangling QI from research. Broadly, ethical risks can exist in both QI and research projects.

Published work is available describing healthcare QI ethics practices more generally. Flaming et al. (2018) propose a useful framework for guiding organizations through establishing a robust QI ethics process. In addition, Hunt et al. (2021) offer guidance for QI ethics. The present discussion does not intend to wade into the debate on ethics policy or provide a comprehensive ethics process guide. Rather, this paper will illustrate how the HF team uses a reference project ethics screening approach to manage most ethical risks.

A pRoject Ethics Community Consensus Initiative

The HF team's ethics process is based on ARECCI. ARECCI stands for “A pRoject Ethics Community Consensus Initiative” and was developed in 2003 to provide ethical guidance for projects that may not warrant a review from a research ethics board (Hagen et al. 2007). Although it has its origins in Alberta, the ARECCI screening tool is used by several organizations across Canada (Glavinovic et al. 2022).

The ARECCI screening tool (Alberta Innovates 2017) asks 35 yes/no questions. Questions one to three inquire about an explicit need for a research ethics board review, and questions 4–13 focus on whether the main purpose of the project is QI or research. Questions 14–35 help project leads determine where there are ethical risks in their project. A “yes” response for any of the ethical risk questions indicates that risk should be mitigated in the project by one or more countermeasures.² For projects with a score indicating “greater than minimal risk,” ARECCI can be consulted for an independent second opinion review.

Ethics Goals

Before the creation of the team ethics process, each HF project lead managed ethical risks differently, and this resulted in inconsistencies and inefficiencies. These challenges highlighted the need for a more robust approach to project ethics.

The HF team completes upwards of 100 projects annually, and many involve interacting with participants. Projects can vary considerably in size and evaluation methods used. An ethics review can be time-consuming as it requires preparing a request with a project description, supporting documents, a completed ethics screening and a description of the key ethical risks identified through the screening. Waiting for a response also adds to the lead time of the project, which could be a barrier to completing urgent work. While the team does not have time to complete a formal ethics review for each project, the safety and well-being of participants cannot be compromised.

The team's ethics goals are to ensure that ethical risks are managed effectively and that projects can be completed efficiently. If these two goals are met, participants are protected and the team can maximize the value provided to AHS. The traditional approach to ethics involves completing a formal ethics review for each project. The HF team, instead, compares new projects to a reference project, and a formal review is only requested when risks are not expected to be managed effectively with existing countermeasures. This alternative approach has been used by the team since 2018.

The Project Management Process

The HF team meets weekly to prioritize new requests. If a project is accepted, one HF specialist is assigned as a lead (other team members may be included as support depending on anticipated demands), and they begin the project management process with the following steps:

1. Schedule a kickoff meeting with project sponsor(s) to discuss the project's scope.
2. Develop a project charter.
3. Obtain signature(s) from project sponsor(s) on the project charter.
4. Complete ethics screening, if applicable.
5. Execute the project, including data collection with participants, if applicable.
6. Determine whether a change in scope is required. If so, return to Step 2. If not, proceed to Step 7.
7. Develop draft recommendations.
8. Review draft recommendations with project sponsor(s).
9. Finalize recommendations.
10. Complete project deliverables.
11. Share project deliverables with project sponsor(s).
12. Archive project materials.
13. Project sponsor(s) implement the recommendations.
14. Follow up on the implementation of recommendations, if needed.

Step 4 (Complete ethics screening, if applicable) is unnecessary for projects that do not involve participants. For instance, expert review evaluations would not require engaging participants. In contrast, projects involving interaction with participants require an ethics screening.

HF Project Ethics Screening

The ethics screening requires comparing the newly started project against a reference HF project, which is designed to encompass the range of work that the team could be involved in. The reference HF project is not an actual project but rather a set of responses to ARECCI questions, which define the boundaries of the team's typical work. To complete the screening, project leads review the ARECCI questions along with the associated reference HF responses. This reference indicates not only the yes/no answer but also the accompanying rationale to help project leads determine whether their project aligns with or deviates from the reference. For some questions, the reference HF response is “yes,” but countermeasures – described in more detail in the next section – are readily available to mitigate the associated ethical risk. An example of this is as follows:

ARECCI question 17: Does your project involve questions that collect information about sensitive issues, illegal behaviour, stigmatizing conditions or behaviours or religious or cultural beliefs or practices?

Reference HF response: Yes – HF projects may collect information on sensitive issues, such as past critical incidents and errors made, and information such as participant age or years of experience.

Applicable countermeasures include the following: Standard protocol, informed consent, sharing Employee and Family Assistance Program (EFAP) information, anonymous data collection, secure data storage and just culture.

Most HF team projects that involve participants align with the reference HF project and, therefore, require no subsequent independent review from ARECCI or a research ethics board. The project lead can use any applicable countermeasures already identified in the reference HF project and execute the project. If, however, there are deviations from the reference HF response, then project leads must complete an ARECCI screening for their project and apply appropriate countermeasures to address the risks. If the ARECCI score indicates “greater than minimal risk,” a second opinion review is required. This can be completed by submitting a request on the ARECCI website. An overview of the ethics screening process is shown in Figure 1.

Many of the core ethics principles outlined in ARECCI broadly apply to HF work. However, the reference HF response for each question is written for the HF project context. The countermeasures to ethical risks are largely domain agnostic, but they are accompanied by team-specific templates and resources that are tailored for HF work. Note that the countermeasures and their mappings to ARECCI questions (Table 1, available online here) are intended as a starting point, and countermeasures may be applied more broadly at the project lead's discretion.

Ethical Risk Countermeasures

Table 2, available online here, describes the ethical risk countermeasures used by the HF team. For each countermeasure, the team has access to additional information, and sometimes templates or other resources, to help manage ethical risk.

Project Examples

A recent HF project involved creating and testing a new patient-handling cognitive aid to support staff in selecting the most appropriate type of patient transfer. Patient-handling activities have been associated with staff injuries (Smedley et al. 1995). Relatedly, inadequate patient-handling practices can compromise patient safety (Elnitsky et al. 2014). The HF team devised a plan to test the usability of a proposed cognitive aid design. Usability is the extent to which a product, system or service is easy to use (Nielsen 1993). In healthcare contexts, usability issues in designs have been shown to contribute to inefficiencies, errors and ultimately patient harm (Lin et al. 1998; Zhang et al. 2005). This evaluation required participants (nurses, healthcare aids, occupational therapists and physical therapists) to complete three scenarios using the cognitive aid to determine the appropriate method of patient transfer.

Project scoping and the project charter were used to ensure that the ARECCI project purpose questions (4–13) aligned with the reference HF responses, based on which, this work was deemed to be primarily QI. For the ARECCI ethical risk questions (14–35), none of the deviations from the reference HF responses increased ethical risk. However, there were several “yes” responses, but each of them aligned with the reference HF responses.These included collecting information about sensitive issues (question 17), the use of tests/surveys (question 20), voice data (question 21), risks/burdens to participants (question 28) and potential to cause distress (question 29).

Beyond project scoping and the creation of a charter, several countermeasures were employed. Before data collection, a pilot test was used to work through the evaluation materials, and minor tweaks were made to the scenarios at this stage. A standard protocol was used to structure the evaluation, and verbal consent was captured from each participant. The evaluation was conducted by two HF specialists and satisfied the conditions described in the “experienced leads” and “trained observers” countermeasures (Table 2). In addition, the evaluation was conducted with the principles of a just culture in mind. Data were stored securely on network drives, and presentations of study findings did not include any identifying information. Although EFAP information could have been shared with participants, no participant distress was noted. Finally, because the evaluation sessions occurred with staff during regular work hours, compensation was deemed to be unnecessary.

Contrasting the example above, HF projects occasionally have deviations from the reference HF responses. In a study described by Altabbaa et al. (2019), the team sought to evaluate how cognitive biases influence diagnostic decision making. To investigate this, medical students and residents participated in simulations with clinical scenarios that were intended to create situations that were vulnerable to cognitive bias. Most elements of this study aligned with the reference HF responses, and the usual countermeasures were identified. However, there was concern about a potential power relationship (ARECCI question 16) between the participants and the project sponsor. Specifically, the physician sponsor who ran the simulation program was expected to take part in the facilitation of and data collection for the study. Consequently, an ARECCI second opinion review was requested. After the second opinion review, it was determined that several existing countermeasures would adequately address this ethical risk. These included working from a standard protocol and setting the stage for effective managed participation. In addition, the HF team oversaw the data analysis for the project; all the results were reported anonymously and in aggregate, and all the participants provided written informed consent.

Lessons Learned and Ongoing Challenges

The team's ethics process and tools have been developed over time and are effective in managing ethical risk in the vast majority of the HF projects. In the past five years, the HF team has completed 164 projects that involved participants and required an ethics screening, and 157 (96%) of these aligned with the reference HF project.

Some past challenges have also shaped the current approach. For example, concerns were raised regarding the remuneration for participants taking part in a usability test while off work. This case highlighted the need to clarify that the evaluation would take place during regular work hours, and consent form templates were updated accordingly. Broadly, the ethics tools cannot remain static, and countermeasures may be improved or new ones can be created as needs arise.

Despite efforts to maintain a robust ethics process, some ongoing challenges remain. For instance, although all the projects undertaken by the HF team have an assigned lead, the HF lead may play more of a consulting role on a broader project that is led by other groups. Depending on the project phase and the HF team's role, the HF lead may have limited influence over the ethics process. In addition, while the team has processes in place for managing changes in project scope, changes can be subtle at times, and the need to re-evaluate the ethical implications may not be readily apparent. Expanding the scope to include more participant roles in an evaluation, changing the debrief questions or adjusting the study design in other ways may introduce new risks.

Final Advice

The reference HF project was created by examining the types of projects the team has conducted and the HF evaluation methods employed. This reference was independently reviewed by ethics experts during an ARECCI second opinion review, which helped identify risks and inform the creation and mapping of countermeasures. Teams leading QI projects, HF or otherwise, can develop similar ethics processes according to their needs. Through a review of past projects and methods used and consultation with ethics experts, a reference project ethics screening with team-specific mapping of ethical risks and countermeasures (see Table 1) can be created to guide project leads. Put another way, the reference project should provide documentation for what is considered a typical QI project, and for work that falls within these boundaries, a prepared suite of countermeasures and associated templates and resources can be utilized. Although ARECCI provides a helpful framework, other frameworks may also be appropriate.

As previously alluded to, the application of the team's ethics process assumes some grounding in research ethics principles. While HF specialists frequently learn about research ethics as part of their education, others who lead healthcare QI projects may not have had the same degree of exposure to key human participant research ethics concepts. For those who lack formal education, investing in ethics training (e.g., ARECCI training) would be recommended to ensure that ethical risks are properly managed.

Conclusion

The importance of managing ethical risk in any project involving participants is evident (Fiscella et al. 2015; Flaming et al. 2009; Stiegler and Tung 2017). In HF projects, specifically, participants may feel that their performance is being scrutinized, and steps should be taken to manage this (Wickens et al. 2004). The AHS HF team has developed a QI project ethics process that builds off ARECCI, and this has allowed the team to complete projects efficiently without compromising participant safety. This ethics process has evolved over time and will continue to do so in response to emerging challenges.

Comments

Be the first to comment on this!