There is growing recognition of the importance of patient, public and community engagement in health research, which has not been used widely in analyzing health administrative datasets. In Ontario, health data are stewarded by ICES, whose strategic decision making is guided by a diverse Public Advisory Council (PAC). In a first foray into publicly led projects, the ICES PAC undertook an analysis project on mental health and addiction health service use. Public members guided the project through all stages of research. This generated critical lessons for ICES on improving participation, collaboration and trust.

Conceiving a Public-Led Analysis Project

In Ontario, almost every interaction with the publicly funded healthcare system – physician visits, diagnostic procedures, emergency department visits and more – generates administrative data (“big data”). These data are securely held and stewarded by ICES – a not-for-profit research institute permitted under Ontario privacy legislation to use data to advance research and evaluate Ontario's healthcare system (ICES 2023a). Most research at ICES is scientist-initiated, but ICES also supports analyses requested by health system stakeholders (such as hospitals or health regions) as defined under the Ministry of Health's Applied Health Research Question (AHRQ) program (Ontario Ministry of Health and Ministry of Long-Term Care 2022).

Recently, there has been growing recognition of the importance of meaningful public engagement in health research, which has not been used widely in administrative data research (CIHR 2023; Teodorowski et al. 2021). In contrast to qualitative and smaller-scale quantitative research, where clear definitions exist for “participants” and “researchers,” research using big data covers a large population unaware of how their de-identified data are stored and used (Teodorowski et al. 2021). Researchers also often feel apprehensive that public engagement will be time-consuming and will compete with many other priorities (Leopold et al. 2020). However, meaningful public collaboration – with genuine intent, transparency and mutual benefit – can enhance research quality and foster public trust in data research (Burt et al. 2022; Paul et al. 2020).

At ICES, strategic activities have been guided by a rotating Public Advisory Council (PAC) since 2018 (Paul et al. 2020). Although a 20-member group could never fully represent the diverse experiences of all Ontarians, careful selection ensures that members encompass a range of geographic locations, ages, abilities, races, gender identities, sexual orientations and healthcare experiences. Their perspectives and values have influenced ICES’ strategic-level activities, but in 2020, PAC members expressed a desire to lead analyses. This was made possible at ICES through funding from the AHRQ program.

Reaching a Consensus on the Project Scope, Topic and Objectives

A consensus process was used to choose a project topic, with multiple feedback sessions with PAC members to support informed decision making and consensus (James Lind Alliance 2021). ICES staff also provided an orientation on ICES’ data repository, idea generation, feasibility assessment and scanning of existing research to avoid duplication (Figure 1). The PAC collectively chose mental health and addictions (MHA) as their project topic to analyze how healthcare utilization patterns differed across time and social circumstances.

Collaborating on Data Analysis and Interpretation

ICES staff developed an analytic plan based on the PAC-identified objectives related to MHA service use. The plan was then presented to the PAC for feedback and discussion. PAC members offered several critical insights for refining the analytic plan, and the second part of the analysis was conducted and presented to the PAC for further discussion (Figure 1).

PAC members also voted on the most important results for knowledge translation. Knowledge translation activities are ongoing, wherein PAC members actively create comprehensive, public-friendly materials to share the project methods and results with a broader audience. PAC members will remain substantially involved with all knowledge translation activities, including co-authorship of this publication.

Reflecting on the Process and Lessons Learned

The PAC's AHRQ project was a novel undertaking for collaborative analysis between public members and academic researchers using administrative data, and it provided insights to improve future collaboration:

Flexibility increased participation

Although evidence-based techniques were used, reaching a consensus within a diverse group was time-intensive. A flexible and lengthy timeline allowed for adequate discussion. Sharing meeting materials in advance and offering different ways to contribute (e.g., e-mail vs. live dialogue) improved participation.

Responsiveness to process feedback improved collaboration

PAC members had several opportunities to comment on the process, and their suggestions were honoured in real time.

Sharing power with the PAC built trust

A values exercise helped align scientific values with PAC values. The importance of the human dimension of the project that went beyond the numbers was an important value. The presence of ICES’ senior leadership at all discussions reinforced ICES’ commitment to collaboration.

The public offered critical insights into data analysis

The PAC's suggestions led to additional analyses of emergency department visits, which enhanced findings compared to when outpatient visits were analyzed alone (Figure 2). The PAC's emphasis on data limitations prompted a re-evaluation of the initial results. This will lead to the development of additional educational resources for scientists using these data in future projects (Figure 2).

Diversity was both a strength and a challenge

Although diverse opinions brought value to the project, equity experts assert that there are cases where representation is more important than diversity (Abebe and George 2022; McFarling 2021). For an MHA-related project, the most representative public members may be those with lived experience related to MHA. One PAC member stated: “Within a ‘majority’ of the general public, ‘minorities' are still marginalized.”

Conclusion

Providing the public with decision-making power in research and better access to data through structures such as the AHRQ program is a key priority for ICES. The lessons from this project will inform planning and process for future projects that aim to empower the public in data analysis through shared decision making. In addition, including the voices of people represented in the data was key to this work and helped to further inform ICES’ Guidance Document and Framework for Anti-Racist Approaches to Research and Analytics (ICES 2023b). As public engagement is a key priority in the ICES 2023–2026 strategic plan (ICES 2023c), ICES will continue to explore ways to engage the public in administrative data analysis.

Comments

Be the first to comment on this!