Healthcare Quarterly

Healthcare Quarterly 24(Special Issue) April 2022 : 44-48.doi:10.12927/hcq.2022.26775
Co-Designing Interventions and Tools

Evaluation of Experiences and Impact of Patient Engagement on e-Health Research: A Qualitative Study

Rachel Y. Pan*, Kendra Zhang*, Arani Sivakumar, Dorothy Choi, Angel H. Wang, Pauline Wijeyesekerap and Catherine H. Yu


Background: Health technology has increasingly moved toward adopting a "user-centred design" approach to include the user/patient throughout the innovation and design process; however, few studies have evaluated the patient's experience of such an engagement.

Objective: The aim of this study was to explore the role of patient engagement (PE) within e-health innovation research.

Method: Using qualitative descriptive methodology, semi-structured interviews were conducted with eight participants (patient partners and research/development team members).

Findings: Key themes were centred on enablers of, challenges to and methods of improving PE.

Conclusion: PE must be prioritized from study conception, explicitly programmed into study conduct and valued by integrating patient partner input.

Key Points

  • This study provided a multi-stakeholder perspective on patient engagement (PE), revealing that stakeholders’ perspectives differed and must be considered separately when planning for engagement.
  • This study identified strategies to improve PE during the development of an e-health tool, including providing support
    for patients throughout the process, ensuring thorough team communication, making patients feel valued for their contributions and fostering a positive research environment.
  • Overall, PE helped ensure the relevance, usability and appropriateness of an e-health tool for different patient demographics; however, the process of PE should pay special attention to recruiting representative patient populations.


Patient engagement (PE) is a meaningful and active collaboration in governance, disease diagnosis and management, priority setting, conducting research and knowledge translation (CIHR 2014). With the shift away from paternalism in healthcare, PE in health research has increasingly become a key research priority in Canada (Manafo et al. 2018). E-health is a category within digital health that denotes the wide range of information, expertise and communication technologies that support health-related fields, including health surveillance, education and healthcare delivery (Baker et al. 2014). Despite its many potential benefits to patients, existing e-health services are largely technology-driven rather than being personalized and user-centred, resulting in tools that fail to address individualized patient goals and that, thus, cannot be widely adopted (Cowie et al. 2013). In response, MyDiabetesPlan (MDP), a web-based shared decision making and priority-setting patient decision aid, was developed by engaging patients in the research process of e-health tool development in order to create a more user-centred e-health toolkit (Yu et al. 2014).

There is a lack of investigation into PE within e-health, which, in today's digital climate, is a deficiency (Leung et al. 2019). Furthermore, few studies have explored patients' perspectives of and experiences in such an engagement. Existing studies tend to leave research topics unspecified and predominantly report only on academic researchers' experiences (Bhati et al. 2020). Moreover, PE within research has only recently garnered attention in North America and warrants findings specific to local populations (Bhati et al. 2020). With the role of patient partners (PPs) in improving research feasibility, acceptability, rigor and relevance, this Engage Multi-stakeholders for Patient Oriented-research Wider Effects and Reach (EMPOWER) Award project explored the experiences and perspectives of the PPs and researchers from the initial research project to guide PE in the future (Forsythe et al. 2019).

The purpose of this study was to explore the role of PE within Canadian e-health innovation research using our experience with MDP as a case study. We aimed to identify how patients were engaged, the impact and benefits of this engagement, challenges that were faced and how PE can be more effective moving forward. The PE in research framework was used to guide this line of inquiry (Hamilton et al. 2018). By involving both PPs and research team members, we obtained a multi-stakeholder perspective.


Our previous work developed and trialled MDP in a multi-centred randomized controlled trial (Yu et al. 2020). Recently, through the EMPOWER grant, we explored patients' and clinicians' experiences regarding its integration into clinical care and interprofessional stakeholder involvement to assess the implementation potential of MDP. Thus, our study acts as an extension of the broader MDP project (Sivakumar et al. 2021). PPs were consulted frequently as members of the research team throughout all phases of the research program, from innovation ideation, development and testing to implementation. In the development of this evaluation of PE study, one of the authors (PW) was consulted to arrive at the most relevant research question from a PP's perspective.

We used qualitative descriptive methodology as it offers an in-depth understanding of participants' experiences and impact of PE. We adhered to the Guidance for Reporting Involvement of Patients and the Public–Long Form (GRIPP2–LF) reporting checklist (Staniszewska et al. 2017).

To date, MDP has involved a multi-disciplinary team that consisted of PPs (n = 3), software development team members (n = 2) and research team members (n = 13). In the EMPOWER study, these same members in addition to Ontario Ministry of Health participants were end-user participants. In order to gain the experiences of all those who worked on MDP, recruitment occurred by contacting PPs and team members through e-mail and obtaining informed consent.

Data were obtained through semi-structured interviews. The interview guides (one each for PPs and research team members) explored experiences with PE in MDP, challenges that were faced and how PE in research can be more effective moving forward, the contents of which were based on the GRIPP2 framework. Interviews were conducted virtually via Zoom due to the COVID-19 pandemic by research team members trained in qualitative interviewing (RP and KZ). All interviews were audiotaped, transcribed verbatim and annotated. Content analysis was carried out in an iterative fashion alongside data collection (Elo and Kyngäs 2008; Korstjens and Moser 2018).


We interviewed eight participants (Table 1).

We describe participants' views on ways to sustain PE and the impacts, enablers and hindrances of PE. See Tables 2 and 3 for representative quotes (available online here).

TABLE 1. Participants
Stakeholder group Participant
PPs Patient living with type 1 diabetes, has been a PP from October 2015 to present.
Patient living with type 2 diabetes, has been a PP from February 2016 to present.
Patient living with type 2 diabetes, has been a PP from December 2017 to present
Research team members Physician and site lead
Physician and site lead
Registered nurse
Research coordinator/assistant
Development team member Software developer


PPs' experiences and perspectives

Ways in which meaningful PE was sustained

PPs felt valued and "listened to" through integration into the entirety of the research process. They appreciated providing input for, testing and evaluating the e-health tool and being consulted on the selection of study outcomes and data collection tools. This was enabled by frequent meetings with the research team.

They reported that their sentiment of feeling respected, valued and able to contribute (despite initially thinking that their own non-medical backgrounds would be a limitation) helped sustain PE. They felt empowered to apply their unique skills, coming from different cultural and ethnic backgrounds, thus contributing to a collaborative team environment. They also identified the creation of opportunities that demonstrated the value of PE; for example, PPs were given the opportunity to present at conferences alongside research team members.

Positive and negative results of PE and its impacts on the research and individuals involved

PE provided first-hand patient perspectives and experiences, which helped ensure the tool's relevance to patients and guided the direction of the research. PPs helped refine the tool's user interface and ensured patient understandability of the tool; they also helped contextualize the tool in terms of language barriers, generational gaps and cultural nuances. PPs themselves were able to get the perspective and support of other individuals with diabetes while learning more about the condition. Furthermore, learning about the time and effort that goes into conducting research was paradigm-shifting for the PPs, and they developed a greater appreciation for the process. PPs felt empowered and confident by the act of providing their perspectives and creating impactful change, and experienced fulfillment from long-term involvement with the research project.

Contextual and research process factors that enabled the impact of PE

PPs' prior interest in diabetes-related topics and their motivation to self-educate was a contextual factor that spurred their engagement. In terms of the research process, PPs felt comfortable and confident voicing their concerns in the team environment and were supported if needed. Having differing contributions be valued helped create a positive experience for the PPs and, in particular, a trusting relationship with team members before and throughout engagement further promoted the same.

Contextual and research process factors that hindered the impact of PE

PPs noted that their comprehension and insecurity about what they bring to the table were potential hindrances to full engagement. A contextual challenge was encountered by patients who spoke English as a second language, which at times acted as a language barrier. Similarly, PPs were sometimes challenged and had difficulty understanding everything discussed at meetings, especially medical terminology. Regarding the research process, PPs found it difficult to maintain motivation during periods when there was no communication from the researchers.

Researchers' perspectives and experiences

Ways in which meaningful PE was sustained

Throughout the research process, researchers appreciated that PPs were engaged consistently alongside research members. Researchers also valued PPs as the main "experts" on diabetes, and their personal experiences and inputs were valued equally to inputs by other research members.

Positive and negative results of PE and its impacts on the research and individuals involved

The first notable impact patients made was providing diverse perspectives and experiences. These insights not only helped to identify discrepancies between researchers' assumptions about patient knowledge and opinions and the reality for patients; these insights also would sometimes change the direction of the research itself. Specifically, in considering the development of an e-health innovation, patients played a critical role in ensuring the relevance, practicality and usability of the tool, as well as other patient-facing research materials, for patients. Researchers particularly appreciated that patient input improved the efficiency of the research. However, researchers noted that a somewhat negative impact of PE included increasing the amount of time needed to conduct research and also that it was important to take this additional time to ensure that the final outcome was appropriate and justified.

Contextual and research process factors that enabled the impact of PE

Firstly, institutional support and resource allocation to PE was a facilitator to MDP as there was a large amount of support, but researchers noted that it was a challenge within the research community, overall. Additionally, researchers noted that relationship building between the research team and the PPs both before and during the project was critical to fostering engagement and successful partnership. Similarly, the effort that researchers took to speak the same "language" as patients was identified as an enabler of PE.

Contextual and research process factors that hindered the impact of PE engagement

Researchers expressed several challenges to PE. Ensuring that PPs constantly feel valued and engaged was identified as a challenge; however, within the context of MDP, researchers felt that PPs were enthusiastic and dedicated. More broadly, researchers held concerns regarding the process of PE, which included the inherent power differential between PPs and research team members and how our sample of PPs may not be representative of the whole patient population of interest.


Prior studies have shown that the usability, understandability and relevance that PPs bring to e-health tools lead to increased uptake of new technologies (Lupton 2013).

Our PPs reported a predominantly positive experience with PE, which was attributed to feeling valued by the research team, finding a community of other diabetes patients, learning about diabetes and feeling empowered about disease management. Many of these positive findings are consistent with a study by Forsythe et al. (2019), which analyzed PE across 126 studies that described the contribution(s) of engagement to their project; however, our study is one of the few that investigated challenges and enablers expressed by PPs themselves specifically within the context of e-health. Age, language, cultural background and prior knowledge impacted the ease with which patients could engage with the research and the tool being developed. The PPs' motivation to self-educate and the research team's supportiveness facilitated their engagement; however, an area of improvement was the need for increasing the regularity of communication with patients.

Researchers found it valuable when patients provided first-hand experience and input as it validated their work and grounded it in real patient perspectives. Their feedback on experiencing the disease, financial constraints and lack of knowledge, education and accessibility to family physicians was insightful. Researchers found that MDP engaged patients meaningfully, with patients being extensively involved throughout the research process. Within the literature, one of the concerns that researchers have for PE is its potential tokenistic usage of patients (Carroll et al. 2017); our findings demonstrate that treating patients as an equal member of the team and a previous positive relationship with a core research member may mitigate this concern. Other facilitators to engagement included increased institutional support and using accessible language. Researchers emphasized the challenge of PPs not being representative of the whole population of interest, which is consistent with the literature (Carroll et al. 2017). Although our PPs had different cultural backgrounds, there should be more measures to involve a wide range of patient populations and uphold equity, diversity and inclusion. A centralized process for patient recruitment and engagement may help address this; however, the difficulty lies in doing so in a meaningful and ethical way (Bishop et al. 2018).

Limitations included our limited sample size, which may not be representative of the diabetes patient population or other e-health research teams.


The value-add of PE was its role in creating an e-health tool that is indeed beneficial and sensitive to end-users' needs. Our work suggests the need for more centralized processes for PE that also involves more diverse patient populations while ensuring adequate relationship building between the research teams and partners; strategies to improve PE during the development of an e-health tool include providing support for patients throughout the process, ensuring thorough team communication, making patients feel valued for their contributions and fostering a positive research environment.

About the Author(s)

Rachel Y. Pan, BHSc, is a medical student at the Temerty Faculty of Medicine, University of Toronto in Toronto, ON, with research interests in the areas of management of complex chronic medical conditions, mental health and medical education.

Kendra Zhang is a medical student at the School of Medicine, Queen's University in Kingston, ON, with research interests in global health, biopsychosocial approaches to managing chronic illnesses and the social determinants of health.

Arani Sivakumar, BMSc, is an MPH student at St. Michael's Hospital, Unity Health Toronto, in Toronto, ON, with research interests in the use of digital health to mediate the management of chronic conditions; diversity, equity and inclusion within healthcare; and Indigenous public health.

Dorothy Choi, BSc, is a research assistant and master of health design student at St. Michael's Hospital, Unity Health Toronto, in Toronto, ON, with research interests in digital health tools and medical humanities in chronic care.

Angel H. Wang, RN, BScN, MN, is a master's-prepared advanced practice nurse at the Michael Garron Hospital in Toronto, ON, with research interests that lie in three main themes: (1) leveraging technology in healthcare, (2) nursing leadership and work environments and (3) gerontology and dementia care.

Pauline Wijeyesekera is a patient, caregiver and knowledge user, who has retired after a 30-year career as a corporate secretary at the Heart and Stroke Foundation of Canada, having a passion for medical research.

Catherine H. Yu, MD, MHSc, is an endocrinologist at the Department of Medicine, University of Toronto in Toronto, ON, with a research interest in the use of integrative health informatics tools, patient and clinician education and behaviour change, shared decision making and medical humanities in improving the quality of care of people with diabetes. She can be reached at


Baker, T.B., D.H. Gustafson and D. Shah. 2014. How Can Research Keep Up with eHealth? Ten Strategies for Increasing the Timeliness and Usefulness of eHealth Research. Journal of Medical Internet Research 16(2): e36. doi:10.2196/jmir.2925.

Bhati, D.K., M. Fitzgerald, C. Kendall and S. Dahrouge. 2020. Patients' Engagement in Primary Care Research: A Case Study in a Canadian Context. Research Involvement and Engagement 6(1): 65. doi:10.1186/s40900-020-00238-x.

Bishop, A.C., M.J. Elliott and C. Cassidy. 2018. Moving Patient-Oriented Research Forward: Thoughts from the Next Generation of Knowledge Translation Researchers. Research Involvement and Engagement 4(1): 23. doi:10.1186/s40900-018-0110-6.

Canadian Institutes of Health Research (CIHR). 2014. Strategy for Patient-Oriented Research – Patient Engagement Framework. Government of Canada. Retrieved February 28, 2022. <>.

Carroll, S.L., G. Embuldeniya, J. Abelson, M. McGillion, A. Berkesse and J.S. Healey. 2017. Questioning Patient Engagement: Research Scientists' Perceptions of the Challenges of Patient Engagement in a Cardiovascular Research Network. Patient Preference and Adherence 11: 1573–83. doi:10.2147/PPA.S135457.

Cowie, M.R., C.E. Chronaki and P. Vardas. 2013. E-Health Innovation: Time for Engagement with the Cardiology Community. European Heart Journal 34(25): 1864–68. doi:10.1093/eurheartj/ehs153.

Elo, S. and H. Kyngäs. 2008. The Qualitative Content Analysis Process. Journal of Advanced Nursing 62(1): 107–15. doi:10.1111/j.1365-2648.2007.04569.x.

Forsythe, L.P., K.L. Carman, V. Szydlowski, L. Fayish, L. Davidson, D.H. Hickam et al. 2019. Patient Engagement in Research: Early Findings from the Patient-Centered Outcomes Research Institute. Health Affairs 38(3): 359–67. doi:10.1377/hlthaff.2018.05067.

Hamilton, C.B., A.M. Hoens, S. McQuitty, A.M. McKinnon, K. English, C.L. Backman et al. 2018. Development and Pre-Testing of the Patient Engagement in Research Scale (PEIRS) to Assess the Quality of Engagement from a Patient Perspective. PLoS ONE 13(11): e0206588. doi:10.1371/journal.pone.0206588.

Korstjens, I. and A. Moser. 2018. Series: Practical Guidance to Qualitative Research. Part 4: Trustworthiness and Publishing. European Journal of General Practice 24(1): 120–24. doi:10.1080/13814788.2017.1375092.

Leung, K., D. Lu-McLean, C. Kuziemsky, R.G. Booth, S.C. Rossetti, E. Borycki et al. 2019. Using Patient and Family Engagement Strategies to Improve Outcomes of Health Information Technology Initiatives: Scoping Review. Journal of Medical Internet Research 21(10): e14683. doi:10.2196/14683.

Lupton, D. 2013. The Digitally Engaged Patient: Self-Monitoring and Self-Care in the Digital Health Era. Social Theory & Health 11(3): 256–70. doi:10.1057/sth.2013.10.

Manafo, E., L. Petermann, P. Mason-Lai and V. Vandall-Walker. 2018. Patient Engagement in Canada: A Scoping Review of The ‘How’ and ‘What' of Patient Engagement in Health Research. Health Research Policy and Systems 16(1). doi:10.1186/s12961-018-0282-4.

Sivakumar, A., R.Y. Pan, D. Choi, A. He Wang and C.H. Yu. 2021. "If We Got a Win-Win, You Can Sell It to Everybody": A Qualitative Study Employing Normalization Process Theory to Identify Critical Factors for eHealth Implementation and Scale-up in Primary Care [Epub]. Canadian Journal of Diabetes. doi:10.1016/j.jcjd.2021.08.005.

Staniszewska, S., J. Brett, I. Simera, K. Seers, C. Mockford, S. Goodlad et al. 2017. GRIPP2 Reporting Checklists: Tools to Improve Reporting of Patient and Public Involvement in Research. BMJ 358: j3453. doi:10.1136/bmj.j3453.

Yu, C., D. Choi, B.A. Bruno, K.E. Thorpe, S.E. Straus, P. Cantarutti et al. 2020. Impact of MyDiabetesPlan, a Web-Based Patient Decision Aid on Decisional Conflict, Diabetes Distress, Quality of Life, and Chronic Illness Care in Patients with Diabetes: Cluster Randomized Controlled Trial. Journal of Medical Internet Research 22(9): e16984. doi:10.2196/16984.

Yu, C.H., D. Stacey, J. Sale, S. Hall, D.M. Kaplan, N. Ivers et al. 2014. Designing and Evaluating an Interprofessional Shared Decision-Making and Goal-Setting Decision Aid for Patients with Diabetes in Clinical Care – Systematic Decision Aid Development and Study Protocol. Implementation Science 9(1): 16. doi:10.1186/1748-5908-9-16.


* Lead co-authors.

P = Patient, caregiver and knowledge user.


Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed