Healthcare Quarterly
Building Capacity for Patient-Oriented Research: Utilizing Decision Aids to Translate Evidence into Practice, Policy and Outcomes
Abstract
Background: The aim of this project was to engage with patient partners to translate knowledge about the decision aids and develop a scaling-up strategy for wider effects and reach.
Method: This project was guided by the World Health Organization and IDEAS (Integrate, Design, Assess and Share) frameworks for design thinking (e.g., ideating creative strategies), dissemination (e.g., sharing locally and widely) and scalability.
Results: We engaged 132 stakeholders in six webinars, had 321 total page views of the decision aids and conducted 16 interviews to determine revisions to the design of the decision aids before scalability.
Conclusion: Patient-partner collaborations assisted with design thinking, dissemination and scalability.
Key Points
|
Introduction
Building partnerships, improving research quality and impact and developing best practices underpin values that impact patient engagement in research (Haywood et al. 2017). Patient partnerships aptly generate patient-reported outcome measures (Staniszewska et al. 2012), and the strongest predictor of patient engagement and partnership in research is investigators' attitudes (Cary et al. 2015). Guidelines for establishing research partnerships with patients suggest that the following are essential: (1) helpful organizational policies, (2) supportive investigator attitudes that are grounded in shared goals and strong communication practices, (3) principles of trust, respect and co-learning, (4) patient-oriented research (POR) training for all team members, (5) tools/resources for successful patient engagement and (6) value for patient partnerships across various stages of the research cycle (Kirwan et al. 2017).
This program of research consists of multiple phases guided by the Strategy for Patient-Oriented Research (SPOR) capacity development framework (CIHR 2015a) and the SPOR Patient Engagement Framework (CIHR 2015b). In Phase 1A (Figure 1a), two decision aids (patient and investigator) were developed to build capacity and increase engagement of patients with clinical trials (Parry et al. 2020). The decision aids are designed to help patients and investigators decide if they are ready to engage with each other on a research team. Each decision aid has five core functionalities: (1) Introduction (get the facts on POR), (2) What Matters (where patient partners can be engaged, including levels of engagement), (3) Learn More (resources to plan, engage and evaluate patient partnerships), (4) My Readiness (comparing personal priorities with perceived benefits and risks) and (5) My Decision (decisions and next steps, such as finding a patient partner or finding a research team) (Figure 1b).
Decision aids provide information for improving knowledge about potential expectations (Coulter et al. 2013), benefits and risks (Coulter et al. 2013) and options and outcomes (Elwyn et al. 2006). They assist users to gain skills in assessing uncertainties and help to clarify personal priorities about the benefits and risks of engaging in a decision (Elwyn et al. 2006; Volk and Coulter 2018). Most decision aids to date have been designed to provide support in making decisions about health treatments or screening, with evidence that indicates their usefulness in improving knowledge (Feldman-Stewart et al. 2013; Stacey et al. 2014, 2017) and offering choices about what matters most to patients (Stacey et al. 2014).
We collaborated with Clinical Trials Ontario (CTO) in Phase 1B to Engage Multi-stakeholders for Patient Oriented-research Wider Effects and Reach (EMPOWER) of Phase 1A of our research to impacts in Ontario. The aim of Phase 1B was to engage with patient partners to translate knowledge about the decision aids and develop a scaling-up strategy for wider effects and reach. We also aimed to use knowledge gained from Phase 1B to inform Phases 2 and 3 of this research program. POR principles and practices were used throughout Phase 1B and reported using the Guidance for Reporting Involvement of Patients and the Public–Short Form (GRIPP2–SF) (Staniszewska et al. 2017).
Materials and Methods
Patient partners were meaningfully engaged in all aspects of the research process as defined by the SPOR patient engagement framework (CIHR 2015a): (1) identifying research priorities, (2) designing the project for wider effects and reach, (3) developing the funded grant proposal, (4) co-designing and co-delivering all webinar presentations, (5) collecting, analyzing and interpreting interview data, (6) disseminating results through a presentation and a publication and (7) making recommendations for further refinements to the design of each decision aid. Patient partners were compensated for their level of engagement (i.e., commitment, responsibility and scope) based on recommendations provided by the SPOR Chronic Disease Networks and the Primary and Integrated Health Care Innovations (SPOR Networks in Chronic Diseases and the PICHI Network 2018). We engaged racially and ethnically diverse men and women as patient partners to ensure that knowledge translation and dissemination strategies for wider effects and reach were delivered through a lens of equity, diversity and inclusion (EDI).
The IDEAS (Integrate, Design, Assess and Share) (Mummah et al. 2016) and the World Health Organization's (WHO 2010) frameworks for disseminating and scaling up innovations were also used to inform our project. The IDEAS framework uses a flexible repetitive approach to design thinking (e.g., ideate creative strategies) and dissemination (e.g., share locally and widely), supporting a multi-stakeholder method for the development and evaluation of the decision aids. The WHO framework considers the elements (e.g., innovation, user organization, environment, and the resource team) and strategies (e.g., dissemination and advocacy, organizational process, costs/resources, monitoring/evaluation) of scalability and is guided by four principles: systems thinking, sustainability, defining scalability a priori and respect for gender, equity and human rights principles.
Data collection and analysis
Changes in study protocol and delays in projects due to the COVID-19 pandemic are reported per recommendations suggested by Perlis et al. (2021). The original project methods included an initial four-hour in-person brainstorming workshop with multi-stakeholders. User organizations defined by the WHO (2010) would have included organizations expected to adopt and implement the decision aids on a large scale. Intended outputs of the brainstorming workshop, aligned with the WHO (2010), included strategies for dissemination and advocacy (e.g., training, policy dialogues and briefs, cultivating champions), the organization process (e.g., identifying the number and type of "other" organizations for future scale-up), costs/resource mobilization (e.g., ensuring adequate budgetary allocation and linking future [beyond six months] scaling-up to other funding mechanisms) and monitoring and evaluation (e.g., a phone "hotline" and Google analytics). We then planned to offer three-hour face-to-face decision aid training webinars to interested user organizations identified in our brainstorming workshop. As a result of the pandemic, slight modifications to the study protocol included presenting one-hour synchronous and interactive online decision aid training webinars to interested organizations. Social media (i.e., Twitter) assisted with wider effects and reach. Instead of the four-hour brainstorming workshop, a series of interviews were planned with interested multi-stakeholders who attended the decision aid training webinars. A phone hotline was used to address questions, and Google analytics was used to track decision aid page views. Semi-structured interviews were intended to achieve similar outputs aligned with the WHO framework (WHO 2010) that would have been achieved in the initial brainstorming workshop. Interviews were not audiotaped but field notes were made. All data were anonymized and identified only by type of stakeholder. A thematic analysis informed the overall qualitative analytical framework (Braun et al. 2012). The original field notes were revisited regularly to ensure that codes and themes were grounded in the data (Kvale 1996).
Results
The aim of the Phase 1B EMPOWER project was to engage with patient partners to translate knowledge about the decision aids and develop a scale-up strategy for wider effects and reach. Additionally, knowledge gained from Phase 1B would inform Phases 2 and 3 of this research program. Two patient partners co-designed and co-delivered the webinars. They were given access to sufficient and appropriate resources to facilitate engagement (i.e., task/role definitions, training, appropriate time allocation and compensation). Patient partners also collaborated in the development of interview questions and assisted in analyzing and interpreting interview data. They were actively engaged in making priority decisions regarding revisions to the decision aid designs before scalability. Patient engagement details for Phase 1B using GRIPP2–SF are described in Table 1.
Table 1. Patient engagement in Phase 1B (GRIPP2–SF) | |
Section and topic | Item |
2. Method | To ensure that principles of EDI were upheld, one patient partner (woman) was a co-principal investigator and three patient partners (two women, one man) were co-investigators. Gender and racial diversity in patient partnership helped to ensure that knowledge translation and dissemination strategies for wider effects and reach were delivered through an EDI lens. Patient partners engaged in all aspects of the research process:
Three patient partners contributed to editing this paper and are listed as co-authors/members of our Patient-Oriented Research Decision Aids Investigative Team. One of our patient partners (a woman) could not continue in the project due to certain life events. |
3. Results | Two patient partners took the lead for co-designing and co-delivering all the webinars. These same patient partners had collaborated in Phase 1A so they were able to seamlessly explain the patient decision aid to webinar participants and answer questions. Patient partners collaborated in the development of interview questions and assisted in analyzing and interpreting interview data. Patient partners offered feedback to prioritize the integration of the refinements suggested to the design of the patient and investigator decision aids in the interviews. They were also instrumental in providing recommendations for sharing, dissemination and advocacy. Patient partners also led and co-delivered presentations to the Diabetes Research Excellence Cluster at the University of British Columbia and at the Canadian Women's Heart Health Alliance Virtual Summit. |
4. Discussion | Patient collaboration in Phases 1A and 1B has been instrumental to the success of this project and the overall research program in building capacity for patient engagement in research. Although the decision aids were initially developed for use by patients and investigators engaging in clinical trials, ongoing engagement and dissemination strategies highlight the importance of the decision aids across all types of research designs. Patient partners co-authored the publication (Parry et al. 2020) detailing Phase 1A during the tenure of the Phase 1B project. Moreover, additional funding was secured from the Canadian Institutes of Health Research to make refinements to the decision aids based on feedback from Phase 1B – to translate both decision aids to French and conduct alpha (e.g., usability) and beta (e.g., field) testing of the decision aids (Figure 1). |
5. Reflections | One challenge included the withdrawal of a patient partner due to time constraints and the extenuating circumstances caused by the COVID-19 pandemic. This experience emphasized the importance of maintaining awareness of patient-partner needs throughout the project. It was imperative that all POR team members recognized that both short- and long-term commitment to a project could be difficult for some patient partners. Empowering patient partners to feel safe to disclose challenges was essential to the success of this project. Investigators and other POR team members needed to maintain respect and be responsive to patient-partner needs. This challenge highlighted the value of engaging more than one patient partner on a POR team. |
A total of 132 international multi-stakeholders engaged in six interactive webinars from June to November 2020; 59 attendees self-identified as having lived experience (i.e., patients), and the remainder were trainees (n = 9), investigators (n = 29), decision makers (n = 23) and individuals from charitable or patient organizations (n = 9) and SPOR SUPPORT Units (n = 3). A separate webinar was recorded and is available on the CTO YouTube channel (https://www.youtube.com/watch?v=H3XIchDrz2E), which received 109 views during the project. No calls were made to the hotline. Google analytics indicated a total of 321 decision aid page views (landing pages of both the patient and the investigator decision aids) from August to November 2020. The "Introduction" and "Learn More" pages were the most frequently accessed sections of both decision aids. A total of 16 interviews were completed with three policy makers, two investigators and 11 patient partners. Themes were aligned with the IDEAS framework (Mummah et al. 2016) and the WHO framework (WHO 2010) (Table 2). Overall, the feedback on design suggested that the decision aids were clear, logical and visually appealing. Suggestions for refinement (Phase 2) included larger font size; adding hyperlinks, glossary and bookmarks; enabling a certificate of completion or profile for sharing; adding culturally sensitive language, including videos/visuals from racial and ethnically diverse populations; and increasing availability through hard-copy versions (i.e., printable patient-partner decision aid). We created a PEP-CT Twitter account (@pep_ctTOOLS), shared the decision aids via presentations (the Canadian Women's Heart Health Alliance Virtual Summit and the Diabetes Research Excellence Cluster at the University of British Columbia [invited]) and a publication (Parry et al. 2020) and created a Policy Brief (Appendix 1, available online here) for wider effects and reach.
Table 2. Themes aligned with the IDEAS and WHO frameworks | |||
IDEAS framework Integrate |
Design | Assess | Share |
WHO framework | Dissemination and advocacy | ||
Race- and ethnicity-specific content | Acceptable literacy, videos/visuals to reflect EDI and translation to French | Detailed analytics | Marketing strategy (e.g., social media, conferences, meeting invitations, online discussions, publications, educational programs, etc.) |
Larger font, less text | Hyperlinks/glossary and hard-copy versions (i.e., printable patient partner decision aids) | Access to decision aids across all populations, including those from racially and ethnically diverse people in Canada | National reach through patientpartner and non-profit organizations, industry, etc. |
More visuals, videos and transitions (e.g., infographics) | Algorithms based on knowledge gaps. Visuals and videos will include racially and ethnically diverse populations in Canada (e.g., South Asians, black Canadians, etc.) | Access to decision aids on all devices (e.g., iPads) | Mandatory training modules for POR funding |
Sample timelines for research projects | Bookmarks | Sustainable policy | |
Negotiating payments with organizations/reimbursement strategies | Creating a shareable patient profile/report/certificate | ||
Clear messaging and direction after My Decision | Hyperlinks to connect patient partners and investigators | ||
Investigator responsibilities for patient-partner training (e.g., screening, data extraction) | |||
Usefulness of tool for investigators already committed to POR (i.e., helpful information, checklists and links) |
Discussion
Funding from the Ontario SPOR SUPPORT Unit (OSSU) EMPOWER award grant allowed us to translate knowledge about the decision aids to at least 562 multi-stakeholders via webinars and web pages and develop a scale-up strategy for wider effects and reach using the IDEAS (Mummah et al. 2016) and WHO (2010) frameworks. Impacts of patient engagement in this project included (1) effective patient recruitment to webinars and interviews, (2) increased translation and dissemination of results through presentation and publication, (3) improved decision aid uptake in the community (i.e., decision aid page views), (4) informed patient and decision aid enhancements, (5) heightened POR knowledge dissemination to investigators and (6) evidence for a policy brief (Appendix 1). The web-based patient and investigator decision aids are the first to provide POR knowledge and decision support beyond the traditional aids used for health screening and/or treatment decisions (The WHO framework, the Innovation). Although our work focused on building capacity for POR in clinical trials, the decision aids are applicable to patient partners and investigators involved in other demonstration and research projects. For example, in the "Learn More" resource section, the Research Process provides a launching point to learn more about engaging as/with patients in the Design of a research project, Development of the Grant Proposal and Dissemination of results.
We have secured further Canadian Institutes of Health Research funding to refine, translate and conduct alpha (usability) and beta (field) testing for each of the decision aids (Phases 2 and 3). Refinement and evaluation of the decision aids to improve patient partnerships will be guided by the International Patient Decision Aid Standards (Elwyn et al. 2006; Volk and Coulter 2018), user-centred design (Abras et al. 2004) and the Ottawa decision-support framework (The Ottawa Hospital 2015). The International Patient Decision Aid Standards provide an evidence framework for the content, development, implementation and evaluation of decision aids (Elwyn et al. 2006; Stacey et al. 2017). In addition, user-centred designs optimize end-user experience (Plaisance et al. 2018; Witteman et al. 2015) and web-based platforms facilitate development (Hoffman et al. 2014). Relevant design feedback from Phase 1B (i.e., including those prioritized by patient partners and investigators) will be incorporated into refinements of decision aids, and then each decision aid will be translated to French, refined through iterative cycles of usability testing with patients and investigators (Phase 2) and evaluated using a pragmatic pre–post pilot study (Phase 3).
Conclusion
Patient partners were actively engaged in translating knowledge about the decision aids and developing a scale-up strategy for wider effects and reach. The IDEAS framework provided a flexible repetitive approach to design thinking (e.g., ideating creative strategies) and dissemination. The WHO framework helped us identify strategies to increase decision aid impacts for research and policy through a gender, equity and human rights lens – for example, use of the decision aids (1) for all research designs (i.e., not only for clinical trials), (2) as a repository of essential POR and sex/gender information for new and experienced patient partners and investigators and (3) for mandatory online training for POR funding applications. Patient engagement in this project and in this research program is a necessity – patients are the heart of SPOR (CIHR 2015b) and our team's patient partners were actively engaged in Phase 1A and continued to collaborate and co-lead Phase 1B. The decision aids will contribute to Canada's strategy for POR to support the collaborative efforts of patients and investigators in building a sustainable, accessible and equitable healthcare system.
About the Author(s)
Monica Parry, NP-Adult, PhD, is an associate professor at the Lawrence S. Bloomberg Faculty of Nursing at the University of Toronto in Toronto, ON. She can be contacted at monica.parry@utoronto.ca.
Dawn P. Richards, PhD, is the director of Patient and Public Engagement at Clinical Trials Ontario in Toronto, ON.
David Wells, PhD, is a patient partner with Diabetes Action Canada in Toronto, ON.
Adhiyat Najam, MSc (candidate), is a patient partner with Diabetes Action Canada in Toronto, ON.
Salima Hemani, PhD (candidate), is a trainee at the Lawrence S. Bloomberg Faculty of Nursing at the University of Toronto in Toronto, ON.
Susan Marlin, MSc, is the president and CEO at Clinical Trials Ontario in Toronto, ON.
The Patient-Oriented Research Decision Aids Investigative Team is composed of senior scientists/investigators (Anne Ellis, Ian Gilron), mid-career scientists/investigators (Monica Parry, Dawn P. Richards, Karine Toupin), early career scientists/investigators (Ann Kristin Bjørnnes), decision makers (Susan Marlin), trainees (Salima Hemani) and patient partners (David Wells, Adhiyat Najam, Tina Ceroni) who are actively engaged in providing scientific and practical knowledge across all phases of this research program.
Acknowledgment
The authors thank the participants who attended the webinars and those who shared further insights in a semi-structured interview.
Funding
Funding for Phase 1B was received from an OSSU EMPOWER Award Grant.
Disclosure
Dawn P. Richards is a full-time employee of Five02 Labs that is under contract to CTO for patient and public engagement activities. The other authors have no conflicts of interest to disclose.
References
Abras, C., D. Maloney-Krichmar and J. Preece. 2004. User-Centered Design. In W. Bainbridge, Ed., The Encyclopedia of Human-Computer Interaction (pp. 1–14). Sage Publications.
Braun, V. and V. Clarke. 2012. Thematic Analysis. In H. Cooper, P.M. Camic, D.L. Long, A.T. Panter, D. Rindskopf and K.J. Sher, Eds., APA Handbook of Research Methods in Psychology, Vol. 2. Research Designs: Quantitative, Qualitative, Neuropsychological, and Biological (pp. 57–71). American Psychological Association.
Canadian Institutes of Health Research (CIHR). 2015a. Capacity development framework. Strategy for Patient-Oriented Research Capacity Development Framework. Retrieved July 23, 2019. <http://www.cihr-irsc.gc.ca/e/49307.html>.
Canadian Institutes of Health Research (CIHR). 2015b. Strategy for Patient-Oriented Research – Patient Engagement Framework. Retrieved July 23, 2019. <http://www.cihr-irsc.gc.ca/e/48413.html>.
Cary, M.S., J.D. Rubright, J.D. Grill and J. Karlawish. 2015. Why Are Spousal Caregivers More Prevalent than Nonspousal Caregivers as Study Partners in AD Dementia Clinical Trials? Alzheimer Disease and Associated Disorders 29(1): 70–74. doi:10.1097/WAD.0000000000000047.
Coulter, A., D. Stilwell, J. Kryworuchko, P.D. Mullen, C.J. Ng and T. van der Weijden. 2013. A Systematic Development Process for Patient Decision Aids. BMC Medical Informatics and Decision Making 13(Suppl 2): S2. doi:10.1186/1472-6947-13-S2-S2.
Elwyn, G., A. O'Connor, D. Stacey, R. Volk, A. Edwards, A. Coulter et al. 2006. Developing a Quality Criteria Framework for Patient Decision Aids: Online International Delphi Consensus Process. BMJ 333(7565): 417. doi:10.1136/bmj.38926.629329.AE.
Feldman-Stewart, D., M. O'Brien, M.L. Clayman, B.J. Davison, M. Jimbo, M. Labrecque et al. 2013. Providing Information about Options in Patient Decision Aids. BMC Medical Informatics and Decision Making 13(2): S4. doi:10.1186/1472-6947-13-S2-S4.
Haywood, K., A. Lyddiatt, S.J. Brace-McDonnell, S. Staniszewska and S. Salek. 2017. Establishing the Values for Patient Engagement (PE) in Health-Related Quality of Life (HQoL) Research: An International, Multiple-Stakeholder Perspective. Quality of Life Research 26(6): 1393–404. doi:10.1007/s11136-016-1465-5.
Hoffman, A.S., H.A. Llewellyn-Thomas, A.N.A. Tosteson, A.M. O'Connor, R.J. Volk, I.M. Tomek et al. 2014. Launching a Virtual Decision Lab: Development and Field-Testing of a Web-Based Patient Decision Support Research Platform. BMC Medical Informatics and Decision Making 14: 112. doi:10.1186/s12911-014-0112-8.
Kirwan, J.R., M. de Wit, L. Frank, K.L. Haywood, S. Salek, S. Brace-McDonnell et al. 2017. Emerging Guidelines for Patient Engagement in Research. Value in Health 20(3): 481–86. doi:10.1016/j.jval.2016.10.003.
Kvale, S. 1996. Interviews: An Introduction to Qualitative Research Interviewing. Sage Publications.
Mummah, S.A., T.N. Robinson, A.C. King, C.D. Gardner and S. Sutton. 2016. IDEAS (Integrate, Design, Assess, and Share): A Framework and Toolkit of Strategies for the Development of More Effective Digital Interventions to Change Health Behavior. Journal of Medical Internet Research 18(12): e317. doi:10.2196/jmir.5927.
The Ottawa Hospital. 2015. Ottawa Decision Support Framework (ODSF). Retrieved July 23, 2019. <https://decisionaid.ohri.ca/odsf.html>.
Parry, M., A.K. Bjørnnes, K. Toupin-April, A. Najam, D. Wells, A. Sivakumar et al. 2020. Patient Engagement Partnerships in Clinical Trials: Development of Patient Partner and Investigator Decision Aids. Patient 13(6): 745–56. doi:10.1007/s40271-020-00460-5.
Perlis, R.H., S.J.P.A. Haneuse, G.D. Rubenfeld, S.D. Fihn and F.P. Rivara. 2021. Reporting Clinical Studies Affected by the COVID-19 Pandemic: Guidelines for Authors. JAMA Network Open 4(1): e2036155. doi:10.1001/jamanetworkopen.2020.36155.
Plaisance, A., H.O. Witteman, A. LeBlanc, J. Kryworuchko, D.K. Heyland, M.H. Ebell et al. 2018. Development of a Decision Aid for Cardiopulmonary Resuscitation and Invasive Mechanical Ventilation in the Intensive Care Unit Employing User-Centered Design and a Wiki Platform for Rapid Prototyping. PLoS ONE 13(2): e0191844. doi:10.1371/journal.pone.0191844.
SPOR Networks in Chronic Diseases and the PICHI Network. 2018. Recommendations on Patient Engagement Compensation. Retrieved July 23, 2019. <https://diabetesaction.ca/wp-content/uploads/2018/07/TASK-FORCE-IN-PATIENT-ENGAGEMENT-COMPENSATION-REPORT_FINAL-1.pdf>.
Stacey, D., F. Légaré, N.F. Col, C.L. Bennett, M.J. Barry, K.B. Eden et al. 2014. Decision Aid for People Facing Health Treatment or Screening Decisions. Cochrane Database of Systematic Reviews 28(1): CD001431. doi:10.1002/14651858.CD001431.pub4.
Stacey, D., F. Légaré, K. Lewis, M.J. Barry, C.L. Bennett, K.B. Eden et al. 2017. Decision Aids for People Facing Health Treatment or Screening Decisions. Cochrane Database of Systematic Reviews 4(4): CD001431. doi:10.1002/14651858.CD001431.pub5.
Staniszewska, S., J. Brett, I. Simera, K. Seers, C. Mockford, S. Goodland et al. 2017. GRIPP2 Reporting Checklists: Tools to Improve Reporting of Patient and Public Involvement in Research. BMJ 358: j3453. doi:10.1136/bmj.j3453.
Staniszewska, S., K.L. Haywood, J. Brett and L. Tutton. 2012. Patient and Public Involvement in Patient-Reported Outcome Measures: Evolution Not Revolution. Patient 5(2): 79–87. doi:10.2165/11597150-000000000-00000.
Volk, R.J. and A. Coulter. 2018. Advancing the Science of Patient Decision Aids through Reporting Guidelines. BMJ Quality and Safety 27(5): 337–39. doi:10.1136/bmjqs-2017-007657.
Witteman, H.O., S.C. Dansokho, H. Colquhoun, A. Coulter, M. Dugas, A. Fagerlin et al. 2015. User-Centered Design and the Development of Patient Decision Aids: Protocol for a Systematic Review. Systematic Reviews 4(1): 11. doi:10.1186/2046-4053-4-11.
World Health Organization (WHO). 2010. Nine Steps for Developing a Scaling-Up Strategy. Retrieved July 23, 2019. <https://www.who.int/reproductivehealth/publications/strategic_approach/9789241500319/en/>.
Footnotes
P = Patient partner.
Comments
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed