HealthcarePapers
Health System Transformation through Research Innovation
Abstract
The Canadian Institutes of Health Research’s Institute of Health Services and Policy Research (IHSPR) has set out an ambitious direction for the next five years. We aim to build the scientific leadership for learning health systems in Canada, tap into the transformative potential of eHealth for Canadian healthcare, find a better system to support aging in the community, and provide research intelligence on the question of how to finance and fund the health system of the future.
Healthcare and Health Services and Policy Research in Canada
Healthcare spending in Canada is growing at an unsustainable rate, exceeding $210 billion in 2013 (CIHI 2013). Canada invests $5,446.50 per person on healthcare (OECD 2015), considerably more than other OECD countries, but ranks second to last in key healthcare areas, such as access, safety, and quality of care (Schoen et al. 2013). Key cost drivers include provider compensation, utilization of services and the emergence of new devices and technologies (CIHI 2013). Spending on healthcare delivery accounts for close to 50% of total budgets in a number of provinces and territories, crowding out spending on other important priorities, like education and social services. These financial pressures provide impetus for transformational change and an increasing pull for cutting-edge research that can pioneer innovations in health system delivery, which can lower costs, improve patient experience, quality of care and the health of Canadians. Canada’s healthcare “system” provides a unique environment for health services and policy research in that it comprises over 13 distinct delivery systems – one in each of the 10 provinces and 3 territories, as well as federal systems for certain populations (e.g., First Nations and Inuit peoples, the military and prison populations). This rich arena of innovation and experimentation generates valuable opportunities for natural experiments and cross-jurisdictional comparative analyses that can shed insight into the successful features of different service delivery models and areas for growth and improvement.
The Canadian Institutes of Health Research
Canada’s health services and policy research enterprise has evolved significantly over the past 20 years and has witnessed growth in many areas, including funding and programs to support innovative research (Figure 1).
Figure 1. The history of health services and policy research in Canada
One of the seminal achievements of the health services and policy research enterprise was the formation of the Institute of Health Services and Policy Research (IHSPR) as one of 13 Institutes within the Canadian Institutes of Health Research (CIHR). Established in 2000, CIHR’s mandate was to create new scientific knowledge and to catalyze its translation into improved health, more effective health services and products and a strengthened Canadian healthcare system.
In the first decade, based on application data to CIHR, health services and policy research grew under this new organization. Between 2001 and 2011, funding for grant applications for health services and policy research increased from $12.6 to $48 million (Figure 2); the annual number of applications increased from 327 to 1,137; and the number of principal investigators applying to CIHR increased from 290 to 659.
Figure 2. Total funding awarded grant applications for health services and policy research in the open and strategic by fiscal year (2001–2011)
However, health services and policy research continues to represent a very small proportion of the strategic and open operating grant funds awarded by CIHR: 3.2% of overall funding in 2001–2002 and 6.3% of all applications funded in 2011–2012 (Figure 3). A paradigm shift is needed if health services and policy research is to drive health system transformation. We will need to create alignment and synergy among health services research funders, researchers and end-users, build a vision of what we want to accomplish, establish what we need to do and build a strategy to get there.
Figure 3. The number of CIHR grant applications by pillar and year
A Common Vision and Strategic Direction for Health Services and Policy Research: Building a Canadian Alliance
Based on the success of the Canadian Cancer Research Alliance, the Canadian Health Services and Policy Research Alliance (CHSPRA) was established to foster collaboration, coordination and strategic investment among health services and policy research organizations in Canada. Aligning our vision and strategic direction creates the capacity to accelerate scientific innovation and discovery in health services and policy research, optimize the impact of research on health and health system outcomes and strengthen the research enterprise.
As an initial step, 27 organizations involved in funding health services and policy research (CIHR 2015) collaborated to create an asset map of the collective investments over a five-year period (2007–2012), by location, type of investment and content area. Overall, $770 million was spent in health services and policy research over the five-year period (Figure 4). Funding was awarded to 225 organizations active in Canada’s health services and policy research enterprise.
Figure 4. Total health services and policy research investment in Canada (2007–2011)
CIHR accounted for 37.7% of health services research funding, opening opportunities to increase synergy by collaborating on common priorities with provincial health research funders, health charities and other funders (Figure 5).
Figure 5. Leading funders of health services and policy research in Canada (2007–2011)
The top funded themes included access to appropriate care across the continuum (14.4%), managing for quality and safety (11.9%) and linking population and public health with health services (9.4%) (Figure 6). Very little investment had been made in healthcare financing and funding (1.6%) and change management/scaling up innovation (0.3%), even though they were hot topics identified by many policy think tanks (Health Services Research Europe 2011; Klein et al. 2013; WHO 2008, 2013).
Figure 6. Total health services and policy research investment by research theme (2007–2011)
To build a vision for the future, input was solicited through a web-based survey of 400 Canadian health services and policy researchers, 55 regional informants, including researchers and policymakers, the general public through a Café Scientifique for 117 participants and international leaders through a panel at the annual meeting of the Canadian Association for Health Services and Policy Research (CAHSPR). The collective input from stakeholders was presented at a national Priorities Forum of over 100 funders, policy and decision makers, researchers and end-users in April 2014, who established a vision and direction for the next five years. Seven foundational strategic directions and five priorities were identified for investment (Figure 7).
Figure 7. Health services and policy research priorities and foundational strategic directions
There was immediate interest in working collectively on two strategic directions where resources were already available and being used to address them: 1) measuring health services and policy research impact, and 2) accelerating the creation of a cadre of scientists that could work within the context of a learning health system.
Strategic Plan: Institute of Health Services and Policy Research 2015–2019
The Institute of Health Services and Policy Research (IHSPR) aligned its new five-year strategy with the pan-Canadian Vision and Strategy for Health Services and Policy Research (CIHR 2015). Both envision a future where research intelligence and strategic partnerships are necessary to drive health system transformation to improve health and health system outcomes for Canadians. IHSPR’s Institute Advisory Board selected four areas that IHSPR was well positioned to advance based on an assessment of: 1) gaps and strengths; 2) potential for international leadership; 3) potential for partnering; 4) alignment with CIHR Health Research Roadmap II (CIHR 2015) and synergies with the Strategy for Patient-Oriented Research (SPOR); and 5) opportunities for inter-Institute collaboration.
Strategic Priority 1: The Creation of Learning Health Systems and the Next Generation of Researchers with the Skills to Partner in Health System Learning and Transformation
Each day, millions of Canadians are seen within the healthcare system and trillions of bits of information are generated. Increasingly, the day-to-day use of health and social services is recorded digitally at the point of care. This information could be harnessed to understand the comparative effectiveness of different treatments, the causes of potentially avoidable adverse events, unnecessary costs, missed opportunities for prevention and to capture the collective wisdom on how to improve patient experience. However, for the most part, we have not used this information to produce knowledge on how we could do better (Gawande 2007). A major initiative that is gaining momentum in the US is to create “learning health systems,” accountable care organizations that use their data in an intelligent fashion as a guide to improving care in a dynamic way (Committee on the Learning Health Care System in America, Institute of Medicine 2013). The learning health system emphasizes collaboration across all health borders to drive an efficient and effective system (Backus et al. 2001; Gooch et al. 2012; James and Savitz 2011).
The gap
There are many challenges to address before it becomes possible to move from the health system of today to a learning system of tomorrow. However, a fundamental requirement for success is capable scientific, clinical and policy leadership that will nurture the ability of a health system to experiment with innovation, learn from failure and scale up success. The skill sets required of scientists within learning health systems are different from those acquired in classic training. They need to be able to partner with clinical and policy leadership to identify relevant priorities for research, develop new methods for rapid scientific investigation using point-of-care patient experience and digital health and social data, collaborate on the most effective use of emerging knowledge for clinical and policy decisions and implement and evaluate innovative solutions.
The objective
To train and fund a new generation of scientists who can provide scientific leadership in learning health systems.
Expected impact
In five years, there will be a new cadre of health system scientists. This group will develop methods of using point-of-care digital data to address priority policy and practice questions in a timely way through both experimental and observational approaches. There will be a corresponding increase in the adoption of new innovations and disinvestment in suboptimal models of care and interventions.
Strategic Priority 2: eHealth
In the upcoming decade, digital platforms will be the backbone of a strategic revolution in the way health services are provided, affecting both healthcare providers and patients (Bahagon and Jacobson 2012). eHealth innovations are appearing in almost all areas of healthcare delivery: from prevention, diagnosis, acute through to long-term care and population health surveillance. Increasing evidence shows its contribution to efficiency (e.g., reductions in wait times, increased speed of referrals and decision-making), effectiveness (e.g., telehealth clinics for dermatology and psychiatric assessment and counselling), patient education and empowerment (e.g., health experience portals) and safety (e.g., prescription drug dispensing) (Elbert et al. 2014).
The emerging potential of eHealth, and its impact on health research, is recognized worldwide, with many funding agencies placing it in the top five priorities for future investment (Viergever 2010).
The gap
Canada is lagging behind in efforts to take full advantage of the global trends in digitization that can transform this innovative knowledge into real benefits for patients and for healthcare systems (Schoen et al. 2012). Analysis of the problems in Canada has identified challenges on all sides (OHIC 2014) that limit the development of practical solutions and the adoption of proven eHealth interventions across clinical, administrative and policy settings. Important limitations include the lack of investment in formal evaluation of new technologies, particularly comparative clinical benefit, effectiveness and comparative cost analysis; insufficient alignment between information and communication technology (ICT) developments and those working to address significant health problems; and the challenges for ICT companies to access healthcare settings where their products and solutions can be tested in real-world contexts with patients and healthcare providers. eHealth innovations of the future will need to be integrated into client-focused solutions that can change outcomes of care by improving access, safety, quality and equity, at the same or lower cost.
The objective
To develop, integrate and evaluate eHealth innovations that will improve the effectiveness and efficiency of patient- and population-centred care; and to increase Canada’s competitive position in the health-related ICT industry to support continuing innovation in Canadian healthcare.
Expected impact
In five years, Canada will have more health innovation communities (local/regional healthcare environments with leadership comprised of researchers, clinicians, patients and decision-makers), which are integrating eHealth innovations into real-world service delivery. These communities will have a dynamic and growing number of technology partners, which are creating and adapting eHealth technologies that reduce the cost of care while increasing access and quality. There will be new international partnerships, and Canadian technology innovators will see the uptake of their products and know-how internationally.
Strategic Priority 3: Healthy Aging in the Community
The Canadian healthcare system is not well designed for chronic disease management, particularly the management of multimorbidity that is most prevalent in the aging population. Canada spends $5,446.50 per capita on healthcare – the fifth highest investment in healthcare among OECD countries (OECD 2015) – with the exception of the US, which has the worst performance in international comparisons.
With the expected demographic shift toward an increasing proportion of older adults, it is paramount that we create communities that can support healthy aging, including health systems that can more proactively manage multimorbidity across the continuum of care.
The gap
Denmark, the Netherlands and Japan are leading in innovative care models to support seniors (British Columbia Ministry of Health and Michael Smith Foundation 2014).
Integrated systems of care involving community-based primary care and home care are an important feature of these innovative systems. However, new models of care have gone beyond the re-configuration of traditional health services to engage communities in providing supportive environments and services for seniors with social innovations, such as age-proof dwellings (e.g., Apartments for Life [Tinker et al. 2013], Dementia Village [Dementia Village Architects 2015]) and volunteer networks (Dementia Friends 2015), SOS Wanderers Network ( Johnson 2015). Regional health authorities in Canada are just beginning to experiment with new community-based models of care for frail seniors (City of Surrey 2014; Park et al. 2014; OMHLTC 2015).
The objective
To accelerate the experimentation and evaluation of community-based integrated care systems and social innovations to support the healthy aging of seniors in the community.
Expected impact
In the next five years, evidence to support policy options and action related to pharmacare, home care and long-term care would be available to support decision-making; new models of care for aging well in the community that delay long-term care admission and reduce avoidable emergency department use and hospitalization will be developed and evaluated.
Strategic Priority 4: Health System Financing, Funding and Sustainability
With healthcare accounting for almost half of provincial and territorial expenditures and delivering poor value for comparative investment internationally (The Commonwealth Fund 2011), it is essential to examine alternative mechanisms of financing and funding and evaluate their comparative effectiveness. In particular, Canada will need to determine how it will finance community-based services that will be essential for effective chronic disease management, but are not covered under the Canada Health Act. Moreover, budget silos for health service sectors along the continuum of care (e.g., hospitals, rehabilitation centres, primary care clinics, home care) act as barriers to innovation and system transformation. Current mechanisms for financing and funding healthcare in Canada provide no incentives for better care at lower cost, improving the patient experience or ensuring the most efficient use of limited resources.
The gap
Various countries, including Canada, are experimenting with a variety of different approaches to financing and funding healthcare. Private–public financing of services (e.g., drug coverage in Quebec) and infrastructure (e.g., new hospitals in Britain) is being employed as a means of improving access and reducing taxpayer costs, but questions about actual effectiveness, efficiency and convenience still remain unanswered (Torchia et al. 2015). There are fears that private–public systems will result in higher healthcare prices and sicker, poorer people being left untreated.
Activity-based funding approaches for hospitals aim to improve efficiencies, but results vary widely across studies: some suggest important benefits and others suggest harmful consequences (Palmer et al. 2014). The impact on the quality of care and outcomes of paying practitioners for performance rather than services remains largely uncertain, particularly as it relates to unintended consequences (Houle et al. 2012). Recent experiments with Accountable Care Organizations (ACO) in the US are of considerable interest in Canada. Within this model, organizations are rewarded for achieving better outcomes and penalized for preventable morbidity, providing an incentive system for front-line innovation in improving health service delivery. The effectiveness of these new models of funding is currently unknown. An emerging approach to improving value for investment in healthcare is through professional engagement and leadership in reducing unnecessary use of resources (American Academy of Family Physicians 2013). The “Choosing Wisely” movement now encompasses the engagement of virtually all medical societies in the US and Canada, as well as Consumer Reports (2015). Choosing Wisely’s impact on reducing preventable morbidity and costs from unnecessary use of drugs, diagnostics and procedures has not yet been evaluated.
The objective
Evaluate alternative approaches to performance-based funding that optimize quality, health outcomes and reduce costs; public–private financing models for providing community-based products and services (e.g., pharmaceutical, home and long-term care, allied health professionals); and new mechanisms for controlling costs through professional leadership and engagement.
Expected impact
In five years, there will be an increase in cross-jurisdictional and international comparative research, which provides evidence about the important attributes of financing and funding that lead to positive and negative effects. Micro-level practice and policy interventions to reduce unnecessary use will be identified and scaled up in some jurisdictions to reduce unnecessary adverse effects and costs.
In summary, Canada has led the world with its pioneering efforts to create innovative cost-effective healthcare systems. The five-year research agenda focuses on key elements that will be necessary to address the challenges of effective health system management of an aging population. These key elements include financing and funding approaches that will either drive or create barriers to innovation; the creation of a new breed of scientists that can collaborate with health system stakeholders; and the co-creation and use of eHealth technologies that can improve the quality and efficiency of care. The Institute of Health Services and Policy Research is one player in this landscape. The creation of the Canadian Health Services and Policy Research Alliance and the national SPOR initiative (CIHR 2011) provides the vehicle and the connectivity to mine the natural experiments in Canadian healthcare and deliver on this ambitious mandate.
Note
- Andreas Laupacis, Li Ka Shing Knowledge Institute of St Michael’s Hospital; Stirling Bryan, Centre for Clinical Epidemiology & Evaluation Vancouver Coastal Health Research Institute; Ivy Bourgeault, University of Ottawa; David Buckeridge, McGill University; Rick Glazier, Institute for Clinical and Evaluative Sciences; Mimi Lowi-Young, Alzheimer’s Society of Canada; Jacques Magnan, Canadian Partnership Against Cancer; Tom Noseworthy, University of Calgary; Amélie Quesnel-Vallée, McGill University; Marcel Saulnier, Health Canada; Vasanthi Srinivasan, Ontario Strategy for Patient-Oriented Research SUPPORT Unit; and Christina Weise, Research Manitoba.
About the Author(s)
Robyn Tamblyn, PhD, Canadian Institutes of Health Research, Institute of Health Services and Policy Research, Ottawa, ON
Meghan McMahon, MSc, Canadian Institutes of Health Research, Institute of Health Services and Policy Research, Ottawa, ON
Jessica Nadigel, PhD, Canadian Institutes of Health Research, Institute of Health Services and Policy Research, Ottawa, ON
Beth Dunning, MA, MLIS, Canadian Institutes of Health Research, Ottawa, ON
Elizabeth Drake, MHA, Canadian Institutes of Health Research, Ottawa, ON
With the Institute of Health Services and Policy Research Advisory Board1
References
American Academy of Family Physicians. 2013. Choosing Wisely: Fifteen Things Physicians and Patients Should Question. Retrieved May 24, 2016. <www.choosingwisely.org/societies/ american-academy-of-family-physicians>.
Backus, L.I., S. Gavrilov, T.P. Loomis, J.P. Halloran, B.R. Phillips, P.S. Belperio et al. 2009. “Clinical Case Registries: Simultaneous Local and National Disease Registries for Population Quality Management.” Journal of the American Medical Informatics 16(6): 775–83.
Bahagon, Y. and O. Jacobson. 2012. eHealth, m-Health and Healthier Social Media Reform: The Big Scale View. London: Poster Abstract. International Congress on Telehealth and Telecare.
British Columbia Ministry of Health and Michael Smith Foundation for Health Research. 2014. Best Practices in Home Care for Seniors: Synthesis Report from the 2014 International Forum. Retrieved May 24, 2016. <www.msfhr.org/sites/default/files/Seniors_Home_ Care_Best_Practices.pdf>.
Canadian Institute for Health Information (CIHI). 2013. National Health Expenditure Trends, 1975-2013. Ottawa, ON: Author.
Canadian Institutes of Health Research (CIHR). 2011. SPOR: Strategy for Patient-Oriented Research. Retrieved May 24, 2016. <www.cihr-irsc. gc.ca/e/41204.html>.
Canadian Institutes of Health Research (CIHR). 2015. Health Research Roadmap II: Capturing Innovation for Better Health and Health Care. Retrieved May 24, 2016. <www.cihr-irsc. gc.ca/e/48964.html>.
Canadian Institutes of Health Research Institute of Health Services and Policy Research. 2015. Pan-Canadian Vision and Strategy for Health Services and Policy Research: 2014–2019. Retrieved May 24, 2016. <www.cihr-irsc.gc.ca/e/47945.html>.
City of Surrey. 2014. Innovation Boulevard. Retrieved May 24, 2016. <www.surrey.ca/business-economic-development/13158.aspx>.
Committee on the Learning Health Care System in America, Institute of Medicine. 2013. in M. Smith, R. Saunders and L. Stuckhardt (Eds.), Best Care at Lower Cost. The Path to Continuously Learning Health Care in America. Washington, DC: National Academies Press.
Consumer Reports Health. 2015. The Choosing Wisely Campaign: Educating Consumers about Appropriate Care. Retrieved May 24, 2016. <consumerhealthchoices.org/campaigns/ choosing-wisely/>.
Dementia Friends – an Alzheimer’s Society Initiative. 2015. Home Page. Retrieved November 4, 2015. <www.dementiafriends.org.uk/2015>.
Dementia Village Architects. 2015. Dementia Village. Retrieved November 4, 2015. <http://dementiavillage.com>.
Elbert, N.J., H. van Os-Medendorp, W, van Renselaar, A.G. Ekeland, L. Hakkaart-van Roijen, H. Raat et al. 2014. “Effectiveness and Cost-Effectiveness of ehealth Interventions in Somatic Diseases: a Systematic Review of Systematic Reviews and Meta-Analyses.” Journal of Medical Internet Research 16(4): e110.
Gawande, A. 2007. Better: A Surgeon’s Notes on Performance. London, UK: Picador.
Gooch, K., D.A. Marshall, P.D. Faris, H. Khong, T. Wasylak, T. Pearce et al. 2012. “Comparative Effectiveness of Alternative Clinical Pathways for Primary Hip and Knee Joint Replacement Patients: a Pragmatic Randomized, Controlled Trial.” Osteoarthritis Cartilage 20(10): 1086–94.
Health Services Research Europe. 2011. Health Services Research: Helping Tackle Europe’s Health Care Challenges. Policy Brief. Retrieved May 24, 2016. <www.nivel.nl/ pdf/Policybrief-health-services-research.pdf>.
Houle, S.K., F.A. McAlister, C.A. Jackevicius, A.W. Chuck and R.T. Tsuyuki RT. 2012. “Does Performance-Based Remuneration for Individual Health Care Practitioners Affect Patient Care?: A Systematic Review.” Annals of Internal Medicine 157(12): 889–99.
James, B.C. and L.A. Savitz. 2011. “How Intermountain Trimmed Health Care Costs through Robust Quality Improvement Efforts.” Health Affairs 30(6): 1185–91.
Johnson, S. 2015. “There Is Still a Shocking Level of Ignorance Regarding Dementia.” The Guardian. Healthcare Network. Retrieved May 24, 2016. <www. theguardian.com/healthcare-network/2015/may/29/ ignorance-dementia-health-social-care>.
Klein, D.J., A.D. Brown, T.M. Huynh, G. Bevan, F. Markel, S.D. Ottaway et al. 2013. Capital Spending in Healthcare: a Missed Opportunity for Improvement? Ottawa, ON: Canadian Foundation for Healthcare Improvement.
Ontario Health Innovation Council (OHIC). 2014. The Catalyst towards an Ontario Health Innovation Strategy. Retrieved May 24, 2016. <http://www. ohic.ca/sites/default/files/Ontario%20Health%20 Innovation%20Council%20Report.pdf>.
Ontario Ministry of Health and Long-Term Care (OMHLTC). 2016. Transforming Ontario’s Health Care System: Community Health Links Provide Coordinated, Efficient and Effective Care to Patients with Complex Needs. Retrieved May 24, 2016. <www.health.gov.on.ca/ en/pro/programs/transformation/community.aspx>.
Organisation for Economic Co-Operation and Development (OECD). 2015. Health Expenditure and Financing. Retrieved July 30, 2015. <http://stats oecd org/index aspx?DataSetCode=HEALTH_STAT>.
Palmer, K.S., T. Agoritsas, D. Martin, T. Scott, S.M. Mulla, A.P. Miller et al. 2014. “Activity-Based Funding of Hospitals and Its Impact on Mortality, Readmission, Discharge Destination, Severity of Illness, and Volume of Care: a Systematic Review and Meta-Analysis.” PLoS ONE 9(10): e109975.
Park, G., D. Miller, G. Tien, I. Sheppard and M. Bernard. 2014. “Supporting Frail Seniors through a Family Physician and Home Health Integrated Care Model in Fraser Health.” International Journal of Integrated Care 14: e001.
Schoen, C., R. Osborn, D. Squires, M. Doty, P. Rasmussen, R. Pierson et al. 2012. “A Survey of Primary Care Doctors in Ten Countries Shows Progress in Use of Health Information Technology, Less in Other Areas.” Health Affairs 31(12): 2805–16.
Schoen, C., R. Osborn, D. Squires and M.M. Doty. 2013. “Access, Affordability, And Insurance Complexity Are Often Worse in the United States Compared to Ten Other Countries.” Health Affairs 32(12): 2205–15.
The Commonwealth Fund. 2011. International Profiles of Health Care Systems. 2011. Retrieved May 24, 2016. <www.commonwealthfund. org/publications/fund-reports/2011/nov/ international-profiles-of-health-care-systems-2011>.
Tinker, A., J. Ginn and E. Ribe. 2013. “Assisted Living Platform – the Long Term Care Revolution: a Study of Innovatory Models to Support Older People with Disabilities in the Netherlands.” Retrieved May 24, 2016. <www.ifa-fiv.org/wp-content/uploads/2013/11/ HLIN-Case-Study76-Netherlands.pdf>.
Torchia, M., A. Calabro and M. Morner. 2015. “Public-Private Partnerships in the Health Care Sector: A Systematic Review of the Literature.” Public Management Review 17(2): 236–61.
Viergever, R. 2010. Health Research Prioritization at WHO. An Overview of Methodology and High Level Analysis of WHO Led Health Research Priority Setting Exercises. R. Terry and M.P. Matsoso (Eds.). Geneva, Switzerland: World Health Organization.
World Health Organization (WHO). 2008. Scaling UP Health Services: Challenges and Choices. WHO Technical Brief #3. Retrieved May 24, 2016. <www. who.int/healthsystems/topics/delivery/technical_ brief_scale-up_june12.pdf>.
World Health Organization (WHO). 2013. Transforming and Scaling Up Health Professionals’ Education and Training. World Health Organization Guidelines. Retrieved May 24, 2016. <www.who.int/ hrh/resources/transf_scaling_hpet/en/>.
Comments
Be the first to comment on this!
Personal Subscriber? Sign In
Note: Please enter a display name. Your email address will not be publically displayed