HealthcarePapers 17(2) October 2017 : 46-49.doi:10.12927/hcpap.2017.25410

More than just Measurement

Malcolm B. Doupe

I have reviewed with pleasure the article by Kuluski et al. (2017) who posit that measures of patient experience are required to more effectively guide healthcare reform. While I am generally in support of the original paper, I argue in this commentary that: (1) measuring patient engagement experience should not be done in isolation from broader change management processes; (2) care must be taken to ensure that measures of patient experience are developed with rigor and do not further complicate the already vast performance metrics literature and, (3) any revised set of performance metrics requires ongoing evaluation, to help ensure its optimal value.

As noted in the Canadian Institutes of Health Research (CIHR) Institute of Health Services and Policy Research Strategic Plan (CIHR 2014a), Canada now leads healthcare expenditures among Organisation for Economic Co-operation and Development (OECD) countries, and – using data from Manitoba as an example – provincial healthcare spending has increased by 26.5% in the last four years, from $4.9 billion in 2012 (Manitoba Health 2013) to $6.2 billion in 2016 (Manitoba Health 2017). Across Canada, the major drivers of this change are not related to patients per se, but are rather attributed to the manner in which healthcare is delivered (e.g., the increasing use of medical technology) (CIHI 2011; Lee 2007). It is also important to note that, despite this increase in healthcare expenditure, Canada ranks poorly among OECD countries using various performance metrics including the quality of patient care (Schoen et al. 2013). Our own history tells us that further healthcare reform without thoughtful discussion (i.e., what CIHR calls a Learning Health System) (CIHR 2014a) is likely a poor direction forward. Broadly speaking, patients have a clear role to play in this innovation process.

I agree with the authors' fourth summary statement (i.e., that engagement-capable environments are required to drive change), and from this perspective argue that developing metrics of patient experience provides a partial reform solution. The literature is replete with information showing the gap between what patients want and get. While people want to use nursing homes as a last resort only (Quine and Morrell 2007), upwards of 20% of new nursing home admissions may be premature (Doupe et al. 2012; McNabney et al. 2007; Mor et al. 2007). While many of the operational standards used to evaluate nursing homes are medically oriented, factors such as dignity, spiritual well-being and food enjoyment are most important to residents (Burack et al. 2012). Emergency department providers feel that downstream factors (e.g., the lack of hospital beds) most strongly impede patient flow (Bond et al. 2007; Cass 2005), while patients feel that most of their time is spent waiting for tests (Swancutt et al. 2017). The point of these examples is to demonstrate that the various players of our healthcare system have different opinions about what is important and what needs to change. While measurement is an important aspect of change management, effective innovation first requires more thoughtful and integrated discussions about what we are trying to achieve and why. I would argue that finding ways to engage with patients more effectively as partners – and not just measuring their experience – is required to drive healthcare change. This statement is in keeping with the Mode 2 philosophy of research which posits that effective change must be jointly approached by partnership teams if sustainability and spread is the goal (Gibbons 2014).

In their review of the related literature, Sunderji et al. (2017) report 148 "unique" quality care measures used to evaluate integrated care models for patients with mental health challenges (Sunderji et al. 2017). It is difficult to imagine any scenario whereby this volume of data could be used effectively, and within healthcare there is a dire need to develop a streamlined list of achievable performance measures. It also seems obvious to state that patient experience metrics should be used to augment (and not replace) the more traditional measures of costing, effectiveness and adverse events, which collectively cannot be ignored. Third, in keeping with Rycroft-Malone et al. (2013), who posit that various perspectives (research, provider and patient) are needed to properly guide clinical decision-making, performance metrics emerging from partnership discussions would seem to have greater value. These arguments, however, come with a strong cautionary note. Without careful planning and consensus building, the creation of patient experience measures has the potential to further complicate the already vast performance metric literature. Similarly, for whatever the measures chosen, more standardized methods of data collection and reporting are also required (e.g., to make comparisons across jurisdictions, providers, institutions and patient groups; to measure improvements over time). Just as provincial ministries have a mandate to guide healthcare reform, I would argue that these agencies have a responsibility to lead the process (perhaps in concert with federal agencies such as the Canadian Institute for Health Information) of metric development and standard reporting. This includes allocating sufficient resources required to record these data in a centralized system, for reporting purposes.

My final comment pertains to the need to evaluate the process of patient engagement including performance metrics. Some authors have developed guiding principles to enhance provider-patient engagement strategies (Keddem et al. 2017; Pushparajah 2017), and some literature shows that more engaged patients have better clinical outcomes (Peters and Keeley 2017). Brett et al. (2014) report that patient engagement in research leads in most but not all instances to increased feelings of patient empowerment and satisfaction, and also to enriched researcher knowledge of salient research questions (Brett et al. 2014a). CIHR has also developed a patient engagement framework that outlines key guiding principles and provides some metrics of success (CIHR 2014b). Important questions, however, remain. What actions do we put in place to ensure that patients have meaningful input and are empowered to have an equal partnership voice at the table? Just as meaningful change requires effective provider leadership, what guidelines exist to select effective patient team members, and what is the best way to build this capacity? How should patient engagement processes be evaluated to demonstrate their value and to highlight areas where ongoing work is needed? In the same way that evidence is an integral part of the change management process, so too is evidence required to assess the value of patient engagement including the performance metrics that emerge. As one example of a future research initiative, researchers could apply the CIHR patient engagement framework to the process of developing new nursing home standards (which currently are provincially based and mostly medical in nature), compare how the new standards emerging from this process differ from those more traditionally used and discuss how their implementation could impact nursing home care (e.g., by balancing the need to provide high quality medical and social care).

Making change requires knowledge about the problems that exist and also about effective and sustainable reform strategies. In an era of cost containment, many reform activities emphasize managing costs and/or improving system efficiency. It is here that patient engagement has the strongest potential to ensure that reform strategies focusing on efficiency do not impede the quality of the patient experience. Ironically, this is also our largest challenge. Reaching consensus (across planners, providers, patients, jurisdictions) both on effective reform strategies and on a (streamlined) set of appropriate performance measures is a daunting but worthwhile task.

About the Author(s)

Malcolm B. Doupe, PhD, Associate Professor, Department of Community Health Sciences, Senior Research Scientist, Manitoba Centre for Health Policy, Director, Manitoba Training Program for Health Services Research, Max Rady College of Medicine, Rady Faculty of Health Sciences, University of Manitoba, Winnipeg, MB


Bond, K., M.B. Ospina, S. Blitz, M. Afilalo, S.G. Campbell, M. Bullard et al. 2007. "Frequency, Determinants and Impact of Overcrowding in Emergency Departments in Canada: A National Survey." Healthcare Quarterly 10(4): 32–40.

Brett, J., S. Staniszewska, C. Mockford, S. Herron-Marx, J. Hughes, C. Tysall and R. Suleman. 2014. "A Systematic Review of the Impact of Patient and Public Involvement on Service Users, Researchers and Communities." Patient 7(4): 387–95. doi:10.1007/s40271-014-0065-0.

Burack, O.R., A.S. Weiner, J.P. Reinhardt and R.A. Annunziato. 2012. "What Matters Most to Nursing Home Elders: Quality of Life in the Nursing Home." Journal of the American Medical Directors Association 13(1): 48–53. doi:10.1016/j.jamda.2010.08.002

Canadian Institute for Health Information (CIHI). 2011. Health Care Cost Drivers: The Facts. Ottawa, ON: Canadian Institute for Health Information. Retrieved November 15, 2017.<>.

Canadian Institutes of Health Research (CIHR). 2014a. CIHR Institute of Health Services and Policy Research Strategic Plan 2015–19. Retrieved November 27, 2017. <>.

Canadian Institutes of Health Research (CIHR). 2014b. Strategy for Patient-Oriented Research - Patient Engagement Framework. Retrieved November 27, 2017. <>.

Cass, D. 2005. "Once Upon a Time in the Emergency Department: A Cautionary Tale." Annals of Emergency Medicine 46(6): 541–43.

Doupe, M., P. St. John, D. Chateau, D. Strang, S. Smele, S. Bozat-Emre et al. 2012. "Profiling the Multidimensional Needs of New Nursing Home Residents: Evidence to Support Planning." Journal of the American Medical Directors Association 13(5): 487–17.

Gibbons, M. 2014. "Mode 2 Society and the Emergence of Context-Sensitive Science." Science and Public Policy 27(3): 159–63.

Keddem, S., A.Z. Agha, J.A. Long, R.M. Werner and J.A. Shea. 2017. "Creating a Toolkit to Reduce Disparities in Patient Engagement." Medical Care 55(Suppl. 9, Suppl. 2): S59–69. doi:10.1097/mlr.0000000000000748.

Kuluski, K., M.L.A. Nelson, C.S. Tracy, C.A. Alloway, C. Shorrock, S. Shearkhani et al. 2017. "Experience of Care as a Critical Component of Health System Performance Measurement: Recommendations for Moving Forward." Healthcare Papers 17(2): 8–20. doi:10.12927/hcpap.2017.25415.

Lee, M. 2007. How Sustainable is Medicare? A Closer Look at Aging, Technology and Other Cost Drivers in Canada's Health Care System. Retrieved November 15, 2017. <>.

Manitoba Health. 2013. Annual Report 2012–2013. pp. 1–98. Retrieved November 24, 2017. <>.

Manitoba Health. 2017. Manitoba Health, Seniors and Active Living Annual Report 2016–17. Retrieved November 24, 2017. <>.

McNabney, M.K., J.L. Wolff, L.M. Semanick, J.D. Kasper and C. Boult. 2007. "Care Needs of Higher-Functioning Nursing Home Residents." Journal of the American Medical Directors Association 8(6): 409–12.

Mor, V., J. Zinn, P. Gozalo, Z. Feng, O. Intrator and D.C. Grabowski. 2007. "Prospects for Transferring Nursing Home Residents to the Community." Health Affairs (Millwood) 26(6): 1762–71.

Peters, A.E. and E.C. Keeley. 2017. "Patient Engagement Following Acute Myocardial Infarction and Its Influence on Outcomes." American Journal of Cardiology 120(9): 1467–71. doi:10.1016/j.amjcard.2017.07.037.

Pushparajah, D.S. 2017. "Making Patient Engagement a Reality." Patient 11(1): 1–8. doi:10.1007/s40271-017-0264-6.

Quine, S. and S. Morrell. 2007. "Fear of Loss of Independence and Nursing Home Admission in Older Australians." Health and Social Care in the Community 15(3): 212–20.

Rycroft-Malone, J., K. Seers, J. Chandler, C.A. Hawkes, N. Crichton, C. Allen et al. 2013. "The Role of Evidence, Context, and Facilitation in an Implementation Trial: Implications for the Development of the PARIHS Framework." Implementation Science 8: 28. doi:10.1186/1748-5908-8-28.

Schoen, C., R. Osborn, D. Squires and M.M. Doty. 2013. "Access, Affordability, and Insurance Complexity are Often Worse in the United States Compared to Ten Other Countries." Health Affairs (Millwood) 32(12): 2205–15. doi:10.1377/hlthaff.2013.0879.

Sunderji, N., A. Ion, A. Ghavam-Rassoul and A. Abate. 2017. "Evaluating the Implementation of Integrated Mental Health Care: A Systematic Review to Guide the Development of Quality Measures." Psychiatric Services 68(9): 891–98. doi:10.1176/

Swancutt, D., S. Joel-Edgar, M. Allen, D. Thomas, H. Brant, J. Benger et al. 2017. "Not all Waits are Equal: An Exploratory Investigation of Emergency Care Patient Pathways." BMC Health Services Research 17(1): 436. doi:10.1186/s12913-017-2349-2.


Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed