A prevailing feature of recent healthcare delivery and reform initiatives is a focus on increasing the value provided by investment in services, alongside a more nuanced understanding of how such value should be considered. Effective measurement of this value remains an elusive goal for most health system performance assessment (HSPA) systems. A more prominent role for the patient voice can enable a better understanding of value at both patient and population levels. The Tuscan HSPA model has evolved over the past several years by adopting the perspective of service users, including multiple dimensions of performance, and illustrating the interactions of these elements. For the heart failure pathway, this approach has now been further developed to combine these dimensions with the systematic electronic collection of patient-reported outcome measures and patient-reported experience measures – initially in a specialist hospital. This enables a richer understanding of the value delivered by professionals as they operate in reality, as opposed to by organizational boundaries, and more timely and actionable insights into the drivers of that value. This commentary sets out the latest developments in the Tuscan HSPA and the lessons from implementation.

Introduction

In the context of growing aging populations and increasing complexity of needs, the long-term financial sustainability of healthcare systems is now a major concern for many advanced economies. A number of approaches have been adopted or trialled globally in response to this challenge.

One approach of interest at senior levels in many health systems is the broad concept of value-based care, perhaps most famously described by Porter as the focus on outcomes for a patient compared to cost, rather than the amount of inputs(Porter 2010). Of more interest in the European context – where national healthcare systems or their ilk are prevalent – is the model proposed by Muir Gray (2013), which captures not just how efficiently a given intervention is conducted but also the real benefit of that intervention for a person, and the relative benefit of the intervention compared to something else that might have been funded from the budget attached to that population. In this way, the concept of value is applied to a whole population and takes account of equity within a set funding envelope. The result is to consider for every medical intervention not just whether it is done safely and efficiently but also whether it is right for this person, in this time and in this setting – and whether this is the best use of funding, all things considered. As noted by Muir Gray (2013), this is a natural intellectual progression from Donabedian's model – chiefly, that more medicine does not necessarily equate to better outcomes.

It is notable that although nuanced conceptions of value are leading in academic and policy circles, these have barely entered the public and political discourse around health services. In part, this may be due to the relative conceptual complexity, but perhaps, also, due to the inevitable progression to practicalities: reduced choice and what might be termed rationing of care (Oliver 2019) – topics of little appeal for elected officials. It is an unanswerable question whether this is to the detriment of more productive debate and effective reforms.

Nonetheless, this description of value has influenced many policy makers and professionals, who developed or increased the focus on initiatives coherent with this approach: reducing avoidable variation within and between geographic areas, reducing procedures of low clinical value, increasing patient involvement in discussions around preference-sensitive care and encouraging clinicians to consider the totality of a patient's needs rather than the biomedical symptoms (e.g., Briggs 2015). In this way, the concept of value becomes practically entwined with that of patient-centred care as the aim is to deliver what is right for the patient in that setting and in that time.

The Role of the Patient Voice

A true patient-centred model requires a greater role for the patient voice in the delivery of healthcare. Accordingly, these initiatives have – alongside patient representatives and rights activists – increased efforts to ensure staff and processes capture and reflect this voice. There is now an ever-increasing use of both quantitative and qualitative methods to capture patients' views, with new survey instruments to facilitate this, typified by patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) (Black et al. 2016; Coulter et al. 2014).

Although substantially utilized in a relatively small group of countries when considered against other performance measures, these tools are used in multiple settings across research and practice, particularly to measure the impact of interventions (e.g., orthopedic surgery), for benchmarking, to measure patient experience or as disease-specific measures in clinical registries (Williams et al. 2016). Although there is some investigation of the use of PROMs in remotely monitoring chronic patients, overall there is less focus on PROMs and PREMs for chronic care (Berwick et al. 2017). In addition, there is limited evidence that patient-reported information is considered a source of data alongside clinical information and administrative data to enable more holistic performance assessment and to identify and evaluate improvement actions. Although the increasing collection of patient-reported data is to be celebrated, this does not always translate into effective use of that data (Coulter et al. 2014; Van Der Wees et al. 2014).

PROMs could have a major role in more effectively assessing value at both individual and population levels: measuring whether a patient has an improved health status in response to an intervention, enabling comparisons of the gains in quality of life between one intervention and another and considering the cost associated with each. Alongside this, the experience of patients (and of staff) can be measured both as an important indicator of performance in its own right and as reflecting the understanding that experience can influence outcomes and costs. Such data can also be used to support professional decision making, relationship management and improvement actions, enabling value to accrue to providers. In these ways, it is possible to measure, incentivize and support delivery of triple-value healthcare at personal, technical and allocative levels (Jani et al. 2018).

The Italian Experience of Measuring Value

The systematic collection at scale of PROMs and PREMs – and the use of this information at different levels in the health system – could help deliver this value (Berwick et al. 2017). In Tuscany, there has been a focus for some years on patients' perspectives within health system performance assessment (HSPA) and a combination of measures that approaches a picture of triple value. Established in 2004 through a partnership between the MeS Lab of Sant'Anna School of Advanced Studies and the regional health system in Tuscany, the Italian Regional Performance Evaluation System (IRPES) now includes 11 regions and contains over 300 indicators for multiple performance dimensions: population health status, progress against priorities, clinical performance, efficiency and financial performance, patient satisfaction and staff satisfaction (Nuti et al. 2017, 2018). Where there is no agreed "right" level for an indicator, measures are reported without evaluation; others are classified in bands according to performance compared to a benchmark (regional, or from literature) and according to the direction of travel (Nuti et al. 2013, 2016).

Over the past few years, the IRPES has increasingly framed performance around the patient rather than around organizations. The case of the maternity HSPA, described in this journal (Nuti et al. 2017), illustrates this well: performance is measured and calculated for several points along the pathway, pre- and post-partum; the contributions of multiple providers in delivering care are considered together; patient-reported measures are included alongside administrative measures; and staff experience is included as an important measure of value. In addition, these measures are calculated dynamically against absolute and relative performance and can be presented in the "stave," which enables easier diagnosis and action from administrators and policy makers. This information – and patient-reported data for other clinical areas – is included in HSPA presentations and annual reports for Italian regions and the public (available at http://performance.sssup.it/netval/start.php). This disclosure approach draws on reputational levers to stimulate improvements in performance and is being used to set goals and rewards for CEOs in Tuscany. The included indicators are selected through discussion between researchers and clinicians – based on validated international surveys – enabling comparisons across providers and countries, although an international benchmarking process is not yet in place.

Latest Developments in Measuring Value in Tuscany

This performance assessment model has proven effective at engaging professionals and policy makers in the Tuscan system and elsewhere. Initiatives are now under way to increase the measurement and utilization of patient-reported data, using a model seeking to enhance its uses as a management tool, as an evaluation mechanism and at the policy level.

The systematic digital collection of patient-reported data is now being implemented for several clinical areas, for both PROMs and PREMs. This model is noteworthy for a number of reasons:

• Data are collected continuously, rather than periodically.
• All patients are eligible and can be enrolled in the survey.
• Digital administration enables improved data accuracy, low cost and fast and simple aggregation and analysis of data.
• The large data set means that information can be reported against different levels in participating providers, down to ward level, enabling a more granular understanding of performance and more sensitive diagnoses of strengths and weaknesses.
• The unique link is sent upon discharge, and the survey is completed at patients' leisure and in a setting of their choice.
• Several free-text experiential questions are included alongside the quantitative scales to enable richer data capture at different points in the patient journey, including the opportunity to highlight good care by specific individuals – which are vetted and shared as written, available for use at ward meetings.

A more thorough discussion of the systematic collection model for PREMs (which follows the same principles as for PROMs) is available from De Rosis (under review).

A pilot program using this model is focused on heart failure, where data are collected from patients across both outcome and experience areas. A combination of established validated surveys and new instruments to suit the specific context is used, including the Kansas City Cardiomyopathy Questionnaire, Self-Care of Heart Failure Index and a set of experiential items used for many years in Tuscany, based on Picker Institute questions. In addition to a survey at discharge, patients with heart failure are sent additional links at 1, 7 and 12 months following discharge from hospital. This provides a longitudinal data set that tracks the patients' reported health and well-being even when they are not being actively monitored by health professionals and can capture the contribution of community-based providers to patient-reported outcomes.

This model was introduced in 2018 at the Fondazione Toscana Gabriele Monasterio, a specialist hospital for cardiovascular diseases in Pisa, Tuscany. As of summer 2019, 202 patients have been enrolled, with 127 responding to the survey at discharge (63% response rate), 108 at month 1 (61% response rate), 73 at month 7 (47% response rate) and 41 at month 12 (48% response rate).

Lessons from the Pilot

A key feature of the pilot that is regarded as critical to the implementation success so far is the deep involvement and leadership from clinicians at all stages of the pilot. This involvement is recognized by all parties as critical to ensuring professional ownership and as low burden a methodology as possible. Notable results from this approach are as follows:

1. a positive reception from hospital executives, recognizing that the program is clinically led and has the potential to support improvement initiatives (although currently in the experimental phase rather than mainstreamed in reporting and evaluation);
2. selection of measures that have the support of clinicians;
3. development of a patient enrolment process that draws on existing data and processes as far as possible and reduces staff burden, including an application program interface that enables automatic sharing of a survey link after discharge; total staff time required to explain the program and record patient details is less than five minutes per patient, which is not viewed as a barrier to inclusion; and
4. operational changes to improve enrolment rates: real-time data aggregation and presentation enabled researchers to identify high variation in enrolment rates over time. After investigation, this issue was attributed to the training and staffing model of cardiologists (requiring rotations and therefore less consistent knowledge of the pilot). In response, responsibility for enrolling patients was shifted to a lead nurse in each ward, resulting in an increase in and stabilization of patient enrolment rates.

These factors, alongside the technical model described earlier, help overcome the typical limitations in using patient experience data to drive change (Flott et al. 2017), as well as bring in additional potential benefits by integrating this with PROM data, for example, enabling analysis of longitudinal data to explore relationships between experience and reported outcomes, and develop appropriate operational responses. This type of research would build on the increasing understanding of the link between experience, clinical outcomes and cost (Anhang Price et al. 2014) and could identify practical lessons for healthcare providers seeking to improve performance in these domains.

Other features are more context specific. Notably, Fondazione Monasterio is a research hospital where the staff are accustomed to new initiatives aiming to improve patient care; the data collection system described is not viewed as a distraction from their daily work but as a useful addition. There was also some previous experience of PROMs, although only occasionally for specific programs and using paper-based surveys. In addition, the hospi­tal is regarded as high performing nationally; clinicians and execu­tives have low aversion to additional scrutiny. The new model of patient-reported data collection therefore offered an improvement on previously available ad hoc approaches, a new source of data that may help in hospital-led improvement initiatives and leadership in patient-reported data collection in Italy.

Presentation of the model and early results to other cardiologists has been positive. Through their impetus, a number of other hospitals are now adopting the data collection model.

The most important feature of the model is early presentation of the results to professionals. A web platform includes real-time aggregate data (small number suppression to preserve anonymity) in simple graphical representations, as well as free-text comments. By avoiding a long period of data collection with no ability to share results with the staff, a clear link between action and outcome is created, and patient enrolment is incentivized – as well as behaviours leading to more positive scores. The ability to review data has been positively received by clinicians, nurses and administrative staff. The PREMs element is typically of most interest to ward staff, particularly the narrative element that enables staff to be valued and recognized individually and collectively.

Clinicians have noted the value in the range of data and that it supports more person-centred care; health and well-being items serve as a reminder to focus on the whole person, not only biomedical markers. Clinicians also regard the program as beneficial for patients, as filling in the questionnaire encourages them to take self-care more seriously and to consider their health status outside of the care they receive for heart failure (data are not yet available to evaluate this).

The primary weaknesses of the model are associated with the need to manage patient data securely and within the Italian legal framework. Most notably, currently, patient-reported data are not embedded within the electronic health record and are managed under a separate arrangement.

In addition, although a low burden, it is not a fully automated process. A critical point is for the responsible physician to inform the lead nurses that a new patient is present who can be enrolled; some patients could be missed at this point. Exploratory thinking is under way to fully automate this process, for example, by auto-flagging patients with certain International Classification of Diseases (ICD) codes for enrolment.

Finally, the web platform's functionality in illustrating outcomes and experience data is a major factor in ensuring enthusiasm for the model and enabling subsequent action. Embedding the review of these data into everyday working requires changes in behaviour and can be challenging to sustain. There is no natural and regular forum for this action: the responsible clinician and the unit director regularly review all data and comments, but the wider team relies on periodic presentations to share data. This limits opportunities for rapid responses to issues identified by patients.

Next Steps and Further Research

An immediate use of the model is to include data in the Tuscan HSPA, enabling a better understanding of the (triple) value delivered. This assessment can be used by policy makers and administrators to identify areas of focus, as well as for incentive schemes. Even in the absence of additional incentives, benchmarking and public reporting of data (as in the IRPES) are known to lead to better performance (Bevan et al. 2019). However, this is not without challenge: data may need adjustment for patient characteristics before these are comparable; non-respondents need to be considered; and only selected items or a compound indicator can be included in visualizations, which must be chosen. Given the novelty of this approach, these issues have not yet been resolved elsewhere. Work is ongoing to develop the methodology for inclusion of patient-reported data as key metrics in the system-wide HSPA. After the technical questions, there are some policy implications for those using the HSPA: determining the benchmark or "right" level of attainment; considering which levers could be applied to encourage strong performance; the weighting of patient-reported data against administrative data; and balancing priorities identified related to improving patient-reported scores with those focused on other domains. These are not minor issues and will likely be addressed iteratively, reflecting the significance of the paradigm shift toward considering triple value integrally in HSPA. The challenge for partners in this endeavour is to maintain enthusiasm and impetus during this transitional period.

These data will also inform a more granular analysis of the relationships between clinical care, community-based care, patient experience and patient-reported outcomes. This analysis can help identify the behaviours or ways of working most strongly associated with the best experience and outcomes. This includes the opportunity to overlay patient-reported data onto primary-care referral networks (Stukel et al. 2013), which can then be used to assess new units of accountability according to working patterns across multiple providers (Casalini et al. 2018; Nuti et al. 2019, in press). This opens up new opportunities for performance improvement and delivery of high-value healthcare, for example, in considering costs for a cohort of registered patients against reported outcomes as delivered by both secondary and primary care providers.

Conclusion

The integration of PROMs and PREMs with administrative and clinical data in HSPA models can provide a better indication of performance against the triple aim than traditional measurement systems with fewer dimensions. Using systematic, continuous and digital collection models for PROMs and PREMs provides data that better integrate with routinely collected information and may provide more opportunities for improvement actions or policy initiatives.

The system adopted in Tuscany provides a useful model for other systems seeking to increase representation of the patient voice in performance management. Critical factors in successful implementation include consistent clinical ownership and rapid presentation of results, whereas greater value would be derived from the model if accessible at the patient level.

Performance assessment models will increasingly measure the value delivered by groups of providers working together. Professionals at all levels should draw on these systems to identify and assess opportunities to deliver better outcomes for patients.

Comments

Be the first to comment on this!