Much of health services and policy research is applied. We offer four provocations to stimulate thinking about the relationship between research and the "systems" it aims to influence. We conclude that a focus on partnership implies that researchers need to be empathetic to the timelines and needs of policy makers, while true relationships are bidirectional. Focusing on the priorities of "systems" will emphasize short-term issues. This leads to researchers often conducting post-implementation evaluation, where they have had little involvement in policy or intervention design. Finally, a focus on single-project return of investment will tend to undervalue riskier – but also potentially more rewarding – research.

Introduction

Health services and policy research is the field of scientific investigation that generates evidence on how to create systems, policies and organizational structures and invest in programs, services and technologies that maximize health and health care system outcomes. (CIHR IHSPR 2021: 4)

In the newly-launched Canadian Institutes of Health Research – Institute for Health Services and Policy Research's (CIHR IHSPR's) strategic plan, the first strategic priority focuses on health systems changes that will "… achieve the Quadruple Aim and improve health equity for all" (CIHR IHSPR 2021: 11). The plan justifies this priority on the grounds that healthcare systems have continuously increasing costs and yet are not meeting all current needs. A lack of system optimization leads to many inefficiencies, including, for example, the high use of "alternate level of care" beds in hospitals (Lavergne 2015), people forgoing care either due to cost or lack of access (Horrill et al. 2019; Law et al. 2012) and health workforce challenges, such as high turnover and burnout. (Stelnicki et al. 2021).

There are, as the IHSPR strategic plan notes, "pockets of excellence" (CIHR IHSPR 2021: 12) and the COVID-19 pandemic both highlighted systemic weaknesses, overall and everywhere, and created new challenges. The pandemic also created the impetus for, and importantly the realization of, some significant policy changes. For example, the system introduced virtual care nearly overnight (Bhatia et al. 2021), after slow, sporadic and not-very-patient-centric progress for the previous decade (McGrail et al. 2017). We have seen what can be done when there is collective focus and clear need. Recovery from the effects of COVID-19 will require long-term effort, drawing upon these same kinds of interdisciplinary and multistakeholder collaboration and policy acceleration that happened over the past two years.

Health services and policy research (HSPR) can make significant contributions both to pandemic recovery and to future health system strengthening. Some of this will come through the benefits of health services and policy research that is highly responsive to the declared immediate needs of decision makers. There are, however, risks to putting too many of our eggs in that one specific basket. The IHSPR strategic plan has many aspirations, and the success of HSPR over the coming years will depend on how that plan is implemented, how emphasis is given to its different priorities, how strategic funding is allocated to those priorities and how HSPR is seen and supported more broadly within CIHR.

With that in mind, in this article we outline four provocations to stimulate thinking about the relationship between HSPR and healthcare systems. In particular, we discuss the strengths of HSPR and the stated goal of partnerships and relationships between HSPR practice and healthcare systems planning and delivery. Our contention is that the current focus on "partnership" in the IHSPR strategic plan and elsewhere implies that HSPR needs to do most of the moving if it is to be of greater use to systems. We contend that while HSPR should indeed be prepared to evolve and respond to system needs, so, too should systems reflect on how each might improve its capacity to value both the theoretical and empirical products of HSPR.

Four Provocations

The need to challenge systems

Fundamentally, HSPR assesses system performance and seeks ways to improve. The difficulty in receiving "constructive criticism" is well documented, yet this is crucial for HSPR to achieve impact (Gnepp et al. 2020; Hardavella et al. 2017). Elements that make constructive criticism more likely to be well received include the context of trusted relationships, often built over many years. The focus on embedded researchers and responding to short-term priorities are two effective strategies to building these relationships. An embedded researcher who is able to transition seamlessly between the cultures of policy making and research creates a bridge across the two siloes; this person can help interpret the needs, translate the language and help the two distinct cultures see value in each other. This will only enhance the understanding of the importance and, over time, the impact of HSPR. In addition, responding to the short-term policy priorities of today strengthens the relationship between HSPR and policy. Helping a colleague respond to the crisis in front of them builds trust.

In the end, however, strong relationships are bidirectional and need to be built to enable and support challenging conversations about big systemic issues, including the failures of current practice. Without two-way trust, and the ability for HSPR practitioners to ask something of systems – equal to system partners asking things of HSPR – the relationship is weak and provisional at best. In that case, it would be better described as a one-way bargain in which HSPR practitioners can contribute only on terms set by the system.

We do not suggest that any individual in health systems wakes up in the morning with ill intent toward health services and policy researchers. The power dynamic, however, is that we as HSPR practitioners increasingly rely on partnerships for success with research grants, which brings us closer to the system, but also increases the risk or at least a perceived risk – to future projects or funding or relationships – of saying anything that might be seen as critical.

There are two possible solutions to this. One is that we accept that it is okay, or even desirable, to have researchers who do not have "friends" in the system. Critics are an important aspect of improvement, and not all grants that are sought will get a "stamp of approval" from policy makers, particularly if someone has been critical in the past or if the last thing policy makers want is empirical evidence about the wisdom (or lack thereof) of past or contemplated measures. The other is that the system itself becomes more resilient and tolerant of critique. It seems to us that the latter is more desirable. If that is not possible, then we need to make sure that HSPR funding streams and priorities clearly and unequivocally value and include more fundamental, and potentially challenging, research on equal terms and conditions to partnered and responsive research.

Overemphasis on short-term issues and challenges

In thinking about how HSPR can help, we tend to think in the short term. The interest is in identifying policy-makers' needs, and, more recently, patient priorities. This framing inevitably leads to articulating the challenges that are facing people today – which is understandable and important – but in its most extreme form, crowds out thinking about the challenges that are on the horizon or that we might be able to predict, but with something less than full certainty. Put another way, we tend to think about the second half of the quote above, the "programs, services and technologies," and much less about the "systems, policies and organizational structures" in which those function (CIHR IHSPR 2021: 4).

Part of this challenge may stem from the tension between what we do as academics in HSPR and the education of "HSPR practitioners." We equip Master's and PhD students with analytic tools and theoretical understanding relevant to HSPR, expecting that most will end up working in, or adjacent to, healthcare systems and not in academia. This training in policy-relevant (i.e., short-term) applied research ideally happens in a partnership that includes health system policy and decision makers; patients and families; and clinicians and faculty mentors. Through this, students learn both the theory and methods of HSPR and also develop broader competencies such as interdisciplinarity, collaboration, leadership and networking (Bornstein et al. 2018).

At the same time, as HSPR scientists, many of us are pursuing research that will likely not have an application for a number of years, if ever. One simple example of this is research related to alternate funding models for health services, or research that helps to inform the design and development of learning health systems. These are complex, multi-year endeavours, which draw on deep thinking and multi-disciplinarity.

The applied HSPR realm is a good training ground for students, but neglects the fact that science must also encompass the larger and longer-term pursuit of generalizable knowledge and "truth," as that can be understood in a context of evolving socio-technical systems that either support or diminish health (Carayon et al. 2011). Academic research in this way can help plant the seeds for fundamental policy change, as well as assess current policy and practice.

Lack of priority on the early "phases" of policy development

A short-term orientation leads to smaller projects with shorter time frames, in order to respond to the "needs" of the system. In such an environment, how does HSPR compare to other parts of science? A useful comparison is to think about the phases of clinical trials. In the early stages of treatment development (phase 1), failure is rife, efficacy is unclear and the widespread adoption of a treatment is far from certain. Potential treatments then go through testing for efficacy (phase 2), effectiveness versus the standard of care (phase 3) and examination of effectiveness and safety after adoption (phase 4).

In health policy, however, there appears to be a limited appetite for seeking "equipoise" and testing new models of care (for example) through studies that are analogous to early-phase clinical trials. Instead, we most often appear to be doing phase 4 studies after the policy has been implemented, and, frankly, the horse has already left the barn. This is the very nature of being responsive to system and patient needs, but it will not drive knowledge generation at the more root level, as this can take years of development and testing and include a lot of failure along the way. To be clear, phase 4 studies of effectiveness and safety are important, but the failure in this case stems from the fact that they are not always, or even often, preceded by exploration of policy efficacy or effectiveness compared to current practice (i.e., phase 2–type or phase 3–type studies), prior to the broad implementation.

We recognize that not all changes can necessarily be evaluated in advance, and there are some examples of "policy trials" in Canada (Persaud et al. 2021; University of Calgary 2018). These examples, however, tend to be small-scale and are not necessarily associated with a clear path to broad implementation if they are successful. Perhaps more importantly, there seems to be an inverse relationship between the scale and cost of implemented policy and its grounding in solid empirical foundations and clear planning for robust phase 4–type assessment. We have all been involved in post-hoc assessments of broad policy initiatives (some, but not all, at the request of or with the support of health systems), and they all suffer for not clearly having such assessments in mind when they were put in place (Lavergne et al. 2016; McGrail et al. 2013).

External pressure for "return on investment" in systems that are at best partially rational

Another challenge for HSPR is the focus on "return on investment" (ROI) in research, knowledge translation and implementation science (Donovan and Hanney 2011). This can have some unintended consequences. First, instead of looking at ROI (for example) as an emergent property of the entire system of funding of health research, current processes (e.g., end-of-grant reporting) conceptualize it as an aggregate of ROI on each individual research project. This puts a focus on the individual rather than the collective, and results in a more risk-averse approach to funding decisions. If every project has to have some kind of ROI, and if failure is not valued, then ideas that are deemed risky will be rejected, even if they are interesting or creative or promising.

To put this bluntly, only small, incremental projects can be known in advance to have a positive ROI. In reality, some research will have impact and some things will not, and as described above, we need to embrace those potential failures. We should acknowledge that showing that something does not work is an important contribution to HSPR knowledge, but if this information is not taken up by the system, then it will by definition have no ROI. Furthermore, some things that do not seem important now will emerge as big priorities later. The ROI in that sense has to be measured over a longer horizon, not just at the point a project is completed. Related to this, it is a foundational principle that science is not about a single study. Instead, it is an accretion of knowledge, with each individual study contextualized within what came before and what needs to come after. This means ROI is really better framed as about a body of work, or a body of research investment.

It is widely accepted by governments and granting agencies that you cannot take an overly bean-counting, utilitarian approach to funding for basic science research because that would leave no room for the creative exploration of fundamental realities, some of which will ultimately generate enormous returns, often in unpredictable ways. The experience of the pandemic, and the fact that the speed of vaccine development rested on decades of science, much of which did not have clear application during its fundamental stages, should make this point clear (Moore and Wilson 2021). HSPR needs to be cut the same slack.

Finally, all of the above-mentioned implies that when a policy window opens (Kingdon 1984), what you throw through it is research that is already completed, or small additions to existing research (e.g., for local context) that can be done in a short period of time. This still does not guarantee that research will be used, because policy is only at best evidence-informed, not algebraically derived from research-based evidence. Equally, this does not mean the research has zero ROI; it just means that other considerations won the day that day. It is sometimes a choice not to harvest the returns from the research at a particular time. That is not the fault of the research.

Conclusion

Our provocations relate to the emphasis we place on short-term priorities and the priority we place on the direct relationship between HSPR and health systems, an apparent ambivalence to the development of macro-level policy interventions parallel to how clinical interventions are developed and a focus on project-based rather than system-based ROI. While the IHSPR strategic plan provides strong guidance on priorities over the next five years, it may underemphasize the trajectory beyond that.

Whether that concern is justified will depend on the investments and priorities that flow from the strategic plan. We absolutely support relationship-based responses to current priorities, the importance of evaluation of existing policies related to funding, organization and delivery of care and the importance of ROI. But this is only one part of HSPR. As tactics are refined and rolled out, they will need to consider, as well, the university-based health services and policy researchers who have interests that might challenge current thinking and current practice. Tactics will need to consider that not all research needs or is able to find willing partners, given the contested nature of some of the work that is of interest to HSPR. They will also need to contemplate ways in which accountability for ROI can be shared with the system. This may mean thinking about opportunities to increase openness of systems to critique, but that project is perhaps not solely the responsibility of IHSPR or CIHR.

The balance in all of this may not be obvious or easy to achieve, but very rarely do important and complex issues have simple solutions. The failure will only be in not trying.

Valeur et nécessité d'une recherche sur les services et les politiques de santé axée sur les défis au niveau macro-systémique

Résumé

Une grande partie de la recherche sur les services et les politiques de santé est appliquée. Nous proposons quatre idées pour stimuler la réflexion sur la relation entre la recherche et les « systèmes » qu'elle entend influencer. Nous concluons que l'accent mis sur le partenariat implique que les chercheurs doivent être sensibles aux échéanciers et aux besoins des décideurs, alors que les véritables relations doivent être bidirectionnelles. Se concentrer sur les priorités des « systèmes » équivaut à mettre l'accent sur les problèmes à court terme. Cela conduit souvent les chercheurs à effectuer des évaluations de suivi après la mise en œuvre de politiques ou d'interventions, dans lesquelles ils ont été peu impliqués. Enfin, l'accent mis sur le retour sur l'investissement pour des projets ponctuels porte à sous-évaluer la recherche plus risquée, mais aussi potentiellement plus intéressante.

Comments

Be the first to comment on this!