The objective of this study was to identify key issues relevant to the development and implementation of a macro-level priority-setting framework (i.e., across broad service areas) within the Calgary Health Region. We used rigorous qualitative methods, including focus groups, meeting observations and interviews to identify views of decision-makers. Key issues relevant to macro-level priority-setting included: application of evidence, incentives, physician involvement, public involvement and application of values. Detailed insight into each of these issues was derived, including how best to handle related barriers to priority-setting in health organizations and important lessons for framework development. These lessons learned should provide insight for similar activity in other jurisdictions.

Introduction

In many Western countries, health authorities and related organizations hold the responsibility of meeting the health needs of local populations. To date, numerous approaches to prioritysetting have been proposed, with varying levels of success (Mitton and Donaldson 2004). One economic framework that has gained widespread use in Australia, Canada and Britain is program budgeting and marginal analysis (PBMA) (Cohen 1995; Miller et al. 1997). This approach provides decisionmakers with an explicit means of shifting or reallocating resources within a fixed envelope if it is held that benefit overall will improve through instituting such changes.

To date, approaches to priority-setting, such as cost-effectiveness analysis, needs assessment and PBMA, have primarily targeted decision-making at more micro levels (Peacock 1998; Mitton and Donaldson 2004). However, at a macro level, across broad service areas within an entire health authority, decision makers are often left without formal tools to guide prioritysetting decisions. In fact, allocation decisions at this broader level often end up being based upon historical proportions of funding for each program area (Miller and Vale 2001), leaving little opportunity to "maximize health gain for the available resources" (Birch and Chambers 1993).

Following the piloting and evaluation of PBMA in several health authorities in Alberta at the level of individual program areas (Donaldson et al. 2002), a research project was initiated to examine its application at a broader level, across programs of care, within a single health authority. The Calgary Health Region (CHR) is a fully integrated, academic health services delivery organization, which provides services across the continuum of care, from community health services to acute tertiary care. The CHR is one of nine integrated health authorities in Alberta, which each receive a pot of resources from the provincial government to administer and deliver services within their geographical domain. The annual operating budget of the CHR is approximately $1.5 billion, with the majority of physician reimbursement falling outside this budget, as it is paid directly from a separate "provincial pool" on a fee-for-service basis.

The purpose of this paper is to highlight challenges and ways forward for priority-setting activity at an organization-wide level specifically from a decision-maker perspective. These issues were identified through a participatory action research project, which had as its overarching aim to develop and implement a macrolevel priority-setting framework in the CHR. The reflections reported herein were used in development of the final approach adopted in the CHR and should provide insight for other health organizations embarking on similar work. To our knowledge, such detailed, qualitative analyses of priority-setting-related themes have not previously been reported in the health policy or management literature.

Table 1.

Phases of the PAR Project Examination of recent and current priority-setting practices Participant observation of senior management team meetings and internal documents to gain an understanding of the current context and practices of macro-level priority-setting in the CHR. Reflection upon recent and current priority-setting practices Focus group and interviews with senior managers and clinicians to critically reflect on current and historical macro-level priority-setting practices and to identify key components for inclusion in a macro priority-setting model. Introduction of priority- setting economic principles Ongoing informal and formal training throughout the PAR project to build understanding among senior managers and clinicians of economic principles for priority-setting activity. Development and implementation of the prioritysetting model Development and implementation of a new approach to priority-setting at the macro level within the CHR. Refinement of the priority-setting model After implementation of the macro-level approach to priority-setting, further interviews and focus group activity to identify areas for process refinement.

Alberta-Based Research Project

Participatory Action Research

Within the CHR, a need was identified for an explicit, systematic process of priority-setting across broad service areas or portfolios (e.g., clinical services, community services). To this end, a participatory action research project was initiated in Fall 2001, as outlined in Table 1. Participatory action re-search is a form of social research that blends knowledge generation with organizational action and change (Greenwood and Levin 1998). In action research, it is assumed that knowledge develops from experience, and that stakeholders can become aware of conditions and learn to take actions to alter the initial practices of the organization. The CHR project was carried out by a team encompassing health economists, a qualitative researcher and senior managers and clinicians seeking to improve their priority-setting practices.

PAR Phases One and Two

This paper focuses on substantive findings from the first two phases of the PAR project, in gaining a decision-maker perspective related to various challenges and ways of moving forward when setting priorities across major service areas. Phases three and four of the PAR project focused on building the process itself and developing CHR's capacity to apply the framework in practice. Details of the framework and a description of resource reallocations resulting from framework implementation are found elsewhere (Mitton et al. 2003).

During phase one, the researchers established contact with all stakeholders and built a sense of ownership for the project. Senior managers were gradually engaged in a dialogue with the researchers about their issues of concern around macrolevel priority-setting and how they saw the research project assisting them (Hart and Bond 1995). Researchers attended regular priority-setting meetings of senior managers and clinicians between October 2001 and April 2002 to document decision-making processes, sources of evidence, group dynamics and roles, and organizational culture. Extensive participant observation notes recorded important elements of each of the meetings, and included descriptions of the formal positions and roles of the senior decision-makers, processes and exercises used to set priorities. The notes also included observations about needs for more rigorous priority-setting practices. The notes taken during the meetings were transcribed and thematically analyzed for substantive issues that were subsequently included in the focus group and interview guides.

The purpose of phase two was to gather reflections from the senior managers and clinicians involved in macro-level prioritysetting about their current practices and processes, which would ultimately inform development of a new approach to prioritysetting. In this phase, members of an internally struck prioritysetting committee (eight individuals) participated in a focus group to gather information about their involvement and suggestions for improvement with respect to the priority-setting practices. The focus group created a venue for interactive construction of information; as participants shared their experiences and perceptions, they expanded or delimited the recall and range of other managers' experiences.

In addition to the focus group, one-onone qualitative interviews were conducted with nine other senior management team members (i.e., both administrators and physician leaders) to gather more in-depth and personal reflections on group dynamics, political and interpersonal influences, and the role of personal values in priority-setting practices. The guide for the focus group and interviews is found in Table 2. Senior managers and clinicians were asked to participate in the interviews based on their organizational roles and interest in priority-setting, rather than by random assignment or other conventional sampling strategies (Stringer 1999).

Table 2.

Questions Guiding the Focus Group and One-on-One Interviews What is your perception of what priority-setting entails in the CHR? What is your overall reflection on the CHR's current priority-setting practices and those in the past? What have been the major driving forces behind priority-setting exercises up to this point? What do you think the major driving forces should be? Is it clear what the values and guiding principles of the CHR are? To what extent are these values considered when setting priorities? To what extent does the CHR have an organizational culture conducive to using evidence in priority-setting activities? What capacities currently exist within the CHR to build a macro-level priority-setting model? (E.g., organizational structure, information sources, links to the university, etc.) How could priority-setting practices/processes in the CHR be improved? What is your vision for priority-setting models and practices in the future? What further information or training do you think the organization needs to get to your ultimate priority-setting model?

The focus group and interviews were audiotaped, transcribed and thematically analyzed with the assistance of QSR N5 software (Denzin and Lincoln 2000). Identified themes were categorized as either descriptive or interpretive themes (Miles and Huberman 1994). The interpretive themes extend beyond pure description to strategic thinking about how patterns in priority-setting practices could be explained, and what could be done to change those practices. Quotes from respondents are included in the Results section to elucidate the main themes.

Results

Key descriptive themes included who the stakeholders are (e.g., board members, staff, public, clinicians), what the external pressures are (e.g., political pressures, public preoccupation with acute care, instability in funding) and what the current and previous priority-setting processes looked like (e.g., basing decisions on historical funding, information used in previous decision-making). As these descriptive themes are not solely under the control of the health region, they are thus difficult to incorporate directly as steps in building a macro-level prioritysetting process.

The five key interpretive themes with respect to prioritysetting at a macro level within the CHR were evidence-based decision-making, incentives, physician involvement, public involvement and values. It is these five areas that are the most amenable to control and change by the health region, and thus are the focus of the remainder of the paper. In-depth discussion of these themes will provide other researchers and decisionmakers with insight into key areas for consideration when embarking on macro-level priority-setting activity.

Evidence-Based Decision-Making

There is a strong understanding of the importance of evidencebased practice for priority-setting at a macro level within the CHR, but there are significant barriers and challenges in terms of capacity, resources, infrastructure and organizational culture. One participant expressed the dilemma: "Ideally, we would make sure that any dollars invested into the system are going to improve people's health. But in so much of our healthcare work, we don't know the real science behind decisions." Based on the information provided from the decision-makers, these barriers could be addressed with development of organizational guidelines about how and what evidence should be applied in priority-setting activity. Such guidelines should take into account variations in availability of evidence, validity of different forms of evidence (e.g., qualitative versus quantitative, published literature versus best expert opinion) and willingness to invest in innovations that may not yet have "high level" evidence. A view held by many managers was that there needed to be more "discipline in using data that is imperfect or incomplete." A priority-setting framework should also incorporate recommendations for organizational processes conducive to sharing evidence across service areas and levels of the organization. Further, the framework should provide a clear recognition that, if evidence as traditionally understood is not available, then decisions based on "expert opinion" may also be valid, so long as the process is transparent.

Incentives

An important component identified by respondents for the development of a macro-level priority-setting process is the incorporation of incentives that would recognize and reward innovations and efficiencies. For example, "we need to build a culture where we invest in trying things differently. You don't get changes in behaviour without incentives programs so that clearly needs to be part of the process." A major challenge identified, however, was that improvements in efficiency may result in the reduction of a program's budget. Under the current system, for example, if a program area offers services at lower cost or using fewer beds, the tendency is to deduct the budget amount left over, creating a disincentive for spending below budget projections. In actuality, there should be a "positive feedback loop that rewards improvement." That is, the health region needs to examine appropriateness of spending rather than simply whether a service area came under or over budget. Further, to ensure that widespread engagement is obtained for a priority-setting process that involves reallocating resources across program areas, strategic reinvestment in areas giving up resources may be required. Importantly, such reinvestment does not have to be solely in monetary terms. "We don't necessarily have to use our growth dollars to add more bricks and mortar. The incentives can be provided in personnel, or supplies, or drugs, or all those things that we need along the way to provide that improved service."

Physician Involvement

In relation, there is a perceived misalignment of incentives in the CHR between physicians (who are not directly accountable to the health region and paid on a fee-for-service basis directly from the government) and the health region, which may be compelled to limit access and constrain spending. This mismatch of incentives translates into a lack of constancy of purpose, poor cooperation and contradictory targets. In fact, inadequate physician involvement in macro-level priority-setting has been identified as a reason for perceived inconsistent prioritization of services within the CHR in the past. Challenges for direct physician involvement include time, but perhaps more fundamentally, the philosophical dilemma physicians face in making macro-level tradeoffs (i.e., rationing decisions) that may create explicit "winners" and "losers."

While health-region decision-makers may be distanced from individual patients who are demanding services, physicians feel primary responsibility to their patients, sometimes placing them at odds with restrictions imposed by the health region. One physician leader stated that he has a vested interest in his own program area but nonetheless also has role to play in macro-level priority-setting by bringing evidence and expert opinion to decisions. The role of these physician leaders would be to provide the evidence to support their needs for resources, and to lead in innovations and new approaches that would save money as well as improve health outcomes. This same physician acknowledged that in situations of resource constraints, he might not be generous to his colleagues in other portfolios. Thus, he stated that "it is fine that those kinds of decisions be handled by someone with a broader mandate." A realistic starting place may be for managers to improve consultative discussions with physician leaders to provide, at minimum, built-in "validity checks" on resource reallocation decisions that will affect clinical practice.

Public Involvement

All of the respondents agreed that the health region, as a publicly funded service, should improve the amount of public engagement and participation. While this seems worthy, it was also held that the public are not always well enough informed to think about broad issues that arise in macro-level prioritysetting. "The public … is offended by the notion of having to ration. It seems that the media doesn't buy it, the politicians don't buy it, the public doesn't buy it." It was thus felt that significant education on the notion of scarcity and the need to make tradeoffs would have to be included in any efforts to engage the public in priority-setting processes. A further challenge is in trying to meet public expectations. Politically charged, highprofile, technology-intensive procedures tend to attract greater public attention, but also can distract from strategies that may be more effective and less expensive. That said, an important outcome of public engagement is establishing buy-in from citizens to support the decisions made by the health region "Otherwise they will just point their finger at us, asking: Why aren't we doing more?"

While respondents indicated that direct public consultation can be an important input to the priority-setting process, it should also be balanced with other sources of evidence and other stakeholders' input (e.g., representatives from the university, medical staff and the provincial ministry of health). Generally, it was felt that the public's input into healthcare priority-setting should not be focused on treatment decisions, as there was a perception that the general public does not have the expertise to make these technical decisions. Rather, the public could assist more appropriately in defining the health region's values and criteria, which would then guide macro-level priority-setting.

Values

Priority-setting, particularly at a macro level in which decisionmakers are required to compare disparate patient groups and health outcomes, is necessarily based on values. Respondents widely held that the CHR needs to think more explicitly about values, incorporate values as part of their ongoing work and refer back to values throughout their business planning processes. They also felt that they needed to remind one another about how beliefs actually influence decisions and priorities. The CHR's values and guiding principles need to be more apparent and serve as a "checkpoint" in the priority-setting process to ensure that there is congruence between the decisions and the values.

A key issue was difficulty in translating values into action so that they explicitly inform the priority-setting process: "There may be a piece of paper somewhere which has the values listed, but that is really useless unless it becomes part of your day-today work. I really think values need to be part of what you do every single day, which includes developing priorities. In any priority-setting session, the first thing that should be done is to put on the table the values, the mission statement." The organization must not only come to consensus about which values deserve more emphasis in certain contexts, but also develop a process for translating those values into practice: "On the surface, I think the values are there, but when we get into crisis and we drill down into issues, some of the principles go out the window."

Discussion

Decision-maker views are a critical part of building an explicit priority-setting process (Mitton and Donaldson 2002). This paper reports on reflections of five interpretive themes, which should be considered in such activity. This insight was utilized in the initial implementation of the PBMA approach in the CHR in 2002/03. PBMA has now been rigorously applied as a planning tool for three successive budget cycles, the latter two without any researcher involvement.

The application of evidence in decision-making is a subject of considerable attention in the policy literature (Retsas 2000), and was highlighted here as an important component of macrolevel priority-setting. While the participants acknowledged a high level of motivation to apply evidence in their work, they noted inconsistencies in the availability and quality of evidence across the organization. The key point from the decision-makers was in recognizing that, if available, evidence should be used, but when not available, reliance on expert opinion, be it from managers or clinicians, is a valuable means of moving decisions forward.

Another integral part of a macro-level priority-setting practice is incentives that recognize and reward innovative service provision practices and improved efficiency. The participants emphasized that these incentives must be tangibly beneficial to the service area creating the innovation but, in contrast to other work (Bailit Health Purchasing 2002), do not necessarily have to be directly financial. In addition, physicians were seen here to provide valuable input into decision-making both as experts and as evidence drivers, as they bring specific expertise about their particular service areas (Ferlie et al. 2000). The role of physicians, however, should not be expected to include the broader mandate of priority-setting across major service areas.

Another ongoing challenge in priority-setting is in involving the public to inform resource allocation. Generally, there was skepticism regarding the public's level of understanding of funding issues and the range of issues facing decision-makers. One fruitful area for public input is in developing the criteria by which priority-setting decisions could then be guided (Shiell and Mooney 2002).

Finally, values were identified as one set of guidelines that must be actively applied in priority-setting exercises and explicitly incorporated into a macro-level priority-setting framework. This project revealed that there is a lack of clarity and inconsistent application of values during priority-setting activities in the CHR. While value statements can be found in various internal documents in this health region, communication and active application of these values need to be improved. As values are at the root of priority-setting activity (Ham 1995), they should be clearly articulated and incorporated to ensure transparency in the practice of priority-setting. Transparency, in turn, is a key component of a priority-setting process that is fair (Singer et al. 2000).

In addition to the data presented in the Results section, the first two phases of this PAR project achieved several further important outcomes:

• Researchers gained an holistic understanding of ongoing priority-setting activities and key social and professional roles by attending senior management team meetings.
  
• Senior managers and clinicians worked directly with the researchers to highlight key issues for incorporation in the development of a macro-level priority-setting model.
  
• There was an opportunity for participants to reflect on their own practices, creating self-directed impetus for the creation (and eventually application) of an explicit priority-setting framework and supportive structural processes.
  
• The PAR process promoted group belonging, fostered creativity and critical thinking, promoted change and growth, and served as a means of resolving shortcomings.

Throughout the PAR project, the researchers introduced and explained economic principles that would be of assistance in building a macro priority-setting model (phase three). This training for senior management team members was both informal (in the form of conversations during meetings) and formal (in the form of presentations), and was expected to generate the knowledge necessary to transform the organization's priority-setting practices over the course of several complete budget cycles. The researchers used a further focus group as well as additional individual consultations with members of the senior management team to facilitate the development of tools and plans for implementation of the new framework (phase four). The researchers and stakeholders collectively finalized a formal approach to priority-setting at the macro-level across major program areas in June 2002.

This project intended to yield a customized prioritization framework grounded firmly in the pressures, expertise and structures of the CHR. The project penetrated the experienced reality of the day-to-day work of senior managers and clinicians. While this contextual specificity limits the generalizability of the study's findings, it is important to note that the goal of action research is not to create purely theoretical and generalizable knowledge about organizational behaviour (Stringer 1999). Rather, the goal here was to extend the understanding of priority-setting practices in order to build processes and structures that are more rigorous and explicit. On the other hand, many of the points raised by the participants in relation to these key elements of priority-setting will likely be relevant to health organizations in other contexts. Issues such as public involvement, application of evidence, incentives, values and stakeholder engagement are clearly universal challenges. The examination of these issues in this paper should generate awareness of essential elements to be included in priority-setting models developed in other jurisdictions.

Conclusion

Health authorities have traditionally made macro-level allocation decisions based on historical trends and ad hoc decisionmaking. The development and application of a formal priority-setting framework at this level is novel. The first two phases of the PAR project within the CHR yielded the prerequisite assessment of current priority-setting practices and reflection on components that present particular challenges in macro-level priority-setting. The challenges and responses presented herein should serve to guide other organizations in the development of explicit priority-setting processes.

Comments

Be the first to comment on this!