Cancer Care Ontario (CCO), the provincial cancer agency, operates under a model of accountable governance that has been hailed as exemplary. We explored cancer system leaders' views on the balance and perceived efficacy of approaches to accountability in this context. Semi-structured interviews were conducted with 19 participants (MOHLTC=5, CCO=14). Adopting a qualitative descriptive approach, we coded data for four policy instruments used in approaches to accountability. Financial incentives are a key lever used by both parties to effect change. Cancer-specific regulations were somewhat weak, but agency-wide directives were a necessary nuisance that had great force. The effect of public reporting on mobilizing consumer sovereignty was questioned; however, transparency for its own sake was highly valued. Professionalism and stewardship, with an emphasis on trust-based partnerships and clinical engagement, were critical to CCO's success. These approaches were seen to work together, but what made each have force was reliance on professionalism and stewardship.

In Ontario, the cancer system is overseen by Cancer Care Ontario (CCO), a provincial agency and primary adviser on cancer services. CCO's approach has been hailed both at home and abroad for providing insights into how a disease-specific government agency has used various levers to improve quality of care and facilitate partnerships among various stakeholders (Nolte et al. 2008; Ontario Ministry of Finance 2012). We explored system leaders' views on the balance of approaches to accountability at play and their perceived efficacy.

Characteristics of the Sector

In recent years, the Ontario Ministry of Health and Long-Term Care (MOHLTC) has moved from managing regional service delivery towards a stewardship role (Lomas and Brown 2009). Legislated under the Cancer Act, CCO is the main adviser to MOHLTC on the cancer system and plays a brokering role between the ministry and cancer care providers (Thompson and Martin 2004). A memorandum of understanding (MOU) between MOHLTC and CCO defines the terms and expectations of their relationship. A three-year rolling business plan and program-specific accountability agreements provide the basis on which yearly funding is negotiated. These financial contracts reflect the priorities identified by CCO and its partners in their three-year Ontario Cancer Plan, which provides a strategic road map for system improvement.

CCO is responsible for the allocation of approximately $700 million in funding of cancer services annually (Duvalko et al. 2009). Following restructuring in the early 2000s, CCO moved from being a provider of a limited number of cancer services to overseeing the provision of a broader range of services delivered regionally (Sullivan et al. 2004). (See Cowan 2004 for a detailed history of the integration of cancer services in Ontario.) Currently, performance-based funding agreements exist between CCO and each of its 14 regional cancer programs (Thompson and Martin 2004).

A model of accountability that aligns clinical and administrative approaches to accountability was established, supported by three advisory councils (Dobrow et al. 2008). The Clinical Council consists chiefly of provincial clinical program heads and is responsible for reviewing and making recommendations to the CEO of CCO on all policies, standards, guidelines and clinical care initiatives. The Provincial Leadership Council advises on the planning and coordination of cancer service provision across the province and consists primarily of regional vice presidents. Public accountability is supported by a quasi-independent advisory body, the Cancer Quality Council of Ontario (CQCO), with a mandate to monitor and report publicly on overall cancer system performance, and to make recommendations for improvements to the minister of health through CCO's board of directors (Dobrow et al. 2006). This model of accountability is supported by a comprehensive performance management system that enabled CCO to tie government funding to healthcare delivery and quality (Cheng and Thompson 2006).

Methods

We conducted a qualitative study of Ontario health and cancer system leaders' perspectives on accountability. With ethics approval from the University of Toronto Health Sciences Research Ethics Board, as well as informed consent from participants, we conducted interviews from June to October 2012 with (a) senior civil servants at MOHLTC and (b) board, executive team and advisory council members at CCO. Participants were selected purposively because of their familiarity with and leadership role within the cancer system and, thus, their capacity to provide information-rich descriptions of their experiences (Patton 2001).

Using a semi-structured interview guide, we explored participants' perspectives on (a) how expectations between MOHLTC and CCO are established, (b) the goals of accountability and (c) lessons learned. The guide was developed using the "promises of accountability" framework, which captures the range of meanings that policy makers and managers attribute to accountability – specifically, the goals of control, integrity, ethical behaviour, legitimacy, performance and justice (Dubnick and Frederickson 2011; Dubnick and Yang 2011). The guide was designed with awareness of existing approaches to accountability at play in the cancer system, but was not used to probe for them specifically, allowing participants to identify what was relevant in their view. Interviews averaged one hour in duration; each was audio-recorded, transcribed verbatim and entered into a web application for the management and analysis of qualitative data (Dedoose version 4.5).

Results

In total, 19 of 24 potential participants completed an interview for the study (five from MOHLTC and 14 from CCO). We review results across the four approaches to accountability: financial incentives, regulations, information, and professionalism and stewardship.

Financial incentives were seen as foundational to the relationship between CCO and MOHLTC. Unlike CCO, which has many levers at its disposal, money was viewed as one of the only tools available to MOHLTC:

… money, I mean, it will keep coming back to money as the biggest lever, right. It's one of the few levers they have. … [CCO] has a variety of levers. … But the ministry, when you think about it, they don't have that many levers in the Ontario healthcare system. [29-CCO]

The way in which MOHLTC funds CCO is still not seen to be ideal. A participant from MOHLTC described the desire to move towards more of an outcomes-based funding model:

We eventually want to get to the point where everything is on the cancer outcomes, but in the interim I think we're not there yet. … we'd like to monitor on those big outcomes, but right now we tend to monitor on activities. [38-MOHLTC]

Similarly, another participant highlighted the desire to move from volume to quality-based funding – and also the challenge of doing so:

Now, what would happen if we didn't achieve a quality metric? Well, it's not clear to me because that's not really the way the ministry works, right. … what would happen then if somebody saw the right number of patients, but didn't provide the right care? I don't think anybody really knows yet how to handle that, but it's clearly a direction that everybody's moving in. [29-CCO]

Each year, MOHLTC undergoes a reconciliation process with its agencies. However, a participant was unsure what would happen if CCO failed to meet its mandate to improve the system overall:

I don't know if there's something as, you know, formal as a kind of remedy if the results aren't achieved. … They're not held to account in the way that you would think. [24-MOHLTC]

Although we heard repeatedly that "it's not only money" [46-CCO] that drives the cancer system, funding is leveraged by CCO to encourage participation from the regions:

The way they tend to get their cooperation is they have a little bit of incremental money. … So they don't have all the money in the cancer centre, but they have sufficient [money] to get people to pay attention. [24-MOHLTC]

Critically, CCO uses money to incentivize participation from clinicians, who are "not going to do things for free." [20-CCO] As this participant described:

… the health professions are pretty slow to change … it's like a big elephant … That's a real problem. So how do you do that? Well, you can do it by incentives – financial or otherwise. [42-MOHLTC]

Although CCO has been quite successful in "fund[ing] a lot of doctors to help us" where "the rest of the healthcare system doesn't" [41-CCO], there remains room for improvement:

I think there is a barrier in the way in which resources, financial resources, are used in terms of delivery in the system – hospitals, doctors' compensation, all of those issues are not as well aligned as they could be. [46-CCO]

Two sets of regulations were seen to apply to CCO: those specific to the goals of cancer care and those applicable to all agencies of government.

The Cancer Act was not seen by all to be particularly useful, although MOHLTC will sometimes use it as a threat to CCO:

It's not very specific. I mean, you could drive a truck through it. … Sometimes the ministry will use that as a threat, though. … It can sometimes be used as a club. … It doesn't really define the relationship. [42-MOHLTC]

With each iteration, the MOU between CCO and MOHLTC becomes more detailed, creating what this participant called a "tighter leash":

… definitely, the accountability and the structure around accountability has changed significantly in the last number of years. It's almost a significant leap towards more accountability. … the MOU that we signed in 2009 … basically provided a tighter leash from the ministry to CCO. … So that MOU becomes tighter and tighter as time goes on. [20-CCO]

The directives to which all agencies of government are held had much more force, and were seen by some to make CCO less nimble. Speaking to the effect of these directives, in general, several CCO participants felt the burden:

… the inefficiency of having multiple levels of oversight is, it's pretty obvious to us. … Like anything, things might have started based on sound problem solving, but they get taken to an extreme that becomes absurd. [29-CCO]

Although these bureaucratic controls were often perceived as a nuisance, they were simultaneously understood to be ultimately necessary:

I think the sad reality is these sorts of mechanisms or processes do serve to remind people of what's right and what's wrong and the importance to stay out of trouble. [30-CCO]

Participants from the ministry were sympathetic and acknowledged that steps were being taken to minimize the burden:

… there might be an overemphasis on some of the control aspects of accountability … we want to try and make the kind of requirements that we have as not too onerous on them, streamline reporting requirements, et cetera. [38-MOHLTC]

Further, for a MOHLTC participant, these controls are a necessary condition for agency success:

I don't think these controls really have nothing to say about whether they [agencies] are achieving their goals. … for organizations to be successful, they have to be impeccable in terms of their administrative oversight. … if organizations want to excel, they have to make sure that their house is in order. [24-MOHLTC]

One of the key ways in which CCO fulfills its "accountability to the public" [20-CCO] is through information, including the work of the CQCO – specifically, the yearly Cancer System Quality Index. As one participant explains, CCO "should be transparent in terms of how we're doing and we should be open to public scrutiny." [46-CCO]

However, there was some discrepancy about who the CQCO really reports to. On the one hand, some participants echoed the council's mandate that "we report to the people of Ontario – that's our audience." [27-CCO] However, other participants questioned this view and felt that the real audience is actually the people working in the cancer system:

I don't think the public looks at it, but the people who are in the organizations do. … it's not the public you're really reporting to. That part is just an illusion. [50-CCO]

To rectify this, some participants recognized the need to modify the council's approach:

We need to start putting the indicators in a different kind of framework so it would be more accessible to the public. … Then, that really makes it accountable to the public because then everyone is actually able to access the information that's in there. I think we're achieving our mandate right now, which is that we are accountable to the public and we release things publicly. We get things out there but I think we could do more. [21-CCO]

Indeed, another participant took the critique one step further – there is a difference in the way that you report to various "publics," and it is not entirely clear what role public reporting plays in actually changing consumer behaviour:

The way you report to publics versus cancer system people versus the ministry of health versus CCO, very different ways of doing that, and particularly, the distinction between reporting to public versus health system folks. … The evidence I've seen is that it actually doesn't do a whole lot towards changing behaviour. How would they choose differently? [31-CCO]

Despite this criticism, participants felt that there was great value in simply being transparent, as "the public has a right to know." [42-MOHLTC] Further, this "transparency is real value because that's public confidence and trust." [46-CCO] However, the same participant who questioned the effect of public reporting on consumer behaviour urged caution:

I think you have to be careful about what you're being transparent about. … You're just putting information out there without any context, and potentially alarming and misinforming. [31-CQCO]

Professionalism and stewardship is a key instrument. Several participants reflected on the legacy of cancer system restructuring. By the early 2000s, wait times had become "a big embarrassment for the ministry" and, in response, a private after-hours radiation clinic was opened, which became "a microcosm of the issues that plagued CCO." [29-CCO] This led to "a lot of animosity" [20-CCO] and "bad blood" [24-CCO] in the system. Rather than terminate CCO, MOHLTC agreed to restructuring. From the perspective of one ministry participant, "that's where I've kind of seen them build back up." [24-MOHLTC]

Participants spoke highly of the current relationship between MOHLTC, CCO and providers. Mutual trust was seen to enable CCO's success:

We have a fixed cancer system. … because the ministry placed in the organization a trust to do it better. A trust has actually built the environment to allow both organizations to succeed. [38-CCO]

A number of performance monitoring techniques were cited as ways in which both the regions and CCO itself are held accountable. CQCO was seen by many as a key tool for holding CCO accountable for overall system performance:

It's very helpful actually to have that kind of public spotlight scrutiny, because it provides some pressure to move the bar higher. Not only for us but for all our partners. [46-CCO]

The quarterly review process between CCO and its regions was a second key performance monitoring tool. As this participant put it, "If you pay somebody to come and paint your house, you want to make sure you inspect it to make sure they've done a good job." [20-CCO] This "holding people's feet to the fire when you don't get results" has become "a very strong brand" [38-CCO] for CCO.

Quarterly reviews provide valuable face time between the provincial and regional offices, allowing regional vice presidents to voice issues to CCO and advocate for their regional team:

… I take advantage of it to make my points to CCO. I also make my points to the people on my team that I'm advocating for them. … It's a whole complex dynamic. [50-CCO]

One participant, however, was cynical about the process beyond providing face time between the regions and the provincial office:

I don't think quarterly reviews are useful, but on performance management I think what they do is they give the required face time of their team and our team. So it's a kumbaya, but it's not going to be a region performance-changing event. [38-CCO]

Importantly, the data generated through this process leverage a collective and individual sense of wanting to improve, because "nobody wants to be near the bottom" [50-CCO]:

… one region, one year they were worst at something, then next year they were the best. And that change was made only by showing them the data. Not spending a penny. Because nobody wants to be the worst. [41-CCO]

Despite the benefits of collecting performance data, many participants expressed concern about the potential for "indicator mania" [41-CCO]:

… we can at times over-measure. Having too many measurements isn't necessarily a good thing. … I think we need to be very clear that it's clinically relevant and make sure that it's for a good reason that we're measuring it. [45-CCO]

CCO's ability to engage clinicians by providing these data was seen as essential to its work and has created "credibility across the system" [24-MOHLTC]:

… a large part of [CCO's] success formula has been that it's always done it through working with clinician leadership and getting people who are recognized in the field, who are credible onside and working, so it's never been sort of perceived as a bunch of bureaucrats … . [30-CCO]

Further, CCO "bring[s] the science and the clinical engagement to the table, which are key assets that the government doesn't have." [35-CCO] Indeed, MOHLTC relies on CCO to "be a bit of a buffer" by "removing decision-making from the political arena and depoliticizing stuff" [30-CCO]:

I think it's very easy for people to say, "What is a bureaucrat going to know about this particular case? It's very easy for them to stand aside and make these black-and-white decisions." And, so, we need Cancer Care Ontario folks to kind of help advance that discussion a bit. [23-MOHLTC]

However, many participants felt that there was still more to be done to engage the clinical community. The difficulty of engaging more clinicians is exacerbated by the fact that, historically, physicians have had "a sort of carte blanche status" [20-CCO] in the healthcare system and tended to be "treated like different animals" [24-CCO]:

… we're probably still falling short on the clinician side. It is not necessarily because we don't reach out, but it's hard to involve everyone. … It's still a minority of physicians that are involved, that are interested in trying to make the system better. [45-CCO]

Discussion

CCO's governance model of aligning clinical and administrative accountability, supported by a comprehensive performance management system and commitment to public reporting, employs each of the four approaches to accountability.

Financial incentives are a key lever used by both MOHLTC and CCO to effect change. While money is foundational to both the ministry and CCO, as we heard, these relationships extend beyond simple funding arrangements. There is a desire on both sides to move towards outcome- and quality-based funding, but challenges persist. Further, barriers exist in the way funding is used to motivate participation throughout the regions down to the level of individual clinicians.

Regulations specific to meeting the goals of cancer care were somewhat weak, whereas agency-wide directives had great force but were felt to be a necessary nuisance. Ministry respondents recognized the potential for burden in meeting agency directives and acknowledged that efforts are being made to streamline reporting requirements. However, on both sides of the relationship, participants recognized that compliance with these types of regulations were not without warrant and ultimately reflected good business practice.

The effect of public reporting on mobilizing consumer sovereignty was questioned by some. There was some discrepancy in terms of which "public" these reports served. As a tool for change, public reporting encourages those working in the system to continuously improve their performance. Transparency for its own sake was regarded as adding value by building confidence and trust.

All these approaches to accountability were seen to work together, but what makes each ultimately have force is reliance on professionalism and stewardship. A variety of performance monitoring tools are used to appeal to administrators' and clinicians' individual and collective sense of wanting to improve. However, some cautioned against over-measurement as detracting from this goal. While CCO's ability to engage and promote regional leadership has generated a sense of its legitimacy across the system, and as an authority the ministry will call on from time to time, there remains room for improvement to further engender a sense of shared responsibility and partnership.

Obligation de rendre compte dans le système ontarien des services de cancérologie: étude qualitative sur le point de vue des leaders du système

Résumé

Le modèle de gouvernance responsable d'Action Cancer Ontario (ACO), l'organisme provincial de cancérologie, est souvent qualifié d'exemplaire. Nous nous intéressons au point de vue des leaders du système de cancérologie sur l'équilibre et l'efficacité perçue des démarches d'obligation de rendre compte dans ce contexte. Nous avons mené des entrevues semi-dirigées auprès de 19 personnes (MSSLD=5, ACO=14). À l'aide d'une méthode qualitative descriptive, nous avons codifié les données portant sur quatre instruments de politique utilisés dans les démarches liées à l'obligation de rendre compte. Les incitatifs financiers constituent un important levier, utilisé par les deux parties, pour accomplir des changements. Les règlements propres à la cancérologie sont un peu imprécis, mais les directives de l'ensemble de l'organisation constituent un mal nécessaire qui présente une force appréciable. L'effet de la présentation de rapports publics comme facteur de mobilisation pour la primauté du consommateur est remis en question; cependant, la transparence en soi est hautement valorisée. Le professionnalisme et la gérance, avec un accent sur les partenariats et l'engagement clinique fondés sur la confiance, sont essentiels pour le succès d'ACO. Il semble que ces démarches fonctionnent ensemble, mais ce qui fait la force de chacune d'entre elles est la confiance accordée au professionnalisme et à la gérance.

Comments

Be the first to comment on this!