Purpose: This paper reports the quantitative component of a mixed-methods study of patient flow in the 10 urban health regions/zones of Western Canada. We assessed whether jurisdictions differed meaningfully in their emergency flow performance, defined as mean emergency department length of stay (ED LOS).

Methods: We used hierarchical linear modelling to compare ED LOS across jurisdictions, based on nationally reported data for 2017 to 2018. We also explored 36-month performance trends. Admitted and discharged patients were analyzed separately.

Results: With the exception of one high performer, no region's performance differed significantly from average for both admitted and discharged patients. The regions' levels of performance remained largely static throughout the study period.

Conclusions: Results precluded any mixed-methods comparison of high- and low-performing regions. However, they converged with our qualitative findings, which suggested that most regions were pursuing similar flow-improvement strategies with limited effectiveness. Deeper changes may be required to address persistent misalignment between capacity and demand.

Introduction

Health systems across Canada have invested considerable effort in alleviating emergency department (ED) crowding and improving patient flow. Numerous initiatives have been implemented that variously address ED input, throughput, output and system-wide factors (Asplin et al. 2003; de Grood et al. 2012). Despite this, Canada continues to fall far short of benchmarks for ED flow performance and recently ranked last in an 11-country comparison of ED wait times (CIHI 2017, 2019). International comparison has offered valuable insight on other countries' contrasting approaches; however, strategies adopted from dissimilar health systems may not prove effective or feasible in Canada (Pines et al. 2011). Accordingly, we set out to assess performance variation among Western Canadian jurisdictions, seeking to glean locally applicable lessons through comparison of high- and low-performing systems.

Whereas an effective system-level response to flow challenges demands more than a mere collection of interventions, most of the literature on potential solutions is focused on specific initiatives (de Freitas et al. 2018; Morley et al. 2018). Literature at the system level is sparse and largely restricted to single-case studies (Kreindler 2017; MacIntosh-Murray et al. 2010) or evaluations of system-wide initiatives, such as pay for performance or Lean (Cheng and Sutherland 2013; Vermeulen et al. 2016). A recent comparative study of American hospitals pointed to the importance of general leadership factors such as executive involvement, data-driven management and performance accountability (Chang et al. 2018). Neither has there been a similar Canadian study, nor one examining multi-hospital systems such as regional health authorities.

This paper reports the quantitative component of a mixed-methods comparative case study of patient flow in urban Western Canada: the Western Canadian patient flow (WeCanFlow) study. The study's intent was to compare jurisdictions on their ED flow performance (quantitative), explore each jurisdiction's flow strategies and context (qualitative) and integrate the findings in order to determine what distinguished high performers from low performers.

The study encompassed all 10 Western regions/zones (hereafter "regions") whose population was at least two thirds urban. At the time of the study, all Western provinces operated under a regionalized or quasi-regionalized model, with flow strategies under the control of regions (Manitoba [MB], Saskatchewan [SK], British Columbia [BC]) and zones (Alberta [AB]). The participating regions were already collaborating through the Western Canada Healthcare CEO Forum and its associated patient flow collaborative – these bodies were partners in designing the study. Decision makers from each region, and from some provincial organizations and Accreditation Canada, were included in the study team.

The qualitative component featured in-depth interviews with 300 managers who had strategic or operational responsibility for flow. Interviews, conducted during a site visit to each region (spring 2016 to winter 2018), sought to furnish a deep understanding of the prevailing approach to patient flow by probing four domains: flow strategies, system design, selection (decision-making) processes and social context. The aim was not to produce a complete inventory of strategies, but rather to explore explanations for perceived success or failure and to identify broader organizational factors that might underpin performance. The qualitative findings were too extensive to be reported in a single article and have instead been developed into separate articles on common strategies and challenges (Anwar et al. 2021; Kreindler et al. 2020, 2021a, 2021b, 2021c).

The planned mixed-methods comparison was predicated on the detection of reliable and meaningful inter-regional differences in ED flow performance. Accordingly, the quantitative component was focused on ascertaining whether the regions' levels of performance truly differed.

Method

Analyses were based on administrative data, drawn from the National Ambulatory Care Reporting System (NACRS) and provided by the Canadian Institute for Health Information (CIHI) for fiscal years (FY) 2015–16, 2016–17 and 2017–18. NACRS coverage varies by province: Alberta mandates that all hospitals report their data, whereas in the participating regions from other provinces, nearly all teaching and large community hospitals report their data, but fewer medium and only a minority of small hospitals do so. We excluded small hospitals from the analysis, not merely because of the unrepresentative nature of the sample thereof, but principally because (according to decision makers on the project team) the flow challenges confronted by smaller and larger hospitals are qualitatively different – the former's challenges revolve around physician coverage and resource availability in rural areas, and the latter's around efficiency and organization. An inter-regional comparison that conflated the two would not be as meaningful. Also excluded were facilities admitting 0 to 10 patients per year; these facilities may be better defined as urgent care centres than EDs and may also face qualitatively different flow challenges. The sample included 7,014,642 visits to 48 EDs in 10 regions with patient and hospital characteristics presented in Table 1, available online here.

There are several possible indicators of ED flow, and most revolve around length of stay (LOS), measuring either the average, median, 90th percentile or proportion of patients exceeding a certain benchmark. We consider the average to be the most informative because, unlike quantiles or benchmarks, it produces a single metric that reflects actual person-hours of ED utilization (e.g., it takes into account whether a long-stay patient occupies a bed for 20, 40 or 60 hours, which has a direct bearing on ED crowding). Moreover, unlike benchmarks, it is robust to distortions that may arise when staff attempt to discharge patients immediately prior to a target time (Mason et al. 2012). Non-LOS-related metrics (e.g., number of patients waiting for a bed at 8 a.m.) are not consistently collected across regions. Thus, we defined the outcome as average ED LOS. As the distribution of ED LOS is typically skewed, with a tail of increasingly long stays, LOS was log-transformed to create a normally distributed variable. However, because log-transformed average values are not readily interpretable, we present raw median and 90th percentile metrics in Table 2.

In keeping with standard practice, the average LOS was assessed separately for admitted and discharged patients and calculated according to the CIHI definition appropriate to each group. According to this definition, LOS begins at the point of registration or triage (whichever is earlier) and ends when the patient leaves the ED (admitted patients) or receives a disposition (non-admitted patients). For 0.1% of the admitted and 3.4% of the discharged patients, the preferred end marker was unavailable, so the non-preferred marker was used instead. Patients who left the ED prior to treatment or disposition, died before/after arrival or were transferred to another facility were not included in either group; these types of non-admitted patients tend to have relatively short stays, and it did not seem appropriate to give a region "credit" for these short stays.

The main analysis was a cross-sectional comparison based on the most recent year of data (FY 2017–18) and conducted using SAS 9.4. To ensure an objective and meaningful comparison, we made several a priori decisions about the analysis and its interpretation.

First, we recognized that apparent differences in the regions' performance may reflect differences in the types of patients they serve. Other than admission status, which is by far the strongest patient-level predictor of ED LOS, patient characteristics that have shown an association with LOS include arrival by ambulance, age, Canadian Triage Acuity Scale (CTAS) level, health condition(s) and (in a minority of studies) sex (Kreindler et al. 2016). All hospitals that participate in NACRS report patients' mode of arrival, age, CTAS level and sex, although the data set does not include comparable information on health conditions. Patient characteristics can affect LOS at both the individual level (e.g., older patients tend to stay longer) and the collective level (e.g., an ED serving mostly elderly patients may become bogged down, resulting in longer LOS for everyone). Accordingly, we used hierarchical linear modelling to test for significant differences in the regions' performance after separating out both measured and unmeasured variation associated with their patient populations.

A hierarchical analysis of ED LOS could be conceived as either two-level (patients nested in regions) or three-level (patients nested in hospitals nested in regions). The appropriateness of including the hospital level depends on one's assumptions about region–hospital relationships. If regions have a major influence on their constituent hospitals and may designate different hospitals to serve different roles, then hospital-level variation is at least partly attributable to regional-level strategy, and removing it from the comparison of regions would seem unfair. On the other hand, if hospitals actually manage flow individually, with little regional oversight, then it would be unfair not to separate out the hospital level. Because our focus was regional, we were most concerned with avoiding underestimating the regional-level effect; accordingly, our primary analysis was two-level. However, we also computed a three-level model to determine how much of the variation in regional performance was a function of variation among specific hospitals.

Finally, to avoid the risk of overinterpreting small inter-regional differences that might be attributable to chance variation or are of little practical importance, we established two criteria for designating a region as a high or low performer. First, the region's performance must differ significantly from average, with statistical significance defined at a threshold appropriate for multiple comparisons (alpha = 0.005; Ioannidis 2018). Second, this must hold for both discharged and admitted patients. This is not to imply that discharged and admitted patient LOS "should" be strongly associated; we recognize that the two may have different determinants (e.g., the availability of in-patient beds might influence only admitted patients' LOS). However, our interest was in regional performance as a global construct, not merely for a subset of ED patients. It would not make sense to hold up as an exemplar a region whose performance is significantly above average on one ED metric but below average on the other. We did not attempt to judge the clinical or administrative significance of observed performance differences, only their statistical significance and consistency.

In addition to comparing performance cross-sectionally, we explored trends in ED LOS over a 36-month period (FY 2015–16 through 2017–18) to see whether any regions appeared to be high/low improvers. As we could not prespecify hypotheses about when and where performance might change, we did not apply tests of statistical significance.

This study was approved by the University of Manitoba Health Research Ethics Board.

Results

Only one jurisdiction met the criterion of performing significantly better or worse on both admitted and discharged ED LOS: Alberta Health Services (AHS) South Zone, AB, which was the high performer on both metrics (Table 3). In three additional jurisdictions, results for both metrics were on the same side of the average: better in the case of Island Health, BC, and worse in the case of AHS Edmonton Zone, AB and the Winnipeg Health Region, MB. In these three cases, however, the comparison reached significance only for discharged patients. The remaining six regions' results were inconsistent across the two metrics or wholly nonsignificant. Thus, the data offered no basis for designating subgroups of regions as high and low performers.

We note that owing to the logistics of scheduling site visits, some regions had their qualitative interviews in 2017–18 and others in 2016–17. To facilitate the mixed-methods analysis, we reran the above models using data from (a) 2016–17 and (b) the year during which interviews were conducted in the region of interest. The pattern of results was highly similar across years, save that in 2016–17 the Winnipeg Health Region scored significantly worse than average on both metrics.

The three-level model showed that once clustering at both the patient and hospital levels was accounted for, "region" was no longer a statistically significant predictor of performance on either metric. As a supplementary analysis, we explored how performance varied across hospitals. We calculated hospital-specific averages for (1) raw LOS and (2) LOS that was both log-transformed and adjusted for patient characteristics (using a score developed from a linear regression model to predict log-transformed LOS, with sex, age, arrival by ambulance and CTAS level as predictors [further details are available from the authors]). Even when adjusted scores were used, hospital-level performance varied considerably within regions (with the exception of the AHS South Zone, whose two hospitals were close neighbours on both metrics). We also explored whether hospital-level performance was sufficiently consistent across metrics and time to reveal high and low performers. A few hospitals did show strong performance on both admitted and discharged LOS. However, two of them served a special function in their respective regions by treating a lower acuity population and admitting less than 1% of patients; two more were the hospitals of the AHS South Zone; and the remainder did not show exceptional performance in both years. In 2017–18, no hospitals performed markedly worse than average on both metrics; one hospital did so in 2016–17, but by 2017–18, it had regressed to the mean. Thus, there was little basis for selecting high- and low-performing hospitals.

Figures 1 and 2 track the regions' monthly performance over the three-year period. During the third year, the Winnipeg Health Region appeared to improve on both discharged and admitted LOS, approaching average performance. The AHS South Zone was consistently the strongest performer on discharged and (especially) admitted LOS, but showed little indication of improvement during the study period, nor did any other region.

Discussion

Findings indicated that the Western urban regions could not be divided into high- and low-performing subgroups. All but one region showed essentially average performance (which, in the Canadian context, means poor performance; CIHI 2017), nor did any region rise above the others during the study period.

These results made it impossible for us to complete a mixed-methods comparison as planned. However, qualitative data corroborated the quantitative findings of inter-regional similarity (Kreindler et al. 2020, 2021a, 2021b). Across the sample, participants reported that service offerings remained badly misaligned with population needs, resulting in pervasive inefficiency, inappropriateness and, in many acute facilities, a perpetual state of overcapacity. Similarity was also observed in the regions' flow strategies (Table 4). All regions relied heavily on intense day-to-day efforts to hasten discharge and manage overcapacity; these appeared to be used not as an adjunct to but as a substitute for rational system design. Some initiatives did redesign parts of the system; common examples included ED streaming, transition units, augmented home care for early discharge and streamlining of access to long-term care. According to participants, some initiatives had failed, fallen prey to cutbacks or been unable to sustain early positive outcomes, but many continued to be helpful. Nonetheless, system-level improvement remained elusive. Indeed, some participants argued that flatline performance should – in the face of escalating patient volume and complexity – be considered a success.

Each of the two regions that separated themselves from the pack has already been the subject of a case study (MacIntosh-Murray et al. 2010; Kreindler 2017). The AHS South Zone comprises two former regions: one undertook a major flow strategy in 2007–2010, including an addition of continuing-care capacity and an extensive, participative process of developing and implementing flow initiatives throughout the system (MacIntosh-Murray et al. 2010); the other subsequently undertook its own system-wide initiative. The AHS South Zone comprises two large hospitals separated by a great distance, allowing each locality to pursue improvement efforts on a manageable scale without adversely affecting the other. In contrast, the Winnipeg region featured six large hospitals in one city as well as a site/program matrix structure. With accountability distributed across groups that defined their patients in potentially incompatible ways, local initiatives not only failed to produce system-level improvement but often clashed with each other (Kreindler 2017). In early 2018, some interviewees attributed the region's recent improvement to an increase in transitional-care capacity or more aggressive implementation of flow initiatives; others suspected a Hawthorne effect, noting that the LOS dropped immediately after the announcement of a planned consolidation of acute care services.

We also found that regional performance was driven to a great extent by hospital-level performance, which varied widely within most regions. Although many of the flow strategies we heard about were regional, they often consisted of a scale-up of interventions that had to be executed at the hospital (or otherwise sub-regional) level. Local tailoring and implementation of such interventions seems entirely appropriate and naturally depends on local leadership. Nonetheless, the observed variability in hospital performance raises questions about whether there are missed opportunities for regional or provincial strategy.

Indeed, much of the (limited) policy that has been directed toward flow, even if characterized by strong central direction, actually focuses on the promotion of local action. One prominent example occurred when British Columbia offered financial incentives for ED flow performance at the regional level (2007–2014). This scheme was associated with improved performance in BC's Vancouver Coastal Health but not Fraser Health (Cheng and Sutherland 2013). Our informants noted that the former, but not the latter, used its pay-for-performance funds to support individual hospitals' improvement initiatives and reward their gains (i.e., to promote local action). In another example, overcapacity protocols introduced by AHS in 2010 were implemented to varying degrees in different zones, which relied on individual hospitals for implementation and enforcement; hospitals, in turn, put pressure on in-patient units to accommodate patients in non-traditional spaces. Calgary Zone participants reported that this approach initially yielded significant improvements in ED flow, but effects deteriorated after two to three years because such pressure could not be maintained indefinitely (Kreindler et al. 2020). The most common regional strategy has centred on encouraging local players to develop their own initiatives, whether through Lean or ad hoc methods (Kinsman et al. 2014; Kreindler 2018). As noted, such an approach succeeded in the two geographies of AHS South Zone, but other regions' attempts (which have, of course, varied widely in content and context) have not shown similar results. While many localized initiatives may have value, there seems to be a limit to what they can achieve, particularly in complex multi-hospital systems. The missing piece may be the establishment of alignment, at the system level, between population needs and service capacity. Notwithstanding much rhetoric about moving care into the community, activities suggested a greater focus on improving ED and hospital throughput than on ensuring that capacity, across the continuum of care, was adequate and well matched to patient needs. Population–capacity misalignment remained rife in the provinces we studied.

Unfortunately, the achievement of population–capacity alignment is not a simple proposition. As elucidated in a companion article based on our qualitative findings (Kreindler et al. 2021a), it would require a deeper understanding of patterns of population need, potentially substantial investments in community and/or institutional capacity and rethinking of models for accessing services (tackling the thorny issue of eligibility criteria). Such changes would need to be designed through careful analysis and the full engagement of providers, patients and informal caregivers lest the reforms create new bottlenecks and absurdities. Even so, they would likely be difficult, disruptive and, at least initially, expensive, demanding a degree of political will that has not been observed to date.

Limitations

This study has several limitations. First, our sample of regions was limited to 10. Making hospitals the unit of analysis would have afforded a larger sample, but at the cost of ignoring the regionalized administration of participating jurisdictions; furthermore, supplementary analyses showed that hospitals did not divide themselves into high- and low-performing subgroups either. Second, the study covered only part (albeit a sizeable part) of Canada and excluded rural regions and small facilities, whose flow-related issues are known to be different. The results are likely generalizable to other urban Canadian areas but cannot be generalized to rural and small facilities. Third, data extend only to early 2018 and do not take into account subsequent developments such as de-regionalization in Saskatchewan or the effects of the COVID-19 pandemic on emergency demand. Fourth, we used only one measure of ED flow; however, brief inspection of regional differences on other indicators confirmed that, whatever the indicator, regions did not split neatly into subgroups of high and low performers. We note also that we attempted multi-level models for other metrics (notably hierarchical logistic regression for LOS benchmarks) as supplementary analyses, but these models failed to converge in SAS 9.4. Fifth, in stratifying patients by admission status only, we may have missed inter-regional differences or temporal changes in LOS for certain subgroups of admitted or discharged patients. Further stratification was unsuited to our purpose of assessing overall regional performance and would have complicated the interpretation of results while inflating the likelihood of Type I error by increasing the number of comparisons. However, further research could examine flow patterns at a more granular level.

Another question that arises concerns the relative strength of patient characteristics, practice patterns, hospital features and environmental factors in predicting LOS. This question has been addressed to some extent by single-region and non-Canadian studies (Capuano et al. 2015; Doupe et al. 2017; Karaca et al. 2012); multi-regional Canadian research would be limited by the lack of universally reported data on several important variables. Our data set was particularly limited, in that we did not collect in-patient data, which precluded the analysis of in-patient metrics (notably bed occupancy) as predictors of ED flow.

Finally, this exploratory cross-sectional study could not test hypotheses about the effectiveness of specific strategies. It was seldom possible to isolate the timing of interventions, as all regions reported numerous initiatives with overlapping dates of implementation, scale-up and sometimes relaunch or revamping; moreover, the qualitative component was not designed to produce an exhaustive inventory. However, because what was observed was stagnation, the study provides at least suggestive evidence that the attempted strategies had limited impact.

Conclusion

Although participating regions were deeply engaged in diverse flow-improvement initiatives, nearly all exhibited a similar, low level of performance. These findings, especially in conjunction with the study's qualitative component, cast doubt on the effectiveness of prevailing approaches to flow. It may be time to look beyond specific initiatives and day-to-day practices to ask what deeper changes may be required to rectify long-term misalignment between capacity and demand.

Funding

The study was funded by a Partnerships for Health System Improvement grant from the Canadian Institutes of Health Research (PHE-141802).

Malgré les interventions, le flux des patients aux urgences stagne dans l'Ouest urbain du Canada

Résumé

Objectif: Cet article présente le volet quantitatif d'une étude à méthodes mixtes portant sur le flux des patients dans les 10 régions sanitaires urbaines de l'Ouest canadien. Nous avons cherché à savoir s'il y avait des différences significatives dans le rendement du flux des patients aux urgences entre les régions, notion définie comme la durée moyenne de séjour au service des urgences (DDS).

Méthode: Nous avons utilisé une modélisation linéaire hiérarchique afin de comparer la DDS entre les régions, selon les données à l'échelle nationale de 2017 et 2018. Nous avons également étudié les tendances du rendement sur 36 mois. Les admissions et sorties des patients ont été analysées séparément.

Résultats: À l'exception d'une région très performante, la performance des régions ne différait pas significativement de la moyenne pour les patients admis et sortis. Les niveaux de performance des régions sont restés principalement les mêmes tout au long de la période à l'étude.

Conclusion: Les résultats ne permettent pas une comparaison de méthodes mixtes entre les régions à rendement élevé et celles où il est faible. Cependant, les résultats concordent avec nos résultats qualitatifs, qui suggèrent que la plupart des régions poursuivent des stratégies similaires d'amélioration des flux, dont l'efficacité est limitée. Des changements plus en profondeur pourraient s'avérer nécessaires pour remédier au désalignement persistant entre la capacité et la demande.

Comments

Be the first to comment on this!