Objective: This study aims to present a proof of concept of a dashboard on a set of indicators of access to primary healthcare (PHC) based on electronic medical records (EMRs).

Methods: This research builds on a multi-method design study including (1) a systematic review, (2) a pilot phase and (3) the development of a dashboard.

Results: Eight indicators were carefully selected and successfully extracted from EMRs obtained from 151 PHC providers. Indicators of access over time, as well as among providers and among clinics, have been enabled in the dashboard.

Conclusion: EMR data enabled the development of a real-time dashboard on access, giving PHC providers a reliable portrait of their own practice, its evolution over time and how it compares with those of their peers.

Introduction

Value in healthcare has been defined as “the contribution of the health system to societal wellbeing” (Smith et al. 2020: 3), which is the cornerstone of health, wealth and the healthcare system. It should be aligned with the values of patients, clinicians, decision makers and health organizations (Kuluski and Guilcher 2019; Wodchis 2019) to optimize patients' health outcomes and the allocation of resources (Porter and Lee 2013; Smith et al. 2020; Teisberg et al. 2020). One lever that has been proposed to improve value in healthcare is to strengthen primary healthcare (PHC). PHC is the cornerstone foundation of the healthcare system, where 80% of patients' needs are met (WHO 2018).

In Canada, access to PHC is one of the main challenges faced by the healthcare system. Resolving this issue is a priority for patients, clinicians and governments. Although the importance of PHC has been highlighted in the past decades, data on the delivery of PHC are underused (Wong et al. 2019). Indeed, access to and use of these data are necessary for practice improvement (Hogg and Dyke 2011) as measuring change is necessary to develop, sustain and spread improvements (Langley et al. 2009). Furthermore, to comprehensively evaluate PHC services and enable actionable decisions, multiple data sources should be linked to capture all relevant professional and organizational information. This may include electronic medical records (EMRs), administrative databases or patient surveys (including patient-reported experience and outcome measures [PREMs and PROMs]) (Wong et al. 2019). The implementation of EMRs in medical clinics over the past decade has provided an opportunity to measure and improve PHC. However, until now, EMR data have been underused in Canadian PHC, depriving the healthcare system of a rich source of timely data.

EMR data have previously been used in Canada to develop dashboards, which are visualization tools based on both text and graphic support, allowing for timely monitoring of clinical or organizational outcomes. Dashboards are quality improvement tools that can optimize the performance of health organizations, document patients' health needs and support timely decision making based on data made available in real time (Ehsani-Moghaddam et al. 2021; Singer et al. 2021). One example in which dashboards are used in PHC is the Canadian Primary Care Sentinel Surveillance Network, a population-level health surveillance system used to monitor chronic diseases in Canada based on EMR data (Williamson et al. 2014). Dashboards represent an invaluable source of information to improve PHC access because they capture the processes and outcomes of access as recorded by healthcare professionals and administrative staff.

This article presents a proof of concept of a dashboard on access to PHC based on EMR data. The specific objectives are to describe the key steps undertaken to develop such a dashboard, namely, to:

• map, conceptualize and validate a set of indicators of access to PHC;
• pilot the extraction and measurement of access indicators; and
• create a dashboard to enable comparisons of indicators longitudinally among professionals and across organizations to support reflective practice.

The findings demonstrate the feasibility and relevance of an access dashboard and inform decision makers and researchers interested in developing such a tool in order to assess the performance of organizations and evaluate large-scale reforms.

Methods

This study was based on a sequential multi-method design informed by (1) a systematic review of indicators of access and validation through expert consultations, (2) a pilot phase of data extraction from EMRs and (3) the development of a dashboard to compare data over time, both among professionals within a clinic and between clinics. Although there is no unique methodology to develop healthcare dashboards, we have followed the steps recommended by the Centers for Medicare and Medicaid Services (Centers for Medicare and Medicaid Services n.d.). These steps include identifying the goal of the dashboard, determining its audience, selecting key indicators to track, extracting data and interpreting findings. They may occur in a different order, iteratively or not, depending on the purpose of the dashboard and the availability of the data.

Objective #1: Map, conceptualize and validate a set of indicators of access to PHC

SYSTEMATIC REVIEW

First, we conducted a systematic review to map a comprehensive list of indicators on access to PHC. The search strategy was inspired by Rose et al.'s (2011) systematic review on “Advanced Access Scheduling Outcomes.” We searched MEDLINE, HealthStar, PsycINFO and CINAHL databases for papers published between 2001 and April 2021 covering three key concepts: “advanced access,” “primary care” and “indicators.” The advanced access model is one of the most recommended models around the world to improve timely access (Breton et al. 2020). The model ensures that patients obtain access to the appropriate healthcare services within a time frame dictated by their needs (Breton et al. 2022). The search strategy is outlined in Appendix 1 online here. The twofold screening process was conducted independently by two analysts. Theoretical and empirical papers were retained, whereas commentaries, theses, protocols and policy briefs were excluded.

The first step, based on the reading of titles and abstracts, consisted of removing duplicates, papers in a language other than English or French and papers on irrelevant topics. The second step involved a detailed review of the remaining articles to identify papers reporting on access indicators. For systematic reviews, we returned to the original studies. Finally, the two analysts extracted all indicators that operationalize access along with their definitions and data sources. Indicators were then ranked by the number of occurrences.

EXPERT CONSULTATION

A committee of 17 experts was purposefully selected to validate the list of indicators identified by the systematic review. Participants were selected based on their expertise on access or healthcare indicators in general. All of the experts who were contacted, except for one, accepted the invitation. The committee was composed of five decision makers, six clinicians and six researchers with expertise on access in Quebec. A two-hour virtual meeting took place in June 2021. Participants were first presented with the results of the systematic review. The definitions of each indicator were then discussed. Participants were then split into two groups to participate in a structured brainstorming activity with a trained facilitator to generate additional indicators that could be used to appreciate access to PHC. The usefulness of each indicator and its feasibility and reliability based on the data available to generate the indicator were discussed.

Objective #2: Pilot the extraction and measurement of access indicators

The research team shortlisted the indicators based on the expert consultation and the data available in the EMRs. We extracted data and generated indicators collected from healthcare professionals involved in another research project conducted by the research team (Gaboury et al. 2021). As part of this project, we had ethics approval to access available EMRs with the involved professionals. To further refine the assessment of the feasibility and reliability of each indicator, our research team manually extracted and measured the indicators on a weekly basis (Tuesday mornings before the clinic opened). Each EMR software tracks data differently and uses different definitions for its indicators. Furthermore, multiple clinics using the same EMR might register data differently within their own processes. Thus, this step necessitated a thorough clean-up and organization of the extracted data to enable the translation of each EMR indicator and tracker into a single codebook and to ensure the quality of the extracted data.

Objective #3: Create a dashboard to enable longitudinal comparisons of indicators

We measured data through EMRs in real time to build the dashboard. Every week for 12 months (January 2022 to December 2022), our dashboard was filled in by eight clinics. The dashboard aims to compare indicators over time, as well as medians, means and standard deviations among professionals and among clinics. The visuals and the presentation of the data in the dashboard were improved through some iterations.

Results

Objective #1: Conceptualize and validate a set of indicators of access to PHC

SYSTEMATIC REVIEW

The initial search identified 1,733 citations, of which 1,305 were excluded based on title and abstract reviews (Figure 1). Of the remaining 428 articles, 81 were included in the analysis, representing 12 indicators of access.

Of the 81 papers identified, 36 studies reported on several indicators. As shown in Table 1, the level of operationalization of the 12 indicators (i.e., providing a definition) varied across papers. Although the definitions were generally homogeneous, some indicators were presented without being formally defined or rigorously operationalized. The two most common indicators were “waiting time for appointment” and “no-show,” which were used in 38 and 36 articles, respectively. “Third next available appointment” and “continuity” each appeared in 20 articles, and “patient volume” appeared in 14 articles. Indicators reported six or fewer times in the literature included “productivity,” “cancellation,” “loss to follow-up,” “open slots,” “visit lost to outside providers,” “patient seen on their day of choice” and “workload.” Appendix 2 (available online here) presents the details of Table 1 and a list of the references from the systematic review.

in 20 articles, and “patient volume” appeared in 14 articles. Indicators reported six or fewer, times in the literature included “productivity,” “cancellation,” “loss to follow-up,” “open slots,” “visit lost to outside providers,” “patient seen on their day of choice” and “workload.”

The data sources for indicators varied from electronic forms, EMRs, clinic logs (i.e., non-electronic databases kept by health services), validated surveys such as the Usual Providers Continuity Index (Jee and Cabana 2006) and Continuity of Care Index (Bice and Boxerman 1977) to survey tools developed by researchers. Only 33% of the articles were based on EMRs, and these covered six indicators (wait time for appointment, no-show, continuity, productivity, cancellation and loss to follow-up).

EXPERT CONSULTATION

The expert committee added six other indicators that could be derived from EMRs or administrative databases to the 12 indicators identified by the systematic review: (1) clinician supply, (2) patient demand, (3) consultations in other PHC clinics, (4) future capacity, (5) professional diversity of care and (6) use of walk-in clinics. The committee recommended adding characteristics of the supply and demand of each clinician to estimate balance. This consisted of creating a profile of registered patients by age and assigning them a vulnerability code to estimate the appropriate number of consultations. These two indicators support clinician reflection on the planning of service delivery.

Objective #2: Pilot the extraction and measurement of access indicators

From the list of 18 indicators, 10 were shortlisted by the research team based mainly on the capacity to measure them using EMR data (see Figure 2 and Table 2).

We piloted the extraction of the data for these 10 indicators. Discontinued care for patients with chronic diseases was a key indicator for the expert committee. However, there was no consensus on which chronic diseases to target. Moreover, the quality of the data entered for chronic disease status varied greatly across clinics and among professionals. For these reasons, we eventually stopped piloting this indicator. In addition, manually extracting data to measure future capacity was found to be quite labour intensive. Thus, the extraction of this indicator was put on hold during piloting and was not integrated into the final indicators.

Objective #3: Create a dashboard to longitudinally compare access indicators to support reflective practice

We developed a real-time dashboard structure based on eight indicators; two indicators related to characteristics of the clinical practice (clinician supply and patient demand) and six related to access (third next available appointment, relational continuity, 48-hour capacity, use of walk-in appointments, professional diversity of care and no-shows). We extracted and measured the eight indicators for 151 PHC professionals at eight clinics, including 123 family physicians, 16 registered nurses and 12 health professionals (psychologists, social workers, pharmacists) over 12 months (January 2022 to December 2022).

Clinician supply and patient demand are measured yearly, third next available appointment and future capacity are measured weekly and 48-hour capacity is measured daily. The remaining indicators are measured every month.

The dashboard was constructed for health professionals to compare data with others in their own clinics and with other professionals sharing the same type of practice in other clinics, and to compare data among clinics. This also helps show the effects of improvements or differences across regions.

ESTIMATED BALANCE

We calculated the yearly supply of each clinician based on the number of appointments given in the previous year. This indicator can be compared with the estimated demand of their practice based on the number and characteristics of their patients in order to evaluate if their practice is balanced or if they need to adjust their workload.

Demand is estimated based on the recommendations of the Fédération des médecins omnipraticiens du Québec (FMOQ), which establish a number of consultations per year for each patient based on a set of characteristics, such as age and vulnerability (Table 3). Patients were considered vulnerable if they were assigned at least one of the 19 vulnerability codes defined by Quebec's Health Insurance Board, the Régie de l'Assurance maladie du Québec, based on the presence of certain diagnoses (e.g., diabetes, chronic obstructive pulmonary disease, mental health disorders) (Breton et al. 2015).

Supply is calculated for each clinician's practice in the previous year (Table 4). Because many clinicians have diverse practices that include non-registered patients or walk-in patients registered with other clinicians at the same clinic, a detailed summary of the various types of appointments provided is included in the dashboard.

These data can help each clinician adjust their practice. As seen in Table 4, the physician did not provide enough consultations for their registered patients in the past year but provided many more services to non-registered patients than the rest of the clinic. Based on these results, they may choose to focus their practice on their registered patients or transfer some patients to other clinicians at the clinic.

THIRD NEXT AVAILABLE APPOINTMENT

The third next available appointment is recognized as one of the key indicators of advanced access. Studying variations over a long period of time and comparing them with other clinicians at the same clinic can help clinicians make better decisions regarding their practices. In the example shown in Figure 3, the third next available appointment is available sooner with Dr. A than with the rest of the clinic. During summer, we have observed a longer delay.

RELATIONAL CONTINUITY

Continuity is an important indicator for advanced access as it evaluates whether or not each clinician's registered patients can obtain an appointment with their own registered clinician or if they have to consult other doctors, who are often less familiar with their health conditions. Comparing continuity of care across different clinics can help managers address issues within their establishments. The example provided in Figure 4 shows the distribution of rational continuity at each quarter at a clinic compared with the region's average and distribution over the past year.

48-HOUR OPEN APPOINTMENTS

The proportion of appointments available within 48 hours is a strong indicator of clinician availability for urgent care, another important part of the advanced access model. This indicator can be captured in real time on a daily basis, and comparisons of the individual results of each clinician with similar clinicians in their region can provide them with critical information to adjust to trends in data. Figure 5 shows that Dr. B has inconsistent availability for appointments at a 48-hour notice, having a high percentage of open appointments for some weeks and no open appointments for others.

Discussion

We demonstrated that it is possible to create a dashboard on PHC access based on real-time data extracted from different EMRs. We identified eight key indicators based on a systematic literature review, consultations with experts and piloting in eight clinics. Two more indicators were identified as relevant, “discontinued care for patients with chronic diseases” and “future capacity,” but required further efforts to ensure their validity and feasibility.

The indicators “third next available appointment,” “relational continuity” and “use of walk-in clinics” offer data related to the general availability of clinicians and insight into timely access to needed services. These indicators measure the number of patients who cannot access the physician with whom they are registered and have to resort to consulting other physicians at the clinic. Continuity and access to primary care are two important and connected concepts (Haggerty et al. 2003). In fact, a recent Canadian study reported that decreased PHC access was associated with a 6% reduction in a physician's continuity of care and a higher number of emergency department visits (Cook et al. 2020). When access to a clinician or team who know the patient is impeded, it follows that the patient will seek services from another clinician or organization, which jeopardizes the continuity of services. Another study from our research team revealed that patients preferred waiting a few days for a consultation with a professional who knew them rather than having a more rapid consultation with any healthcare professional, even for urgent or semi-urgent needs (Breton et al. 2022). The main challenge becomes finding a way to improve both timely access to PHC and continuity with a professional and an organization that know the patient.

Two of the key indicators retained are designed to evaluate the level of day-to-day availability of clinicians for their patients. The indicator “48-hour capacity” represents the availability of primary care providers for urgent care for their patients, whereas “third next available appointment” quantifies the general week-to-week availability of doctors. The third next available appointment is the most accurate indicator for measuring timely access in PHC (Pickin et al. 2004). It reflects availability more accurately than the first or second available appointments as these can result from a recent cancellation or an unanticipated event (Breton et al. 2020). Along with “future capacity,” these indicators provide insights into whether patients can be seen at the right time with regard to their needs (urgent, semi-urgent or non-urgent) and whether a professional has a balanced schedule and can manage variations in demand over time.

“Professional diversity of care” is a good measure of quality rather than access and ensures that each patient is seen by the appropriate professional. This is central to a timely access strategy (Murray et al. 2003). Often patients default to seeing a physician for any of their healthcare needs, which is not always effective to ensure either the quality of care or an optimal use of resources. For example, prescription renewals for a condition deemed stable could be done by a registered nurse or a pharmacist. Some mental health consultations could also benefit from direct access to a social worker or a psychologist. Making sure that all professionals at PHC clinics work in a way that maximizes their scope of practice is crucial to achieving high quality in healthcare.

The proportion of appointments where the patient was not present (i.e., “no-shows”) has been recognized in the literature as a good proxy for general access to care. Patients who cannot access quality care within a reasonable time frame are more likely to miss appointments because they no longer need treatment, have received care elsewhere or have simply forgotten. Therefore, an underlying timely appointment system leads to greater efficiency with fewer missed appointments. Of note, our study revealed relatively fewer no-shows (average 2%) and greater stability in this indicator when the implementation of timely access strategies was well underway. Paying attention to this indicator may, therefore, be more relevant during the early stages of the implementation of such strategies or when access is quite limited for a given clinician (Steinbauer et al. 2006).

The indicator of “discontinued care for patients with chronic diseases” is quite relevant in the context of implementing timely access strategies. Most often, such strategies rely on making patients responsible for booking appointments when they need them as opposed to maintaining a backlog or recall lists (Breton et al. 2017). Several professionals have voiced concerns with respect to losing some of their most vulnerable patients by not providing appointments in advance (Abou Malham et al. 2017). This indicator could then function as a balancing indicator and act as an unseen benefit to help avoid unintended consequences of such a care organization strategy.

The piloting of data extraction was done as part of a research project under the regulation of an ethics committee. Such a process was instrumental and essential to access nominal data to develop a dashboard. Creating a dashboard within a research project under the jurisdiction of a research ethics committee temporarily allowed us to circumvent regulatory requirements, a strategy that has been shown to be efficient for other innovation scale-up projects in PHC, such as eConsultation (Breton et al. 2019; Moroz et al. 2020). This facilitated the consent process, from the clinics' perspective, to provide access to confidential and sensitive data. Access to nominal data is an important barrier to scaling up the use of a dashboard by policy makers.

Another barrier to gathering data into a single dashboard is the fact that the data must be extracted from various EMRs, which are under the governance of various private EMR providers. This is an important barrier to accessing and extracting data in a secure manner for a single repository. EMRs differ substantially within and between provinces, and the regulations surrounding data management and use may also vary. Standardizing data management policies would greatly facilitate the process of data extraction and comparisons within and among organizations and across provinces.

One of the success factors for clinics to adopt the dashboard was the offer of a personalized feedback report in the near future along with comparisons with similar individuals and organizations. We believe that providing each professional with a reliable picture of their own practice, its evolution over time and how it compares with those of their peers will contribute to the implementation of key elements to facilitate an improvement process to increase access to and, more broadly, quality of care.

Another success factor was working in partnership with professional associations to provide continuing education credits following a reflective activity supported by the personalized feedback report. To do so, professionals will be invited to complete a brief continuous quality improvement activity and to document the experience (identifying areas for improvement, setting an improvement goal and identifying ways to assess whether the change resulted in improvement). For this project, the FMOQ will provide the equivalent of one hour of training to complete this reflective exercise.

The creation of a dashboard is an essential component to improve access and support professionals and decision makers in quality improvement projects. Investing in the development and implementation of such a dashboard is even more important considering that Canada is one of the worst countries on access indicators (CIHI 2017, 2021). The next step in the creation of a provincial dashboard will be to increase the number of participating clinics, which will be facilitated by using an automated extraction approach fed by various EMRs from all PHC clinics. An automated extraction software fed by five EMRs has been developed in collaboration with a specialized programming firm and is operational. We are in the process of obtaining accreditation from the Ministry of Health and Social Services in Quebec to authorize external connections between the automated extraction software and EMRs in compliance with safety standards.

Ultimately, the adoption of a provincial dashboard by professionals and practices will play a key role in establishing a comprehensive benchmarking system. Benchmarking can be defined as the process of measuring key indicators, identifying the organizations or settings that score most favourably on these indicators, understanding the characteristics and practices of these top organizations and spreading these practices to bring other organizations to the same level (Ellis 2006; Ettorchi-Tardy et al. 2012). Benchmarking is one of five approaches to instigate significant changes in healthcare. The next step is to spread the dashboard and create a benchmarking system to learn from the top-performing organizations and professionals.

Strengths and limitations

One of the strengths of this study is that it draws on several complementary methods to develop a valid, acceptable and reliable dashboard on PHC access. We have demonstrated that it is possible to connect different EMRs to a common dashboard in real time.

Another strength lies in our capacity to respect data privacy when producing comparative reports and our independence from any governing or decision-making bodies. Our research team has access to these nominal data with the permission of an ethics committee and the agreement of the clinicians. Data are always presented anonymously, even when comparisons are made at the organizational level.

At the current time, one of the main limitations of the access indicators extracted from the EMRs is that they do not allow for the evaluation of any indicators outside the clinic. Access to data on services used outside the clinic would make it possible to better grasp the trajectory of patients for access to PHC services.

Conclusion

Timeliness of data reporting could support quality improvement initiatives and add value to our healthcare systems. The use of EMRs embedded in organizations offers exceptional potential for appreciating and contributing to the improvement of access in real time. Access is a priority for health system improvements, and tools must be developed to assess PHC access in a contemporary and meaningful manner. The dashboard on access is a key step to support quality improvement. To support professionals and organizations to improve PHC, political authorities have to support and fund the creation of a provincial dashboard on PHC to fuel data-informed reflective practice.

Funding

This study was funded by the Quebec's Ministry of Health and Social Services, the “Soutien aux recherches axées sur le patient” support unit and the Fond de Recherche en Santé du Québec (grant no. 279742) and the Canadian Institutes of Health Research (CIHR) (grant no. 399757).

Correspondence may be directed to: Mylaine Breton. Mylaine can be reached by e-mail at mylaine.breton@usherbrooke.ca

Utilisation des données du dossier médical électronique pour créer un tableau de bord sur l'accès aux soins primaires

Résumé

Objectif : Cette étude vise à présenter une preuve de concept d'un tableau de bord sur un ensemble d'indicateurs d'accès aux soins de santé primaires (SSP) basés sur les dossiers médicaux électroniques (DME).

Méthode : Cette recherche s'appuie sur une étude de conception multi-méthodes comprenant (1) une revue systématique, (2) une phase pilote et (3) le développement d'un tableau de bord.

Résultats: Huit indicateurs ont été soigneusement sélectionnés et extraits avec succès des DME obtenus auprès de 151 prestataires de SSP. Des indicateurs d'accès dans le temps, ainsi qu'entre prestataires et entre cliniques, ont été activés dans le tableau de bord.

Conclusion : Les données des DME ont permis de développer un tableau de bord en temps réel sur l'accès, donnant aux prestataires de SSP un portrait fiable de leur propre pratique, de son évolution dans le temps et de sa comparaison avec celles de leurs pairs.

Comments

Be the first to comment on this!