What did we do? We reviewed the degree to which existing population-based data in Canada can be used to describe and report on primary healthcare (PHC) performance. We identified gaps in current data sources and made recommendations on how these gaps might be addressed to support quality improvement and public reporting for PHC.

What did we learn? Population-based survey and administrative data are available to describe population characteristics and other contextual factors for PHC, as well as some aspects of the material, financial and human resources inputs, and selected activities and decisions at the policy, management and clinical levels. Existing data can also be used to describe some volumes and types of PHC outputs. However, we currently have limited population-based data to assess selected qualities of PHC services (e.g., coordination and interpersonal effectiveness) and most immediate outcomes of PHC. The ability to link data to assess outcomes and attribute changes in outcomes to PHC is limited. A full report describing more than 130 indicators from existing data sources and gaps in current data is available at www.chspr.ubc.ca.

What are the implications? As we look to the future, there is a clear need to build on existing data sources to expand PHC data capacity in Canada. Data are needed to inform an understanding of PHC outputs, outcomes and the linkages among PHC dimensions. Commitment to a comprehensive PHC data collection strategy and information system is needed across Canadian provinces and territories to inform policy development and planning, to evaluate PHC redesign initiatives and to meet the accountability expectations of Canadians.

Canadians have expressed concern about access to and quality of primary healthcare (PHC), as well as support for changes to PHC including collaborative team-based care (Pollara 2005; Watson and Krueger 2005). They have also indicated concern for accountability and a desire for increased transparency and reporting on the results of healthcare expenditures and renewal initiatives (Pollara 2003). In response, federal, provincial and territorial governments have made substantive, multi-year investments in healthcare reform and have committed to regular reporting on health system performance and progress towards system renewal in the First Ministers' communiqué on health (Canadian Intergovernmental Conference Secretariat 2000), the Primary Health Care Transition Fund (Health Canada 2000), the First Ministers' Health Accord (Health Canada 2003) and the First Ministers' 10-Year Plan to Strengthen Health Care (Health Canada 2004).

Despite investments in PHC renewal and a commitment to increased public reporting, a performance measurement and accountability framework for this sector has not been established in Canada. Such a framework is needed, along with population-based data, to support policy and management initiatives to improve quality and report on progress resulting from renewal initiatives. The creation of information systems to support this agenda should leverage existing data collection, monitoring and reporting and be supplemented with new data sources and systems that fill information gaps in priority areas.

In response to this challenge, researchers at the Centre for Health Services and Policy Research developed the Results-Based Logic Model as a guide for developing a population-based PHC information system and evaluating PHC (Watson et al. 2004). Concurrently, we undertook a review of PHC performance measurement in Canada to identify existing data sources for PHC reporting. The purpose of this paper is to illustrate how existing population-based administrative and survey data can be used to describe dimensions of PHC. We identify gaps in existing data and priorities that should be addressed for PHC reporting, and offer suggestions to address these gaps. This report will be of interest to those responsible for planning future data requirements and information systems for PHC, along with those undertaking PHC evaluation and research across Canada.

What Did We Do?

Our work was guided by the Results-Based Logic Model for PHC described earlier in this special issue of Healthcare Policy (Watson et al. 2009a; see page 33). The Results-Based Logic Model was developed using the approach of the Treasury Board of Canada to performance management and accountability reporting. This approach focuses on measuring and reporting outcomes throughout the life cycle of a policy, program or initiative and integrating strategy, people, processes and measurement to improve decision-making and drive change. Beginning with the creation of a results-based logic model, this approach uses indicators to assess performance along with analyses to link variation in resource inputs and activities performed to changes in outputs and outcomes achieved (Treasury Board of Canada 2001).

We used the Results-Based Logic Model to guide our review of population-based data sources and performance measures in Canada. To identify performance indicators that are currently available for dimensions of the Results-Based Logic Model, we examined existing population-based data sources, including surveys of citizens, surveys of providers and reports describing the use of administrative data. This information illustrates the potential to use existing data sources to measure PHC inputs, activities, outputs and outcomes. The data sources and performance measures identified in this paper are not exhaustive; rather, the information is intended to illustrate availability from existing robust and accepted data sources. Similarly, this information is not intended to describe the "best" or preferred performance measures. Our full report, including more than 130 performance indicators, is available on the website of the Centre for Health Services and Policy Research (Broemeling et al. 2006).

This information was then used to identify gaps in current data available to report on PHC dimensions and renewal initiatives. We also used results from work, as well as consultations with citizens, to identify priorities for information and make recommendations to guide planning for future data collection and PHC information systems (see Watson et al. 2009a; page 35 in this special issue).

What Did We Learn?

Table 1 provides examples of measures that are available or can be derived from existing survey and administrative data sources. Measures of intermediate and final outcomes are not described in Table 1 because the attribution of these outcomes to PHC is limited by the impact of external factors.

Population characteristics and contexts

Population characteristics include health status, behavioural, psycho-social and social support measures and health-related attitudes and preferences. Contextual factors such as social cohesion, social capital and income disparity, and policy and legislative contexts also influence health, patterns of use and delivery of PHC services. A number of population and contextual measures are available or can be derived from existing data sources. Administrative data from provincial medical services plans, vital statistics and program utilization files can be used to describe populations, morbidity, treatment prevalence rates for chronic health conditions and mortality. Survey data, including the Canadian Community Health Survey (CCHS) (Statistics Canada 2006b), also provide such important information on population characteristics as self-rated health, disability, activity restrictions, self-reported chronic health conditions, behavioural risks and medical interventions.

PHC inputs

Human, material and fiscal resources are the inputs that PHC systems rely on to carry out activities, deliver care and achieve results. Health human resources input measures are available both from provincial administrative data sources (provincial colleges of physicians and surgeons registrations and colleges of registered nurses, provincial medical services plans) and from national databases such as the National Physician Database. Survey data sources include such measures as intention to retire and plans to move during the coming year from the National Physician Survey (NPS). Material inputs (e.g., information systems, electronic health records) and financial inputs are also available from administrative and survey data sources.

PHC activities

PHC activities describe the work processes undertaken in preparation for delivery of PHC products and services. Activities support the production of outputs and the achievement of outcomes. Three levels of activities are identified: policy and governance, healthcare management and clinical. Policy and governance measures include financing and funding of services. Funding information is currently collected from provincial medical payment administrative data; financing information could be, but is not currently, collected from patient surveys.

Healthcare management activities include decisions about the degree to which practices include interdisciplinary teams or offer services 24 hours a day, seven days a week. Data relating to practice settings and organization are available from the NPS (CFPC 2008). Information about the degree to which patients receive PHC from multidisciplinary teams could be obtained from population surveys, such as the Canadian Survey of Experiences with Primary Health Care (CSE-PHC) which was conducted in 2007 and 2008 (Statistics Canada 2009). Clinical-level activities and decisions can also be measured using existing data sources. For example, physician participation in delivery of maternity care is available from NPS data. Many healthcare management and clinical activity measures focus on physician data sources; less information is available for other providers.

PHC outputs

Outputs are the services and products that result from PHC activities: that is, the services delivered by PHC providers and received by patients. The volume and type of PHC services used by the population overall and by population subgroups can be described using such measures as annual check-ups, screening tests and physician visits for mental health conditions with information from both administrative and survey data. Administrative data sources quantify the number and type of visits covered by publicly funded services, while survey data describe service use from patient and provider perspectives. The 2008 CSE-PHC was conducted by Statistics Canada and designed to offer national and provincial-level estimates regarding the types of services that adults receive (e.g., health promotion, disease prevention) as well as the places they visit for that care (e.g., walk-in clinics).

Qualities of PHC outputs describe the distinctive features of PHC – first-contact accessibility, comprehensiveness of services, continuity, cultural sensitivity, interpersonal communication, respectfulness, technical quality of clinical care and so on. Selected qualities of PHC outputs can be described using existing data sources. Accessibility can be reported using both survey and administrative data. Surveys such as the CCHS (Statistics Canada 2006b) include perceptions of accessibility and barriers to care, availability of services when needed, convenience of PHC services and degree to which there is unmet need. The 2008 CSE-PHC (Statistics Canada 2009) includes information on wait times and acceptability of those wait times for routine and ongoing care as well as immediate care for minor health problems. Administrative data provide measures of realized access such as use of services.

Relational continuity of care can be assessed using existing CCHS data from patients (e.g., respondents report having a regular medical doctor) and CSE-PHC data (e.g., how long have you been receiving care from your regular primary care provider), while administrative data have been used to derive continuity of care indices such as usual care provider or sequential continuity of care. Measures of informational and management continuity are generally not available from administrative data sources and require either survey or clinical information to assess (Reid et al. 2003). The CCHS does not currently include questions to assess patient experiences of PHC continuity of care.

Comprehensiveness measures describe the span of services provided in PHC, from health promotion and disease prevention through acute episodic care, ongoing chronic care and end-of-life care. Measures of comprehensiveness, such as the provision of primary or secondary preventive services, provider activity and referrals to specialist and other providers, are available using NPS survey data, as well as administrative data from provincial medical services plans. The CSE-PHC does not include questions to assess receipt of a comprehensive array of health promotion, disease prevention and curative healthcare services from a patient perspective but it does ask respondents if they receive the full range of services they need.

Coordination, another key attribute of PHC, cannot be assessed using administrative data and must rely on survey data. While co-occurrence of services can be identified using linked administrative data (e.g., post-acute use of home care services), it is not clear whether services were actually coordinated with shared provider information and shared care management. Interpersonal communication describes provider–patient communication, shared decision-making and interpersonal style of care, and reflects the degree to which providers offer patient-centred care (Watson et al. 2004). Measures of interpersonal communication cannot be assessed using administrative data, but the CSE-PHC includes a number of items intended to measure coordination and interpersonal communication.

Finally, effectiveness of care or technical effectiveness reflects the degree to which care is consistent with evidence and knowledge and there is high consensus on the importance of this type of information to drive quality improvement (Barnsley et al. 2005). To a limited extent, administrative data can be used to assess technical effectiveness by comparing actual care provided to patients to that recommended; that is, comparing the reported use and frequency of tests, procedures and medications compared with evidence-based guidelines. Measures of technical effectiveness are not currently part of the CCHS, but a module to assess the degree to which people with diabetes reported receiving recommended laboratory tests and procedures was included in the CCHS in 2005. This module was optional, and only six jurisdictions funded data collection in their regions. More recent modules of the CCHS are being designed to measure technical quality for other chronic conditions and the CSE-PHC includes a number of items intended to measure this quality of PHC services.

Primary healthcare outcomes

PHC outcomes are the result of inputs, activities and outputs, with immediate outcomes being most attributable to PHC.

IMMEDIATE OUTCOMES

Immediate outcomes describe increased knowledge about health and healthcare among the population, reduced risk, duration and effects of acute and episodic conditions, reduced risk and effects of continuing health conditions, and maintenance or improvement of the work life of the PHC workforce (Watson et al. 2004). Perhaps the best information to assess PHC immediate outcomes from the provider perspective is to assess job satisfaction and work life measures which are available from existing survey data such as the NPS (for physicians). The other PHC immediate outcomes are best measured from the perspective of patients and using longitudinal data. The CSE-PHC does offer some cross sectional data regarding Canadians' knowledge about their health and care management which can be used as a basis for performance indicators and that survey offers data on PHC activities and outputs to support attribution (causal) analyses.

Performance measures for reduced risk and effects of chronic health conditions are best measured from the perspective of patients and using longitudinal data. The CSE-PHC does offer some cross sectional data in this regard, but population-based administrative data are not readily available. While administrative data have been used to measure receipt of recommended care for patients with diabetes (technical effectiveness: PHC output), there are few jurisdictions that have administrative data which include the results of laboratory work (e.g., cholesterol or hemoglobin A1c levels) or diagnostic procedures (e.g., blood pressure) that can be used to measure the effects of care for chronic health conditions such as diabetes. Where such administrative data capacity exists, there is little capacity to link these data to information about PHC inputs, activities and outputs.

INTERMEDIATE OUTCOMES

Intermediate outcomes include outcomes on which PHC is expected to have an impact, albeit with less control: appropriateness of provider and place, healthcare system efficiency, acceptability and equity (Watson et al. 2004). Measures for acceptability and appropriateness of place are available from survey data, including patient perspectives (CCHS and CSE-PHC) and physician perspectives (NPS). On the other hand, existing administrative data have been used to measure emergency department and hospital use (Cree et al. 2006) and ambulatory care sensitive (ACS) admissions to hospital (CIHI 2005b).

Canadian researchers have had to be creative in measuring PHC efficiency related to differences in technical effectiveness of specialists and general/family practitioners in the delivery of chronic disease management, because there are cost differences associated with variations in physician mix (Shah et al. 2003) or work to assess equity and efficiency in general/family practitioner, nursing, emergency department and home care services relative to the health status of populations (Watson et al. 2005).

FINAL OUTCOMES

Final outcomes include a sustainable healthcare system, improved/maintained individual health and functioning, and improved level and distribution of population health and wellness (Watson et al. 2004). The complex interplay of PHC, other system factors and variations in population contexts limit the extent to which final outcomes can be attributed to PHC. The outcome of a sustainable healthcare system reflects expectations that PHC is a key lever to manage inputs, costs and efficiency of the healthcare system. Evidence from other jurisdictions suggests that healthcare systems with strong PHC have lower overall costs than those that do not (Starfield and Shi 2002). However, there are few assessments of sustainability of the PHC sector; some explore the impact of temporal trends at the population level on future sustainability (Watson et al. 2007). In Canada and elsewhere, researchers have used area-level measures of PHC to study the influence of the PHC sector on the health of populations, healthcare and health status (Pierard 2009; Shi et al. 2003). For all intermediate outcomes and final outcomes, the challenge is to attribute changes in outcomes to PHC as opposed to other system factors.

What Are the Implications?

Population-based data are available to describe some dimensions of the Results-Based Logic Model (Watson, Broemeling et al. 2009) and to evaluate selected aspects of PHC renewal. However, despite availability of these data and the importance of accountability and performance reporting to Canadians, there is little routine monitoring and public reporting for this sector. Such reporting would provide important information for delivering PHC services, planning PHC renewal, monitoring changes in PHC performance and assessing the impact of renewal initiatives. It would also inform the 93% of Canadians who reported that PHC is of high importance (79%) or medium importance (14%) to their future health (Health Canada 2006).

Although data are available to measure some aspects of PHC performance, many input and activity measures focus on physician data and there is less information available for other PHC providers or interdisciplinary teams. Recent studies by our team using administrative data to identify PHC nurses (Wong et al. 2006, 2009; see page 91 in this special issue of Healthcare Policy), and by CIHI and Statistics Canada to survey nurses (Shields et al. 2006), are useful additions to describe nursing's contribution to PHC service delivery.

Moreover, because an increasing proportion of physician services are financed through alternative funding arrangements (CIHI 2006), the availability of administrative data for performance measurement is declining. Two papers in this special issue (Broemeling, Kerluke et al. 2009 and Watson, Peterson et al. 2009) assess the degree to which completeness and accuracy of administrative data are eroding for measurement of provider and population perspectives of PHC and what can be done so that administrative data can still be used for performance measurement in Canada.

As well, our current ability to assess some qualities of PHC outputs – interpersonal communication, continuity and coordination of care – has been limited; the new CSE-PHC will enhance capacity to measure these dimensions based on individuals' experiences and linkage of that survey to administrative data will enhance capacity to measure the impact of variation in these qualities on longer term outcomes such as use of hospitals.

There are notable gaps in existing data to assess the immediate outcomes of PHC, particularly those outcomes that require longitudinal data. Little information is currently available to assess the impact of PHC on the risks, duration and effects of acute and episodic health conditions, or the risks and effects of chronic health conditions. Data are also not readily available to measure the impact of PHC on the immediate outcome of increased knowledge about health and healthcare among the population. It is possible to incorporate this information into surveys or to extract it from electronic data; however, if the information is to be used to improve the quality of PHC, it must be linkable to information on outputs (at a minimum) or, more ideally, to inputs, activities and outputs to support evaluative research and analyses.

Selected intermediate and final outcomes can be assessed with existing data sources, but these measures are neither comprehensive nor easily linked to support the attribution of PHC inputs, activities and outputs to PHC outcomes. Many external factors, in addition to PHC, contribute to these intermediate and final health outcomes, limiting our ability to assess PHC's contribution. Longitudinal data sources linking patients and providers are needed if we are to assess the impact of PHC on individual and population outcomes and health improvements over time, as well as threats to, and opportunities to support, healthcare system sustainability.

A full evaluation of PHC renewal will require a comprehensive data collection strategy that accounts for the dimensions of PHC and the links among PHC dimensions. For example, do interdisciplinary teams improve access to recommended care for those with chronic health conditions? And how do interdisciplinary teams working to improve access to care influence the relational continuity of care between patients and providers?

A comprehensive data collection strategy would have to provide population-based, multi-level data for individual residents, patients, providers, clinics and organizations. Such an information system must support linkage of individual patients to providers, clinics and other healthcare organizations in order to attribute changes over time to PHC services and renewal initiatives. Longitudinal data are required to support temporal analyses of factors contributing to the health of Canadians. Finally, comprehensive data are needed across the PHC dimensions, from contexts to inputs, activities and outputs to outcomes, as well as the linkages among each of these dimensions.

Recommendations

We conclude that there is population-based data capacity to begin to describe the population characteristics, contexts, inputs and activities of PHC, but insufficient data to describe outputs and outcomes and to attribute outcomes to this sector. Focusing on data sources to measure immediate outcomes for PHC, along with the linkages among contexts, inputs, activities, outputs and outcomes, will be an important priority in supporting local, provincial and national evaluation of PHC renewal.

We recommend commitment to a comprehensive PHC data collection strategy and information system. In the short term, priorities should be established as follows:

1. Establish standardized data collection strategies across jurisdictions to address the completeness and accuracy of traditional administrative data holdings for general and family practitioners. This action is required to address the erosion in administrative data resulting from the increase in alternative payments and PHC services delivered via such arrangements.
2. Implement a regular survey of Canadians to track the unique and distinguishing features of PHC, such as type and volume of health promotion, disease prevention and curative services received, as well as patient experiences with continuity, coordination of care and interpersonal communication. Such data could be used to assess the degree of variation in volume, type and qualities of outputs with self-reported outcomes such as use of other healthcare services. While the CSE-PHC was conducted in 2007 and 2008, there are no plans dedicated towards ensuring that it be conducted on an annual or biannual routine basis.

3. Implement studies to assess policy-relevant activities such as impact of the introduction of multidisciplinary teams on PHC outputs and outcomes. Ad hoc studies are required because administrative and survey data cannot be linked to information about PHC organizational design.

   In the medium to longer term:

4. Ensure that electronic health records are designed and implemented so that data for PHC outputs and outcomes are complete and reliable, and can be routinely linked to information about PHC inputs, management and clinical-level activities, population contextual factors and use of other healthcare services. A comprehensive information system would support analyses to inform quality improvement efforts and enable public reporting.

There is remarkable potential to develop comprehensive, longitudinal and multi-layered data for PHC that supports performance measurement and research at the population or system level and, at the same time, supports providers with information on care delivery to inform quality improvement efforts and planning for future needs. We recommend the development of a Canadian PHC data collection strategy that builds on the unique qualities and strengths of population-based data. A comprehensive, collaborative PHC data collection strategy, combined with a common evaluation framework, would provide the necessary building blocks for PHC evaluation and performance measurement. The final paper in this issue (Watson 2009) offers a roadmap of options for new information systems so as to address existing gaps in high priority information about PHC in Canada. Commitment to this work is necessary if we are to meet the expectations of Canadians.

Mesure du rendement en matière de soins de santé primaires : capacité actuelle et information potentielle pour les analyses fondées sur la population

Résumé

Ce que nous avons fait : Nous avons examiné à quel point les données démographiques actuelles peuvent être employées pour faire un compte rendu du rendement des soins de santé primaires (SSP). Nous avons décelé des lacunes dans les sources actuelles de données et nous avons fait des recommandations afin de traiter ces lacunes, dans le but de favoriser une amélioration de la qualité et de la diffusion publique d'information en matière de SSP.

Ce que nous avons appris : Il existe des données administratives ou issues de sondages auprès de la population qui peuvent servir à décrire les caractéristiques de la population ou d'autres facteurs contextuels pertinents aux SSP, de même que certains aspects touchant aux intrants en matière de ressources matérielles, financières et humaines, et certaines activités ou décisions aux niveaux politique, clinique ou de gestion. Les données disponibles peuvent également servir à décrire certains types et volumes d'extrants dans les SSP. Toutefois, il existe actuellement peu de données de nature démographique qui permettent d'évaluer les critères de qualité en matière de services de SSP (par exemple, la coordination et l'efficacité interpersonnelle) ou encore les résultats immédiats pour les SSP. La possibilité d'associer les données pour évaluer les résultats et les changements connexes est également limitée. Un rapport détaillé décrivant plus de 130 indicateurs à partir des sources de données actuelles, et présentant les lacunes dans les données, est disponible sur le site www.chspr.ubc.ca.

Répercussions : Dans une perspective d'avenir, il faut clairement enrichir les bases de données actuelles afin d'accroître le potentiel des données sur les SSP au Canada. Les données sont nécessaires pour faciliter la compréhension des extrants et des résultats des SSP et pour faire des associations entre les divers aspects des SSP. Un engagement envers une stratégie pour la collecte de données détaillées et envers un système d'information est indispensable dans toutes les provinces et tous les territoires canadiens afin de renseigner l'élaboration et la planification des politiques, d'évaluer les initiatives de restructuration des SSP et de satisfaire aux attentes des Canadiens en matière d'obligation redditionnelle.

Comments

Be the first to comment on this!