Healthcare Policy

Healthcare Policy 5(Sp) November 2009 : 77-90.doi:10.12927/hcpol.2009.21188
Research Paper

Methods to Develop and Maintain a Valid Physician Registry in Evolving Information Environments

Diane E. Watson, Sandra Peterson, Ella Young and Bogdan Bogdanovic

Abstract

What did we do? As the amount of alternative funding for physician services grows in Canada, the usefulness of fee-for-services (FFS) payment data as a source of population-based information declines. This paper describes methods used to develop and validate an anonymous registry of the physician workforce to support policy-relevant analyses in environments where doctors are increasingly funded through diverse arrangements.

What did we learn? Among the 8,558 physicians in clinical practice in British Columbia in 2004, 97% could be identified via FFS payment data. In 1996, 2000 and 2004, a similar number of physicians in clinical practice (N=340, 326, 290) did not submit FFS claims, and a stable proportion of these doctors (65%, 67%, 69%) could be identified using hospital discharge abstract data. Province-wide, local health areas varied in the proportion of total physicians' expenditures in 2004 attributable to FFS payments (0 to 100%).

What are the implications? FFS and hospital files could be used in tandem to identify physicians in clinical practice in order to create complete registries intended to support population-based workforce analyses. FFS and alternative funding payment files could be used together to calculate each physician's income and clinical activity, and to identify physicians for whom there is high likelihood of measurement error because they reside in local health areas where a large proportion of clinical activity is not captured.

Recommendations: Systems designed to track alternative funding should be uniformly structured within and across jurisdictions to maintain or increase the availability of population-based clinical data useful for secondary analyses. Only then can these data be used to support Canadian policy, management and clinical decision-making.

Health human resources planning, deployment and effectiveness have been identified as top priorities in national consultations intended to illuminate issues where new research and analysis could inform efforts to renew healthcare in Canada (CIHR and CHSRF 2008). Better information on the clinical activity of the healthcare workforce has been deemed necessary to inform the decisions, actions and policies of governments, as well as educational, regulatory and professional bodies.

Over the past decade, provincial fee-for-services (FFS) payment data have increasingly been used to understand the supply and distribution of physicians, patterns of practice among these providers and the use of their services by Canadians. These databases have proven particularly useful to profile trends in services delivered by physicians and received by patients. More recently, however, the validity of analyses based on FFS payment data have come into question, with growth in alternative (i.e., non-FFS) funding schemes and a lack of uniform standards to guide the establishment of new databases designed to track those payments or to support pooling of data from FFS and non-FFS payment files.

By 2003/04, the percentage of physicians who received some alternative funding from provincial ministry funders ranged from 7% in Alberta to 73% in Prince Edward Island and Nova Scotia. Moreover, the percentage of clinical payments attributable to alternative funding ranged from 10% in Alberta to 42% in Newfoundland and Labrador (CIHI 2006). As the size of alternative funding for physician services grows in Canada, the usefulness of FFS payment files as a sole source of information on physician–patient encounters may be eroding.

FFS payment data can no longer be used in isolation to profile supply of physicians, practice patterns of the physician workforce or individual physicians if (a) a sizable proportion of doctors are no longer funded by governments with FFS, (b) doctors no longer receive a substantial share of their income from this source and (c) similar data structures are not available across FFS and alternative funding payment files to permit data pooling so as to ensure completeness of clinical service data for each physician. Thus, new approaches to designing population-based information systems are necessary in jurisdictions where physicians are increasingly funded through diverse arrangements.

An important first step to support valid, policy-relevant research and analyses in this evolving information environment is the creation of a valid physician registry that identifies the entire population of physicians as accurately and completely as possible, supports data linkage and pooling from an increasing array of sources and offers insights regarding the degree to which the data represent a complete account of clinical services (i.e., measurement error). While population-based registration information systems, such as those held by licensing bodies, can be used to identify the entire workforce, they cannot be used routinely as a registry to support data linkage. Additionally, the proliferation in the number and type of payment files reduces the likelihood that any one of them, such as FFS payment files, could be used as a registry.

This paper describes methods used to develop and validate an anonymized registry of physicians deemed to be delivering clinical services, illustrates how the registry can be used to estimate measurement error at the regional and physician levels and documents the degree to which increased use of alternative funding has altered capacity to count the physician workforce in clinical practice in British Columbia. While the existence, structure and composition of service and payment files varies across jurisdictions in Canada, the conceptual approach to the design of the anonymized registry and, to some extent, the methods used to create it should be useful to others to (a) assess the degree to which the completeness and accuracy of FFS payment files have (or have not) eroded with increases in alternative funding and the growth in databases used to describe physician services and (b) design a registry for use in this evolving information environment.

What Did We Do?

In order to develop an anonymized registry of physicians delivering clinical services in 1996/97, 2000/01 and 2004/05, we used a two-stage process to identify doctors in British Columbia who appeared in one of a number of service or payment files, and then calculated each physician's income by source of payment. The purpose of both stages was solely to create a registry of doctors deemed to be in clinical practice. A third stage was undertaken to determine the relative volume of physician services delivered to local health areas through block funding (i.e., annual budgets negotiated for a group of physicians). The purpose of this stage was to ensure that the registry included this information in order to enable analysts to calculate measurement error in physician-level estimates of income and clinical activity. This project was approved by the Ministry of Health Services, the College of Physicians and Surgeons of British Columbia (CPSBC) and an ethics committee at the University of British Columbia. More details regarding methods are described elsewhere (Watson et al. 2006).

To differentiate the types of electronic data files, we refer to registration files, which include information about people but not services or payments; services files, which record services delivered or received; and payment files, which record remuneration to individual physicians or provider organizations. The anonymized registry included registration files from CPSBC; Medical Services Plan Practitioner Files, which include physicians ever eligible to bill the Ministry of Health Services; payment files from the Medical Services Plan, reflecting FFS claims for physician services delivered to BC and out-of-province residents; service files, including those from the Hospital Discharge Abstract Database and from primary healthcare organizations that receive capitation funding; and payment files from the alternative payment program in British Columbia (APP-BC), which include data that can be attributed to specific physicians (e.g., salary, sessional payments, rural incentives) or organizations (i.e., block funding). (See Figure 1.)

 

Click to Enlarge

 

Stage 1

The foundation of the registry is a computer algorithm that was used to scan the CPSBC registration file and the Medical Services Plan Practitioner File to create a registry database that would include a parsimonious set of information (e.g., sex, date of birth, unique identifier, practice location) regarding physicians who met all the following criteria:

  • eligible to bill the Medical Services Plan;
  • registered with CPSBC; and
  • resided in the province, as evidenced by a BC postal code, during a select fiscal period.

Stage 2

To identify physicians who delivered clinical services and to calculate the amount of remuneration each physician received through FFS and alternative payments attributed to specific physicians, we next created a computer algorithm to scan service files and payment files, as illustrated in Figure 1. The result is the identification of physicians who delivered clinical services, and for each doctor the total amount of FFS and alternative funding received over the period. Physicians' remuneration from organizations that receive block funding is not included, since it is not possible to determine which physicians receive funds from this source; the resultant measurement error regarding estimates of each physician's remuneration is addressed in Stage 3. Using any one file or subset of files listed in Figure 1 to count the number of physicians and measure their clinical activity would underestimate the population of doctors that deliver clinical services in British Columbia.

Stage 3

The third stage of registry development is intended to create an estimate of measurement error, for each physician, regarding calculations of his or her income or clinical activity. This measurement error relates to the extent to which physicians' services delivered in their local health areas receive block funding – for example, to organizations that are responsible for delivering cancer care, diabetes care or mental health services through global budgets – where it is not possible to identify which physicians have been remunerated with those funds.

This stage required the identification, for each local health area, of the proportion of total physician services attributable to block funding. This information was then included in the registry by creating a variable for each physician, to be used by analysts to identify doctors for whom payment data might be considered incomplete or biased. For example, a study regarding workloads among family physicians or the average number of patients seen by each might exclude doctors who reside in local health areas where 30% or more of primary care physician services are financed through block funding. In those regions, each family physician's clinical activity data derived from methods outlined in Stage 2 might underestimate the doctor's work, as it is not possible to determine which physician delivered services remunerated through block funding. Thus, all family physicians in the local health area would need to be excluded from the study to reduce estimation error. In essence, physicians should be excluded from analyses when they reside in areas where physician-level data are so incomplete that analyses based on their data would be considered biased.

What Did We Learn?

The number of physicians who delivered clinical services in British Columbia and who were included in the anonymized registry was 7,534, 7,822 and 8,558 in 1996/97, 2000/01 and 2004/05, respectively. Among the 8,558 physicians in clinical practice in 2004, a full 97% (8,268 physicians) could be identified using MSP FFS payment file (Table 1); the remaining 3% (290 physicians) could be identified using other payment or service files (Table 2).

 

Table 1. Count of physicians who received Medical Service Plan fee-for-service payments, by database, 2004/05

Note: MSP = Medical Services Plan Fee-for-Services payment database for provincial residents; Hosp = Hospital Discharge Abstract Database; APP-BC = Alternative payment program data for physicians in British Columbia; PHCO = Primary healthcare organizations; PTO = Medical Services Plan Fee-for-Services payment database for out-of-province residents.

 

Table 2. Count of physicians who do not receive Medical Service Plan fee-for-service payments, by database, 2004/05

Note: MSP = Medical Services Plan Fee-for-Services payment database for provincial residents; Hosp = Hospital Discharge Abstract Database; APP-BC = Alternative payment program data for physicians in British Columbia; PHCO = Primary healthcare organizations; PTO = Medical Services Plan Fee-for-Services payment database for out-of-province residents.

The exclusive use of FFS payment data significantly underestimates the number of physicians in clinical practice. Among the 3% or 290 physicians in clinical practice who could not be identified in the FFS payment files in 2004/05 (Table 2):

  • 69% could be identified in service files from hospitals (N=199, of whom 25 physicians were also identified via the alternative payment program data); and
  • 39% could be identified in APP-BC payment files (N=114, of whom 25 physicians were also identified via the Hospital Discharge Abstract Database).

In 1996/97, 2000/01 and 2004/05, a similar number of physicians in clinical practice (N=340, 326, 290) were unaccounted for using FFS payment files, and a similar proportion of them (65%, 67%, 69%) could be identified using the Hospital Discharge Abstract Database. What is unknown, however, is the number of physicians that are solely funded through block or capitation funding for which their services are not recorded in service files such as the Hospital Discharge Abstract Database or the PHC organization databases.

Tables 1 and 2 can be used in tandem to calculate the percentage of the physician workforce that could be captured with various combinations of payment and service files. For example, the creation of an anonymized registry through reliance on 2004/05 FFS and APP-BC payments would include 8,382 physicians (97.9% of physicians in clinical practice). A registry based on FFS and hospital files would include 8,467 physicians (98.9% of practising physicians); and a registry based on FFS, APP-BC and hospital files would include all but two of BC's practising physicians.

Over the study period, the percentage of clinical payments attributable to alternative funding of all types in British Columbia increased from 8% in 1996 to 10% in 2004. Across the 89 local health areas in the province in 2004, alternative funding of all types accounted for 1 to 100% of total physician expenditures. Alternative funding was also not confined to rural and remote areas that have few physicians. For example, these methods of funding represented 22% and 27% of total physician expenditures in the local health areas of Vancouver Midtown and Vancouver Downtown in 2004.

Across BC local health areas in 2004, block funding accounted for 0 to 100% of total physician expenditures. Block funding was not confined solely to remote areas that have few physicians. For example, this block funding represented 69% of total physician expenditures in the local health area of Kootenay Lake in 2004. Given the absence of physician and patient identifiers in the database designed to track block funding for physician services in British Columbia, neither supply- nor patient-based analyses would be possible in this jurisdiction.

What Are the Implications?

Between 1996/97 and 2004/05, there was an increase in the proportion of alternative funding to physicians in British Columbia and elsewhere in Canada. New databases designed to track alternative funding have tended to result in loss of physician-, service- and patient-specific identifiers that were historically available in FFS payment files. While much has occurred in British Columbia and other jurisdictions in Canada since this project was undertaken, in response to growing awareness that lack of uniformity in data structures results in loss of analytic capacity, important implications from this work still hold true as efforts continue to redesign old information systems and design new ones.

A complete registry of physicians in clinical practice now requires FFS and hospital data but is likely to underestimate community-based practice

In 2004/05, sole use of FFS payment files would have resulted in underestimates of the supply of physicians. At the moment, the problem is not big – at least in British Columbia. In 2004/05, 97% of the population of physicians in clinical practice could be identified using FFS payment files despite the fact that FFS accounted for 90% of total expenditures on physician services. Of the physicians who could not be identified with FFS payment files, 69% could be identified through the use of the Hospital Discharge Abstract Database. Similar findings were evident in 1996/97 and 2000/01. Thus, combined use of FFS and hospital files now offers an efficient means of improving estimates of physician supply in British Columbia.

Insofar as some community-based physicians are remunerated solely through block or capitation funding and receive no FFS income, the only databases useful for identifying them as active in clinical practice are service files. In British Columbia, the only service file available for community-based physicians as of 2004 was for those in the PHC organization program. This database includes services received by patients, and reporting relies on a claims system. The benefits of this arrangement include continued ability to track physicians who deliver clinical care, as well as collection of diagnostic and service information about the population served. This data strategy demonstrates the feasibility of collecting data about services from capitation-funded organizations through FFS infrastructure.

Community-based physicians who are remunerated solely through block funding, received no FFS and delivered care outside PHC organizations would not be identified using the methods outlined here. Importantly, following release of a review of the APP-BC in 2003, health authorities and agencies were required to submit service files similar to those used in the PHC organization program (Office of the Auditor General of British Columbia 2003, 2006).

The magnitude of underestimation in physician supply counts through methods that rely solely on traditional FFS payment data will become greater over time, insofar as (a) a ministry increasingly uses block payments to purchase services and does not include these data in payment files or does not simultaneously require providers to track activity in a service file, and (b) physicians increasingly seek remuneration solely from service organizations that receive block or capitation payments. We know that at least the former situation is true. At the time of writing, we did not have any data from alternative funding for the Medical On-Call Availability Program or the Rural Practice Programs. The Canadian Institute for Health Information (2006) reports that these programs accounted for $8 million, $31 million and $163 million in expenditures in 1996/97, 2000/01 and 2002/03, respectively.

Support for high-quality analyses now requires information on the relative volume of services each physician delivers through FFS and alternative funding

Although there is little erosion in capacity to count doctors in British Columbia, given the growth in use of new databases to support increases in alternative payments, the fact that alternative funding has increased indicates that the proportion of physicians' income attributable to these sources is growing. As alternative payments increase, fewer physicians will be eligible for inclusion in studies that rely on information available solely in FFS payment files. In British Columbia, there are a number of local health areas in rural, remote and metropolitan areas for which FFS represents a small proportion of total expenditures on physician services. Importantly, the anonymous physician registry can be used to identify physicians to be included or excluded in studies on the basis of the type of data available for each of them or the magnitude of estimation error in areas that are block funded for which no shadow claims or service files exist.

Alternative funding payment files vary in the degree to which they can be used in tandem with FFS payment files to measure constructs important to planning, evaluation and policy. Alternative payment databases that include physician-specific information, such as salary and sessional payment data, tend to include the size of remuneration but no other service- or patient-specific information, including the diagnostic condition(s) responsible for the visit. These data can be combined with FFS payment data for the purposes of calculating workload metrics such as full-time equivalents, but cannot be combined with FFS payment files to measure other volume metrics, such as visits per annum.

Significantly, the physician registry described can be used to create valid study populations and to support valid measurement of clinical activity. For example, a study to count the number of family physicians and full-time equivalents in clinical practice would rely on the registry to identify all doctors, irrespective of their source of funding. Without the registry, an analyst who relied solely on FFS payment files would underestimate the supply of physicians in a jurisdiction that used alternative funding. With the registry, physicians who receive all their income from block or capitation funding, for example, would be identified through the inclusion of service files. Analyses of the workloads of family physicians would rely on the registry to identify sources of income for each physician, and then pool data from appropriate payment files to calculate each physician's full-time equivalent. As another example, a study regarding the average number of patients seen by family physicians would rely on the registry to identify and exclude from analyses the doctors who receive 10% or more of their income from alternative funding when data associated with this method of payment do not include patient-level information.

Population-based analyses require estimation of measurement error due to missing data

Alternative payment databases designed to support block funding tend to include, at least in British Columbia, data regarding the size of payment, the type of service (e.g., cancer care, diabetes care, mental health services) and the local health area in which the funded organization was located. No other provider- or patient-specific data are included. These payments, therefore, cannot be attributed to physicians or patients in order to support population-based analyses of supply or utilization. But the physician registry can be used to estimate measurement error in analyses based on data from physicians who reside in the jurisdiction.

The magnitude of estimation error for a geographic population of physicians or cohort of specialists or generalists will be directly related to the degree to which the proportion of total physician expenditures in a jurisdiction or to a specialist/generalist group is derived from block funding. The size of estimation error is highest when a high proportion of total physician expenditures in a jurisdiction or a specialist/generalist group are funded through block payments. Furthermore, the size of estimation error is highest when there are no service files to accompany block funding. This type of estimation error will vary by jurisdictions across Canada, depending on the degree to which block payments are used to fund physician services and the degree to which funders require shadow claims or use of service files to track clinical activity.

Failure to capture patient-specific data in alternative funding payment files introduces bias in population-based analyses of BC residents who rely on FFS payment data. The magnitude of estimation error for a geographic population or cohort of BC residents will be directly related to the likelihood that they received services from alternative-funded physicians (e.g., salaried physicians) or organizations (i.e., block or capitation payments). For example, the accuracy of estimates of visit rates to paediatricians among a geographic population may be skewed if paediatric service organizations in their community receive block funding, or if the vast majority of paediatricians are funded through alternative payments. The anonymous physician registry cannot help estimate bias due to the former, but it can help with the latter.

Conclusion

This paper outlines innovative methods used to aggregate a parsimonious set of data from all available sources and create an anonymized physician registry at a time marked by growth in alternative funding, an increase in the number of databases to track these funds and a decline in the completeness of FFS payment files in Canada. This type of registry offers the most valid source of information to support physician workforce research and analyses. It can be used to count physicians in clinical practice, create study cohorts to conduct policy-relevant analyses and estimate bias in provider-based analyses.

Because the responsibility for alternative funding programs tends to be spread across different units within health ministries and, in most jurisdictions, each administrative unit is responsible for setting its own information requirements, databases designed to support these payments vary within and among ministries of health; some include information about providers and patients and others do not (CIHI 2006). But new databases designed to track physicians' services funded through alternative funding programs typically include less information than is available in FFS payment data in Canada.

In the future, information systems to track these services should be uniformly structured within and across jurisdictions and retain or increase the array of data needed to support the creation of evidence demanded by healthcare policy, management and practice communities.


Méthodes pour la création et le maintien d'un registre de médecins dans le contexte évolutif de l'information

Résumé

Ce que nous avons fait : Étant donné que les modes de financement pour les services médicaux se diversifient au Canada, la pertinence des données sur la rémunération à l'acte (RAA) comme source d'information démographique perd en importance. Cet article décrit les méthodes employées pour créer et valider un registre anonyme de l'effectif des médecins, afin de permettre des analyses politiques dans un contexte où la rémunération des médecins prend des formes de plus en plus diverses.

Ce que nous avons appris : Parmi les 8558 médecins qui travaillaient dans des cliniques en Colombie-Britannique, en 2004, 97% pouvaient être retracés grâce aux données sur les RAA. En 1996, 2000 et 2004, un nombre semblable de médecins provenant de cliniques (N = 340, 326, 290) n'ont pas présenté de réclamations pour la RAA, et une proportion stable de ces médecins (65%, 67%, 69%) pouvait être retracée au moyen des données provenant du registre des sorties des hôpitaux. Dans la province, en 2004, les circonscriptions sanitaires variaient en proportion de la somme des dépenses pour les médecins attribuables à la RAA (de 0 à 100%).

Répercussions : La RAA et les dossiers des hôpitaux peuvent être utilisés conjointement pour identifier les médecins en pratique clinique afin de créer des registres complets permettant des analyses démographiques de la main-d'œuvre. Les dossiers sur la RAA et sur les autres modes de financement peuvent être utilisés conjointement pour calculer les activités cliniques et le revenu de chaque médecin, et pour identifier les médecins pour lesquels il existe une forte marge d'erreur de mesure, puisqu'ils résident dans des circonscriptions sanitaires où une grande proportion des activités cliniques n'est pas consignée.

Recommandations: Les systèmes conçus pour retracer les divers modes de financement devraient être structurés de façon uniforme au sein des autorités et entre elles afin de maintenir ou d'accroître la disponibilité des données démographiques cliniques qui permettent d'effectuer des analyses secondaires. Ce n'est qu'alors que les données pourront être utilisées afin d'appuyer les politiques, la gestion et la prise de décisions cliniques au Canada.

About the Author(s)

Diane E. Watson, MBA, PHD, Centre for Health Services and Policy Research, University of British Columbia, Vancouver, BC

Sandra Peterson, MSC, Research Analyst, Centre for Health Services and Policy Research, University of British Columbia, Vancouver, BC

Ella Young, MHA, Research Coordinator, Centre for Health Services and Policy Research, University of British Columbia, Vancouver, BC

Bogdan Bogdanovic, BCOMM, BA, Programmer/Analyst, Manitoba Centre for Health Policy, University of Manitoba, Winnipeg, MB

Correspondence may be directed to: Dr. Diane E. Watson, Chief Executive, Bureau of Health Information, Tower A, Level 16, Zenith Centre, 821 Pacific Highway, Chatswood, New South Wales, 2067, Australia; E-mail: Diane.Watson@bhi.health.nsw.gov.au.

Acknowledgment

The British Columbia Ministry of Health Services provided funding for this project as well as some of the funding for the special issue in which this article is published. We would like to acknowledge the College of Physicians and Surgeons of British Columbia (CPSBC) for access to physician registration data. Conclusions are those of the authors and reflect no official endorsement by the Ministry or CPSBC.

References

British Columbia Ministry of Health Services, Information and Analysis Branch. 2000. MSP Information Resource Manual Fee-for-Service Payment Statistics 1999/2000. Victoria, BC: Medical Services Plan.

British Columbia Ministry of Health Services, Information and Analysis Branch. 2002. MSP Information Resource Manual Fee-for-Service Payment Statistics 2001/2002. Victoria, BC: Medical Services Plan.

Canadian Institute for Health Information (CIHI). 2006. The Status of Alternative Payment Programs for Physicians in Canada, 2003-2004 and Preliminary Information for 2004-2005. Ottawa: Author.

Canadian Institutes of Health Research (CIHR) and Canadian Health Services Research Foundation (CHSRF). 2008 (June 3). Listening for Direction III: A National Consultation on Health Services and Policy Issues. Retrieved January 6, 2009. <http://www.cihr.ca/e/20461.html>.

Office of the Auditor General of British Columbia. 2003. Report 4: Alternative Payments to Physicians: A Program in Need of Change. Victoria, BC: Author.

Office of the Auditor General of British Columbia. 2006. Report 7: Follow-up of 20003/04 Report 4: Alternative Payments to Physicians: A Program in Need of Change. Victoria, BC: Author.

Watson, D.E., S. Peterson, E. Young and B. Bogdanovic. 2006. Developing an Information System to Identify and Describe Physicians in Clinical Practice in British Columbia: 1996/07-2004/05. Vancouver: Centre for Health Services and Policy Research.

Comments

Be the first to comment on this!

Related Articles

Note: Please enter a display name. Your email address will not be publically displayed