The operational definition of first-contact accessibility is "the ease with which a person can obtain needed care (including advice and support) from the practitioner of choice within a time frame appropriate to the urgency of the problem"; accommodation is "the way healthcare resources are organized to accommodate a wide range of patients' abilities to contact healthcare providers and reach healthcare services, that is to say telephone services, flexible appointment systems, hours of operation, and walk-in periods."

Objective: To compare how well accessibility is measured in validated subscales that evaluate primary healthcare from the patient's perspective.

Method: 645 adults with at least one healthcare contact in the previous 12 months responded to six instruments that evaluate primary healthcare with four subscales that measure accessibility: the Primary Care Assessment Survey (PCAS), the Primary Care Assessment Tool – Short Form (PCAT-S, two subscales) and the first version of the EUROPEP (EUROPEP-I). Scores were normalized to a 0-to-10 scale for descriptive comparison. Exploratory and confirmatory (structural equation modelling) factor analysis examined fit to operational definition, and item response theory analysis examined item performance on common constructs.

Results: The subscales demonstrate similar psychometric measures to those reported by developers. The PCAT-S First-Contact Utilization subscale does not fit the accessibility construct. The remaining three subscales load reasonably onto a single factor, presumed to be accessibility, but the best-fitting model has two factors: "timeliness of obtaining needed care" (PCAT-S First-Contact Access, some EUROPEP-I items) and "how resources are organized to accommodate clients" (PCAS Organizational Access and most of EUROPEP-I organization of care). Items in the PCAS and PCAT-S subscales have good discriminability.

Conclusion: Only three of the four subscales measure accessibility; all are appropriate for use in Canada. The PCAT-S First-Contact Access subscale is the best measure for first-contact accessibility, and PCAS Organizational Accessibility has good metric properties and measures for accommodation.

Acessibility receives a lot of attention in research and policy debates. However, its definition and assessment pose significant challenges.

Background

Conceptualizing accessibility of healthcare services

Accessibility is a complex notion, as evidenced by the heterogeneity of definitions and conceptualizations in the literature and the almost interchangeable use of the terms "access," "accessibility" and "utilization of healthcare services." The Canadian Oxford Dictionary (1998) defines accessibility as the "condition of being readily approached." In this sense, accessibility is a characteristic of something that can readily be reached, entered or used.

Donabedian (1973) describes accessibility as characteristic of health systems that impede or promote service utilization. Thus, health services are accessible if their specific characteristics – geographic availability, organization, price, acceptability and so on – allow a broad range of persons to reach, enter and use them (Bashshur et al. 1971; Donabedian 1973; Penchansky and Thomas 1981). From this perspective, evaluation of accessibility is amenable to both objective and subjective assessment of the geographic and temporal availability of services, their organizational availability, their costs and their social and cultural acceptability (Lévesque 2006).

Evaluating the accessibility of primary healthcare services

In a consensus consultation of primary healthcare (PHC) experts across Canada to formulate operational definitions of PHC attributes to be evaluated (Haggerty et al. 2007), two distinct definitions of accessibility emerged. The first, labelled first-contact accessibility, is "The ease with which a person can obtain needed care (including advice and support) from the practitioner of choice within a time frame appropriate to the urgency of the problem." This is specific to PHC and is one of its essential functions. The second, accommodation, is applicable to all levels of healthcare: "The way healthcare resources are organized to accommodate a wide range of patients' abilities to contact healthcare providers and reach healthcare services (telephone services, flexible appointment systems, hours of operation and walk-in periods)" (Haggerty et al. 2007).

Various instruments have been developed to evaluate PHC accessibility from the user's perspective, but there is little comparative information about these to guide evaluators in their selection of tools. Our objective was to provide insight into how well validated subscales from different instruments measure accessibility. Specifically, we wanted to know how subscale scores from different instruments compare and whether the accessibility subscales measure a single construct, presumed to be accessibility. If analyses suggested more than one factor, we wanted to judge how those factors reflected our operational definitions of accessibility. Finally, we sought to examine how well individual items measured the dimensions that were common across instruments.

Method

The method of this series of studies has been described in detail elsewhere (Haggerty, Burge et al. 2011). Briefly, six validated instruments that evaluate PHC from the patient's perspective were administered to 645 healthcare users balanced approximately by English/French language, rural/urban location, high/low education, and poor, average or excellent overall PHC experience.

The analytic strategy, described in detail elsewhere (Santor et al. 2011) consisted of examining the distributional statistics and subscale correlations, followed by common factor analysis and confirmatory factor analysis (structural equation modelling) to identify dimensions common to the entire set of items. Finally, we examined the performance of individual items and response scales using parametric item response theory analysis against the original subscale and non-parametric analysis against the common factors that emerged across instruments.

Measure selection

Three of the instruments had accessibility subscales. The Primary Care Assessment Survey (PCAS) (Safran et al. 1998) six-item Organizational Accessibility subscale elicits ratings for obtaining services at the clinic using a six-point Likert response scale (1=very poor to 6=excellent). A parallel set of questions report average wait times, but these are not included in the validated subscale. The Primary Care Assessment Tool – Short Form (PCAT-S) (Shi et al. 2001) has two subscales: the three-item First-Contact Utilization (eliciting usual patterns of care-seeking), and the four-item First-Contact Access (eliciting the probability of being seen at the clinic under various scenarios with a four-point Likert scale from "1=definitely not" to "4=definitely"). However, during the attribute-mapping process, the PCAT-S First-Contact Utilization subscale was flagged as fitting poorly with the concept of accessibility, despite its label. The first version of the European general practice instrument, EUROPEP-I (Grol et al. 2000) has a seven-item Organization of Care subscale in which most items elicit ratings of accessibility using a five-point semantic differential response scale with "1=poor" and "5=excellent" as anchors of each extreme.

Results

Comparative descriptive statistics

The content of the four subscales and item distributors are summarized in Table 1. Only two subscales have ≥5% true missing values (PCAS on being able to talk by phone to the doctor, and EUROPEP-I on the helpfulness of staff). However, both PCAT-S subscales and the EUROPEP-I offer respondents a "don't know/not sure" or "not applicable" option, which is used by a significant proportion of respondents in PCAT-S First-Contact Access items and four EUROPEP-I items. Approximately 17% were "not sure" about arrangements for getting care or advice outside office hours on the PCAT-S, or being able to contact the doctor by phone (13%) on the EUROPEP-I.

Only the PCAT-S First-Contact Utilization showed strong ceiling effects, with the majority endorsing the most positive response. The parametric estimate of discriminability within the original subscale generally showed good capacity to discriminate between different levels of the subscale score. Exceptions were the convenience of location on PCAS Organization Accessibility and need for approved referrals to specialists on PCAT-S First-Contact Utilization.

Table 2 presents the subscale scores as item means normalized to a common 0-to-10 metric to permit comparison. The normalized means and standard deviations differ substantially from one scale to another. All except the PCAT-S First-Contact Access subscale are quite skewed towards positive values, with the median higher than the mean. The PCAT-S subscales had relatively low reliability.

Table 3 presents the Pearson correlations between the accessibility subscales. With the exception of the PCAT-S First-Contact Utilization subscale, the accessibility subscales correlate strongly with one another (.45 to .68), suggesting a common construct. In contrast, the PCAT-S First-Contact Utilization subscale correlates only modestly (.24 to .29) and correlates most highly with relational continuity subscales (.28 to .37). The EUROPEP-I Organization of Care subscale correlates as strongly with subscales of other dimensions as it does with accessibility, suggesting that it is measuring an overall experience of care rather than accessibility specifically. In contrast, the PCAS Organizational Access subscale and the PCAT-S First-Contact Access subscale have much lower correlations with other attributes of care, suggesting they are more specific for accessibility.

Do all items measure a single attribute?

We excluded from factor analysis all subjects who had at least one missing value on any item (listwise missing). This dramatically reduced our effective sample size from 645 to 340. Most exclusions (267/340) were for selecting the "don't know/not sure" or "not applicable" response options. Those excluded from the factor analyses were more likely than those included to be English-speaking, to report better health status and to have lower usual wait times for an appointment; they did not differ by their overall experience of care and other individual characteristics. Because this conservative approach can introduce bias, we repeated all the confirmatory factor analyses using maximum likelihood imputation of missing values (Jöreskog and Sörbom 1996) to examine the robustness of our conclusions. Running the models on a sample of 559 improved the fit statistics of confirmatory factor analysis models but did not alter any of the general conclusions.

We had little expectation that all the items would load on a single construct because we had defined two dimensions of accessibility and the PCAT-S First Utilization subscales seem to fit poorly. Nonetheless, constraining common factor analysis to one factor found loadings >.30 for all but two items. The same model with structural equation modelling generated statistics suggesting only moderate fit, with a root mean squared error of approximation (RMSEA) of p=.11, considerably higher than the .05 standard for good fit. Fit statistics improved dramatically when we removed the PCAT-S First-Contact Utilization fit, with the RMSEA=.09, a normed fit index (NFI) of .98 (well above the .90 standard) and considerably lower chi-squared and model conditional Akaike information criteria (CAIC) values (better fit with lower values). Figure 1 presents a second-order model in which items are grouped within their original subscale as one construct (first-order latent variable), which in turn emerges from a single underlying construct, presumed to be accessibility (second-order latent variable). Compared to the one-factor model, grouping items by parent subscale improves fit and shows that PCAT-S First Utilization loads much less strongly on the construct presumed to be accessibility.

Based on this and our a priori concerns, we decided that the PCAT-S First-Contact Utilization subscale, despite its label, does not fit the construct of accessibility. Subsequent analyses indicated a best fit with comprehensiveness of care (Haggerty, Beaulieu et al. 2011). It was removed from further analyses on accessibility.

How do underlying factors fit with operational definitions?

Exploratory factor analysis suggested a two-factor model. Using our operational definitions as a guide, we judged that the first factor (eigenvalue=7.59) captured "obtaining needed care … in a time frame appropriate to the urgency of the problem," or timeliness, and the second (eigenvalue=1.19) touched on "how resources are organized to accommodate clients," or accommodation.

Using the operational definitions and the exploratory analysis as a guide, we grouped items on timeliness and accommodation for confirmatory factor analysis, shown in Figure 2. Goodness of fit improved slightly with respect to the unidimensional model (RMSEA=.084, NFI=.98). The correlation between the dimensions of timeliness and accommodation is .95. We tried other item groupings based on our judgment, but all other configurations resulted in poorer fit statistics. The model shows that the PCAS Organizational Access subscale relates to the sub-dimension of accommodation, whereas the PCAT-S First-Contact Access subscale relates to timeliness. The EUROPEP-I Organization of Care subscale mostly measures timeliness, but one item, wait time in the waiting room, loads highly on accommodation. Figure 2 shows that some items do not have high loadings and have a high proportion of residual error (shown to the right of each item). These items relate poorly to the construct, either because they are not discriminatory or because they relate better to another construct that is not part of the latent variable.

A number of items were identified a priori as potentially poor indicators of accessibility. For instance, the EUROPEP-I Organization of Care subscale item, "How would you rate your general practitioner's care in preparing you for what to expect from specialist or hospital care?" does not appear to measure accessibility. However, this item correlates well with other items in the subscale (item–total correlation=.55), and removing it does not improve the model's goodness of fit. Since we had decided to respect as much as possible the developers' factor solutions, we retained this item as part of the subscale.

Individual item performance

Non-parametric item response theory graphs were modelled on dimensions of timeliness and accommodation and provide further insight into item performance. For accommodation, we found that all but one item (PCAS convenient location) in the PCAS Organizational Access subscale demonstrate excellent performance. The probability of endorsing each response option is highest in a relatively narrow and unique zone of accommodation, and is clearly ordinal, reflecting the assigned value for each option. The item score varies linearly and strongly with accommodation, demonstrating excellent discriminability. The single EUROPEP-I item that measures accommodation (EU_oa6, wait in waiting room), shows good discriminability, but the extreme response options ("1=poor," "5=excellent") are overwhelmingly endorsed, suggesting that the assigned values for middle options may not be appropriate.

For timeliness, the first two items in the PCAT-S perform relatively well (probability of being seen same day when provider is open, and getting advice over the phone). The value assigned to each response option is mostly appropriate, except the option "probably," which appears to be non-specific, as it is endorsed across the entire range of timeliness. The item score correlates highly with overall timeliness, indicating good discriminability. In contrast, the two items addressing more rare scenarios (when provider is closed) have modest discriminability, and the response options and assigned values perform less well. Only the options "1=definitely not" and "4=definitely" are endorsed with any frequency, raising the question of the appropriateness of a four-point response scale. The probability of choosing the "don't know" response option (over 15% of respondents) mimics the probability of endorsing options "2=probably not" or "3=probably," depending on the item.

All but one EUROPEP-I item performed poorly on timeliness. Only the item "getting quick services for urgent problems" demonstrated good discriminability despite problems with middle response options. The remaining items demonstrate poor discriminability, and the behaviour of the middle response options indicate that differential weights may not be warranted. Based on the content of some items (preparation to see specialist, helpfulness of staff, phone contact with the clinic, and phone contact with the general practitioner), we tested them on accommodation rather than timeliness, but item performance did not improve.

Discussion

Capacity to measure accessibility

Of the four subscales used in the concurrent validation study, three seem to evaluate clearly the attribute of accessibility, whereas PCAT-S First-Contact Utilization appears to be measuring comprehensiveness of care or possibly relational continuity (Burge, Haggerty et al. 2011). This finding implies that evaluators interested in evaluating accessibility could use any one of these three subscales. However, the PCAT-S First-Contact Access subscale measures timeliness, and the PCAS Organizational Access subscale principally measures accommodation. The EUROPEP-I Organization of Care subscale appears to capture both dimensions, but despite high internal consistency, item performance on the constructs of timeliness or accommodation are problematic. The high correlation between the EUROPEP-I Accessibility subscale and all the other scales in the evaluation study further suggests it measures a generic experience of care rather than accessibility specifically. This is not surprising, because the scale developers did not specifically intend to evaluate accessibility, but rather organization of care based on patient priorities (Grol et al. 1999). It has adequate discriminability for its own construct. Recent revisions to the instrument are minor, but may improve metric properties (Wensing 2006).

The PCAT-S First-Contact Access is the best subscale for measuring first-contact accessibility as the dimension of accessibility that is specific to PHC. The item's content well reflects policy initiatives to enhance 24/7 access to PHC, and the items are ordered by difficulty, suggesting that a summary score is informative and meaningful. However, despite good discriminability, its performance could be improved by changing the response options. First, the option "probably" is non-specific, shown both on item response analysis and discussions with respondents (Haggerty, Lawson et al. 2011). Most importantly, the "don't know" option should not be offered as it creates missing values on analysis. The item response theory analysis suggests that the developers' recommendation of imputing "not sure" as "probably not" is not appropriate. The two items referring to more rare access experiences – when the office is closed or at night – would function better with a yes/no response format but can be used to identify those with excellent access.

The PCAS Organizational Accessibility subscale demonstrates excellent performance for measuring accommodation. The good discriminability and reliability might improve further if the item on convenience of location were removed. Since accommodation is not specific to PHC, it is likely equally relevant to general healthcare accessibility. Two questionnaires that were largely derived from the PCAS (General Practice Assessment Questionnaire – GPAQ and General Practice Assessment Survey – GPAS) include a question on timely care: "If you need to see a doctor urgently, can you normally get seen on the same day?" and may be a good option for including an indicator of first-contact accessibility as well as accommodation.

Aspects not covered in the studied instruments

The subscales we studied did not address the element of "obtaining care from the provider of choice" in our operational definition, though it could be argued this is captured by the PCAT-S First-Contact Utilization subscale that we excluded. Although studies show that patients prioritize timeliness over affiliation when faced with an acute illness (Love and Mainous 1999), on the whole, patients strongly prefer to consult their own physician for most care, as is shown on measures of relational continuity elsewhere in this special issue (Burge et al. 2011). The provider of choice – and therefore the PCAT-S First-Contact Utilization subscale – may be the link between accessibility and relational continuity, or may be particularly relevant to accessibility for non-urgent problems.

None of the identified questionnaires assess geographic accessibility, which is often the first aspect of accessibility considered (Frenk 1992). The PCAS item on "the convenience of location" is problematic owing to its skewed distribution and poor discriminability. Although long distances from care may affect a minority of Canadians, it would be important to develop a sensitive measure of geographic accessibility, given that Canadian geography makes it a major consideration in healthcare planning. We developed a measure of ease of reaching care and local availability of services that may meet this need (Haggerty, Lévesque et al. 2009).

We did not assess the subscales that evaluate economic accessibility, as these were not considered relevant for Canada because they refer to direct costs of services. However, indirect costs, such as transportation costs and pay lost when receiving medical care during working hours, can result in forgone care. We have developed a measure of economic accessibility that addresses these issues and performs very well in the Quebec context (Haggerty, Lévesque et al. 2009).

Study limits

Limiting the study to those having visited a regular provider in the previous 12 months may have selected subjects with good accessibility. However, including users and non-users in a study sample should increase the variance and improve, not compromise, measure performance. Eliminating subjects with missing values not only reduced statistical power, but may have biased the final sample, even though our sensitivity analysis using imputation of missing values did not alter our overall conclusions. The item response analysis shows that missing values tend to occur among respondents with more negative experiences of accessibility. This finding would underestimate the reported reliability and attenuate the factor analysis results, but is not expected to radically change overall differences among instruments. Finally, this study did not have the benefit of an objective assessment of aspects of accessibility at the usual source of care that could have enabled us to assess the correlation of different scales with actual measures of availability and barriers to care.

Conclusion

Despite the study's limitations, the results indicate that the PCAS Organizational Access subscale is an excellent choice for measuring accommodation. The PCAT-S First-Contact Access subscale is the best for timely first-contact care and is, therefore, highly relevant for accessibility reforms in Canada, but it could be psychometrically improved. The EUROPEP-I Accessibility subscale probably assesses a more general experience of care, including accessibility dimensions.

L'accessibilité du point de vue du patient : comparaison entre instruments d'évaluation des soins de santé primaires

Résumé

La définition opérationnelle de l'accessibilité de premier contact est « la facilité avec laquelle une personne peut obtenir les services requis (y compris des conseils et du soutien) du clinicien de son choix dans un délai approprié à l'urgence du problème »; l'accommodation est « la façon dont les ressources en santé sont organisées afin de permettre à une large gamme de patients d'entrer en contact avec les fournisseurs de soins avec les services de santé, c'est-à -dire accès téléphonique, flexibilité pour la prise de rendez-vous, heures d'ouverture et périodes allouées aux visites sans rendez-vous. »

Objectif : Voir à quel point l'accessibilité est mesurée par les sous-échelles validées servant à évaluer les soins de santé primaires du point de vue du patient.

Méthode : 645 adultes ayant eu au moins un contact avec les services de santé au cours des 12 mois antérieurs ont répondu à six instruments servant à évaluer les soins de santé primaires, incluant quatre sous-échelles qui mesurent l'accessibilité : Primary Care Assessment Survey (PCAS), Primary Care Assessment Tool – version courte (PCAT-S, deux sous-echelles) et la premiere version de l'EUROPEP (EUROPEP-I). Les résultats ont été normalisés selon une échelle de 0 à 10 pour permettre des comparaisons descriptives. Les analyses factorielles exploratoires et confirmatoires (modélisation par équation structurelle) ont permis d'examiner l'adéquation à la définition opérationnelle, et l'analyse de réponse par item a permis d'en examiner la performance en fonction des construits communs.

Résultats : Les mesures psychométriques des sous-échelles sont similaires à celles indiquées par les concepteurs. La sous-échelle « utilisation de premier contact » du PCAT-S ne concorde pas avec le construit de l'accessibilité. Les trois autres sous-échelles correspondent raisonnablement à un facteur unique, qui serait probablement l'accessibilité, mais le modèle qui s'ajuste le mieux comprend deux facteurs : « rapidité d'obtention des soins nécessaires » (« accessibilité de premier contact » du PCAT-S, certains items de l'EUROPEP-I) et « comment sont organisées les ressources pour accommoder les patients » (« accès organisationnel » du PCAS et la plupart des items « organisation de soins » de l'EUROPEP-I). Les items des sous-échelles du PCAS et du PCAT-S présentent une bonne discriminabilité.

Conclusion : Seules trois des quatre sous-échelles mesurent l'accessibilité; toutes sont appropriées pour leur usage au Canada. La sous-échelle « accès de premier contact » du PCAT-S est la meilleure mesure pour l'accessibilité de premier contact et la sous-échelle « accessibilité organisationnelle » du PCAS possède de bonnes mesures et propriétés métriques pour l'accommodation.

Comments

Be the first to comment on this!