The operational definition of relational continuity is "a therapeutic relationship between a patient and one or more providers that spans various healthcare events and results in accumulated knowledge of the patient and care consistent with the patient's needs."

Objective: To examine how well relational continuity is measured in validated instruments that evaluate primary healthcare from the patient's perspective.

Method: 645 adults with at least one healthcare contact in the previous 12 months responded to six instruments that evaluate primary healthcare. Five subscales map to relational continuity: the Primary Care Assessment Survey (PCAS, two subscales), the Primary Care Assessment Tool – Short Form (PCAT-S) and the Components of Primary Care Index (CPCI, two subscales). Scores were normalized for descriptive comparison. Exploratory and confirmatory (structural equation modelling) factor analysis examined fit to operational definition, and item response theory analysis examined item performance on common constructs.

Results: All subscales load reasonably well on a single factor, presumed to be relational continuity, but the best model has two underlying factors corresponding to (1) accumulated knowledge of the patient and (2) relationship that spans healthcare events. Some items were problematic even in the best model. The PCAS Contextual Knowledge subscale discriminates best between different levels of accumulated knowledge, but this dimension is also captured well by the CPCI Accumulated Knowledge subscale and most items in the PCAT-S Ongoing Care subscale. For relationship-spanning events, the items' content captures concentration of care in one doctor; this is captured best by the CPCI Preference for Regular Provider subscale and, to a lesser extent, by the PCAS Visit-Based Continuity subscale and one relevant item in the PCAT-S Ongoing Care subscale. But this dimension correlates only modestly with percentage of reported visits to the personal doctor. The items function as yes/no rather than ordinal options, and are especially informative for poor concentration of care.

Conclusion: These subscales perform well for key elements of relational continuity, but do not capture consistency of care. They are more informative for poor relational continuity.

Continuity of care is central to the delivery of primary healthcare (PHC). Indeed, in a cross-disciplinary review of the literature on continuity of care, the single largest source of research reports came from PHC (Reid et al. 2002).

Background

Conceptualizing relational continuity

The concept has evolved within some health disciplines but has remained stable in family medicine as referring to a provider–patient relationship over time and across different health events. This form of continuity, referred to as "relational continuity," is distinct from other forms that connect services received from different providers (Haggerty et al. 2003). PHC providers see relational continuity as a distinguishing characteristic of their work and a core value (McWhinney 1998). It is also prioritized by patients (Mainous et al. 2001; Baker et al. 2005; Turner et al. 2007; Cheraghi-Sohi et al. 2008). We would argue that in no other segment of the healthcare system is relational continuity more important.

Our consensus consultation of PHC experts across Canada unanimously identified relational continuity as an essential function of PHC regardless of organizational model (Lévesque et al. 2011). It is defined operationally as "a therapeutic relationship between a patient and one or more providers that spans various health care events and results in accumulated knowledge of the patient and care consistent with the patient's needs" (Haggerty et al. 2007).

Evaluating relational continuity in primary healthcare

Within PHC research, relational continuity has been inferred most commonly from the degree to which patient care is concentrated in a single physician (Steinwachs 1979; Rogers and Curtis 1980). Research evidence suggests that seeing the same provider over time for multiple health events is associated with positive outcomes, including better doctor–patient communication (Bertakis and Callahan 1992; Berry et al. 2008), greater uptake of preventive and health-promoting strategies (Ettner 1996, 1999; O'Malley and Forrest 1996; O'Malley 1997; Flocke et al. 1998), reduced diagnostic testing (Weiss and Blustein 1996), reduced emergency department utilization (Burge et al. 2003) and reduced emergency hospital admissions (Wasson et al. 1984).

Most often, relational continuity is conceived as a relationship between a patient and a single doctor. However, new models of PHC evolving in Canada and internationally are moving to interprofessional, team-based care that may be disruptive to relational continuity (Smith 1995; Rodriguez et al. 2007), making it important to measure relational continuity in the evaluation of reforms.

The objective of our study was to compare validated instruments thought to be most pertinent to the Canadian context, and in this paper we focus on how well subscales from different instruments measure the constructs of relational continuity. Specifically, we examined the equivalence of the scores of different instruments' subscales and whether all the relational continuity subscales measure a single construct or factor. If analysis suggested more than one factor, we aimed to determine how these corresponded to the operational definition. Finally, we examined how well individual items perform in measuring the common construct of relational continuity that emerged across instruments.

Method

The conduct of the study (Haggerty 2011) and the analytic approach (Santor et al. 2011) have been described in detail elsewhere. Briefly, six validated instruments that evaluate PHC from the patient's perspective were administered to 645 healthcare users balanced by English/French language, rural/urban location, low/high level of education and poor/average/excellent overall PHC experience. The analysis consisted of examining the distributional statistics and subscale correlations, followed by common factor and confirmatory factor analysis (structural equation modelling) to identify dimensions common to the entire set of items. Finally, we examined the performance of individual items and response scales using item response theory analysis.

Measure description

Three instruments in our study have five subscales that mapped to our operational definition. The Primary Care Assessment Survey (PCAS) (Safran et al. 1998) has two. The two-item Visit-Based Continuity subscale elicits how often the "regular doctor" is consulted for routine and sickness care. The Contextual Knowledge subscale contains four items asking patients to rate, on a six-point Likert response scale (1=very poor to 6=excellent), the doctor's knowledge of different dimensions of the patient and a single item on how well their doctor would know their wishes if they were in a coma; this latter item's 1-to-10 response scale is collapsed into a 1-to-6 scoring.

The Primary Care Assessment Tool – Short Form, adult (PCAT-S) (Shi et al. 2001) has a four-item Ongoing Care subscale that elicits the probability, on a four-point Likert response scale (1=definitely not to 4=definitely), that the patient always consults the "primary care provider" and that asks about the provider's knowledge of the patient.

The Components of Primary Care Index (CPCI) (Flocke 1997) has two relational continuity subscales that each use a six-point semantic difference agreement response scale (poles of 1=strongly disagree and 6=strongly agree) on various statements about the "regular doctor." The eight-item Accumulated Knowledge scale assesses the doctor's knowledge of various dimensions of the patient, including one item on the persistence of the relationship. The five-item Preference for Regular Physician subscale assesses the extent to which care is concentrated, by choice, with the regular doctor.

Results

Comparative descriptive statistics

Table 1 summarizes the item content and behaviour of the five subscales. No items have more than 5% missing values. Most respondents select positive expressions of relational continuity, especially for items in all three instruments asking whether the same provider is consistently consulted; over 50% select the highest response option. The parametric estimates of the discriminability within the original subscale indicate that all but one item (PCAT-S, seeing same doctor or nurse) discriminate well between different levels of the subscale score (a>1.0). Less discriminating items are those that elicit care concentration. This finding likely reflects the different sub-dimensions revealed in factor analysis.

Table 2 presents the descriptive statistics for each subscale. The subscale scores are normalized to a 0-to-10 metric to permit comparison. The scores are skewed towards positive values, with medians higher than means, especially for the PCAS Visit-Based Continuity subscale and the CPCI Preference for Regular Physician. The normalized means for the scales differ substantially from one to another, but the standard deviations are relatively similar. The subscales have adequate reliability.

Table 3 presents the Pearson correlations between the relational continuity subscales. The PCAS Visit-Based Continuity subscale does not correlate well with other continuity subscales (r=.24, .26) and only modestly with the CPCI Preference for Regular Provider subscale (r=.26 to .54). The remaining continuity subscales correlate highly with one another, but also with subscales measuring interpersonal communication, respectfulness and trust. When correlated with the reported percentage of visits to the regular physician, we found no correlation with the PCAS Visit-Based Continuity subscale or the PCAT-S Ongoing Care subscale, and only weak correlations with the PCAS Contextual Knowledge (r=.09), CPCI Accumulated Knowledge (r=.12) and CPCI Preference for Regular Provider subscales (r=.08).

We had hypothesized a priori, based on item content, that the PCAS Trust and PCAT-S First-Contact Utilization subscales might relate to relational continuity. Despite high correlations with the PCAS Trust subscale, the items did not load with other relational continuity items in common factor analysis. The First-Contact Utilization subscale, which elicits the tendency to consult the primary care provider first, neither correlates highly nor loads with continuity subscales, except for one item: "If I am sick, I would always contact a doctor at this clinic first."

Do all items measure a single attribute?

Our effective sample size for factor analysis was reduced from 645 to 495 by excluding respondents with at least one missing value on any item (listwise missing). Respondents excluded were more likely to be older and to have a chronic health problem. However, because this conservative approach to dealing with missing values can introduce bias, we repeated all the confirmatory analyses using maximum likelihood imputation of missing values (Rubin 1987) to examine the robustness of our conclusions, but the larger sample did not change the direction or essence of these.

Most of the 21 items loaded reasonably well (>.40) onto a single factor using common factor analysis; exceptions were items related to concentration of care (PCAS Visit-Based Continuity, one item of the PCAT-S and two from the CPCI Preference for Regular Provider). Confirmatory factor analysis of a one-dimensional model indicates adequate model fit, with a root mean squared error of approximation (RMSEA) of p=.086 (slightly higher than the .05 level indicating good fit, Model 1) and a normed fit index (NFI) of .98, well above the .90 level indicating good fit.

How do underlying factors fit with operational definition?

The eigenvalues of exploratory factor analysis suggest a two-factor model. Using our operational definition as a guide, we judged that the first factor (eigenvalue=10.19) captures "accumulated knowledge of the patient," while the second (eigenvalue=1.66), "relationship ... spans various health events," focuses on concentration of care rather than on duration of relationship. Both are specific to the regular provider. No items or factors captured "experienced care being consistent with individual needs."

Some individual items had problematic loadings. For instance, the PCAT-S item, "ability to talk to a known provider," loads weakly on knowledge (.46), and the content appears to relate to access rather than to relational continuity. As a whole, the CPCI Preference for Regular Physician subscale fits within concentration of care, but the first two items do not load on either factor ("contact own provider when sick," loadings .21, .16, and "care improves with concentration of care," loadings .33, .27). These were associated with concentration of care for confirmatory factor analysis. One item in the CPCI Accumulated Knowledge subscale loads very modestly on knowledge (.48), but this finding is most likely due to its reverse wording, "doctor does not know history," rather than to poor conceptual fit.

With confirmatory factor analysis, the best-fitting model items are grouped in their original subscales, which in turn are associated with a single construct, presumed to be relational continuity, as illustrated in Figure 1. The figure shows that the PCAS Visit-Based Continuity subscale has the lowest loading on the latent variable of relational continuity (.78).

The goodness-of-fit statistics for the model in which items are grouped by sub-dimensions of knowledge and concentration of care indicate an improvement over the one-dimensional model (using the difference in chi-square, change Χ² = 1,183 – 1,047 = 136, 1 df, p<.001). The model and the loadings of different items on the scales are presented in Figure 2, where we can see that some items do not have high loadings on the sub-dimension and have a high proportion of residual error (shown to the right of each item). These items may be poorly related to the construct, either because they are not discriminatory or because they relate better to another construct that is not part of the latent variable.

Individual item performance

We used non-parametric item response analysis to evaluate the performance of individual items as a function of knowledge and concentration of care.

For knowledge, all but one item from the PCAS Contextual Knowledge ("knowledge of responsibilities at work or home") demonstrated good performance. The probability of endorsing each response option is highest in a relatively narrow and unique zone over the range of knowledge, and the zones are clearly ordinal, reflecting the assigned value for each option. The item score varies linearly and strongly with overall knowledge, demonstrating excellent discriminability. The exception was the agree–disagree item about "my doctor knowing what I would want done if I were unconscious or in a coma," which is transformed from a 1-to-5 to a 1-to-6 scale. The values attributed to the response options are not endorsed in an ordinal manner, and the item score does not correlate well with increasing knowledge.

The CPCI items eliciting knowledge perform well in terms of discriminability. The best-performing items are about the doctor's understanding of health needs and taking beliefs and wishes into account. For the remaining knowledge items, the two response scale extremes (1=strongly disagree, 6=strongly agree) are most likely to be endorsed, and the mid-options are not always distinguishable from one another, suggesting that a three- or four-point scale would be most appropriate. The three knowledge items in the PCAT-S perform relatively well, though the highest response option is endorsed at average knowledge levels, so the response options principally discriminate between below-average knowledge. The value assigned to each response option is largely appropriate, except the option "probably," which appears to be non-specific as it is endorsed across the entire range of knowledge.

All the items were problematic for the dimension of concentration of care, demonstrating similar problems across instruments. The slope between the item score and the expected total concentration of care was quite steep, demonstrating good discriminability between different levels of concentration, but only in the below-average range. The most positive option is overwhelmingly endorsed starting at below-average levels of concentration. The items are discriminating because the response scales all function essentially as a binary choice of the most positive extreme versus all the other options. The result is that all these items are excellent for detecting low concentration of care, but not for distinguishing between average and excellent concentration of care.

Discussion

Our study demonstrates that these validated subscales appear to be measuring a common underlying construct of relational continuity as we conceived it. They capture two sub-dimensions: accumulated provider knowledge of the patient and concentration of care (seeing the same provider over a series of health events). Consistency of care with the individual's needs – arguably the main benefit of relational continuity – is not captured in any of the subscales we examined. It is presumed that knowledge of the patient translates into consistency, but this presumption is not directly verified.

Accumulated provider knowledge is well measured. The high correlations between PCAS Contextual Knowledge and CPCI Accumulated Knowledge, along with the similar factor loadings of the items in these subscales, lends support that each is measuring a similar sub-dimension. The PCAS Contextual Knowledge subscale shows the best capacity to discriminate between different levels of provider knowledge over the entire range of the scale, but the CPCI Accumulated Knowledge subscale and most items in the PCAT-S Ongoing Care subscale also perform well. Overall, our results show that program evaluators can count on these robust measures to monitor whether health reforms have any negative impact on provider knowledge of the patient. Being known is one of the characteristics patients value most (Turner et al. 2007; Cheraghi-Sohi et al. 2008). It is also associated with safer care (Kuzel et al. 2004), including accurate diagnosis and application of wait-and-see techniques (Hjortdahl and Borchgrevink 1991; Hjortdahl 1992). However, participants in our qualitative discussion groups of the questionnaires were divided on expectations of the depth of the physician–provider relationship; several expressed discomfort with the extent of personal knowledge suggested in the instruments, while others defended this as part of patient-centred care (Haggerty, Beaulieu et al. 2011).

Concentration of care is captured best by the CPCI Preference for Regular Provider subscale, and adequately by the PCAS Visit-Based Continuity subscale and one relevant item in the PCAT-S. These subscales are only weakly correlated, suggesting they are measuring different aspects of concentration of care. In addition, weak correlation between concentration subscales and the percentage of visits to the regular provider, as reported by respondents, raises doubts about whether relational continuity should be inferred from utilization data. Some have questioned whether concentration of care is a good proxy for relational continuity (Reid et al. 2002; Rodriguez et al. 2008). A qualitative study suggests that patients do not consider that consulting other providers necessarily diminishes their commitment to or continuity with their own physician (Roberge et al. 2001). In addition, the items function as yes/no responses rather than as the intended ordinal scales, and they are more informative and discriminatory for poor than for good concentration of care. This finding suggests they can be good indicators of discontinuity or care fragmentation, but are less useful for measuring high levels of continuity. Using any of these instruments, then, to capture improvements in the concentration of care may be useful only if the baseline experience is poor. All the analyses suggest more development is needed in both our conceptual understanding of this aspect of relational continuity and how best to measure it.

Earlier in this paper, we raised concerns about relational continuity in team-based care. Rodriguez and colleagues (2007) found that patient assessments of all aspects of care declined when the proportion of visits to their own primary care provider declined. This finding seems particularly important if the teamwork is not visible to patients (Safran 2003). The new version of the PCAS, the Ambulatory Care Evaluation Survey (Safran et al. 2006), includes a subscale on team care, but it appears to measure patients' perception of teamwork rather than the experience of relational continuity with the team per se. Starfield (1998) found that outcomes associated with "site continuity" were considerably less strong than those associated with physician continuity, unless the providers shared a common approach and philosophy of care. This finding suggests that a proxy for team relational continuity might be obtained from providers' perceptions of team cohesiveness. A recently developed measure of team relational continuity may be promising (Haggerty, Roberge et al. 2011).

Study limits

This study has several limitations. First, limiting the study to those having visited a regular provider in the previous 12 months constrains the range of relational continuity, and the resulting positive skewing of responses compromises the capacity to detect underlying factors. However, because our sampling design essentially oversampled for persons with a poor experience of care, we may have introduced greater variance than would be found in the general population. Second, eliminating subjects with missing values not only reduced statistical power but may have biased the final sample. However, sensitivity analysis using imputation of missing values did not alter our overall conclusions. Finally, our operational definition of relational continuity may be different from that of the instrument developers.

Conclusion

Overall, we found that validated subscales perform relatively well for measuring one dimension of relational continuity: accumulated knowledge. We can recommend that this dimension be used to evaluate the impact of reforms on relationships of patients with individual providers. Subscales measuring concentration of care are most useful for capturing fragmentation and discontinuity. The relational continuity dimension of making care consistent with patient needs is not captured in any instrument and may require further development, as would instruments measuring team relational continuity.

La continuité relationnelle du point de vue du patient : comparaison entre instruments d'évaluation des soins de santé primaires

Résumé

La définition opérationnelle de la continuité relationnelle est « une relation thérapeutique entre un patient et un ou plusieurs soignants, inscrite dans le temps et pour divers services de santé, qui donne lieu à des connaissances accumulées au sujet du patient et à des soins cohérents selon ses besoins. »

Objectif : Examiner à quel point la continuité relationnelle est mesurée par les instruments validés qui servent à évaluer les soins de santé primaires du point de vue du patient.

Méthode : Six cent quarante-cinq adultes ayant eu au moins un contact avec les services de santé au cours des 12 mois antérieurs ont répondu à six instruments qui servent à évaluer les soins de santé primaires. Parmi ces instruments, cinq sous-échelles ont trait à la continuité relationnelle : Primary Care Assessment Survey (PCAS, deux sous-échelles), Primary Care Assessment Tool – version courte (PCAT-S) et Components of Primary Care Index (CPCI, deux sous-échelles). Les résultats ont été normalisés pour permettre des comparaisons descriptives. Les analyses factorielles exploratoires et confirmatoires (modélisation par équation structurelle) ont permis d'examiner l'adéquation à la définition opérationnelle, et l'analyse de réponse par item a permis d'en examiner le rendement en fonction de constructs communs.

Résultats : Toutes les sous-échelles présentent un point de saturation raisonnablement acceptable pour un facteur unique, qui est probablement la continuité relationnelle, mais le meilleur modèle comprend deux facteurs sous-jacents qui correspondent à (1) les connaissances accumulées au sujet du patient et (2) une relation qui se déroule sur plusieurs événements dans les services de santé. Certains items demeurent problématiques même avec le meilleur modèle. La sous-échelle « connaissances contextuelles » du PCAS est la plus discriminante entre différents niveaux de connaissances accumulées, mais cette dimension est également bien captée par la sous-échelle « connaissance accumulées » du CPCI et par la plupart des items de la sous-échelle « soins continus » du PCAT-S. Pour ce qui est de la relation continue sur plusieurs événements, le contenu des items capte la concentration des soins pour un docteur; ce qui est mieux capté par la sous-échelle « préférence pour un clinicien régulier » du CPCI et, à moindre niveau, par la sous-échelle « continuité axée sur les consultations » du PCAS, ainsi que par un item pertinent de la sous-échelle « soins continus » du PCAT-S. Mais la corrélation entre cette dimension et le pourcentage de visites au médecin personnel reste modeste. Les items fonctionnent selon une dichotomie oui/non plutôt que par choix ordinal, et ils renseignent particulièrement dans les cas de faible concentration de soins.

Conclusion : Ces sous-échelles présentent un bon rendement pour les éléments clés de la continuité relationnelle, mais elles ne permettent pas de capter la cohérence des soins. Elles renseignent davantage sur la faible continuité relationnelle.

Comments

Be the first to comment on this!