Healthcare Policy

Healthcare Policy 9(1) August 2013 : 97-117.doi:10.12927/hcpol.2013.23467
Online Exclusive

The Association between Health Information Technology Adoption and Family Physicians’ Practice Patterns in Canada: Evidence from 2007 and 2010 National Physician Surveys

Sisira Sarma, Mohammad Hajizadeh, Amardeep Thind and Rick Chan

Abstract

Objective: To describe the association between health information technology (HIT) adoption and family physicians' patient visit length in Canada after controlling for physician and practice characteristics.

Method: HIT adoption is defined in terms of four types of HIT usage: no HIT use (NO), basic HIT use without electronic medical record system (HIT), basic HIT use with electronic medical record (EMR) and advanced HIT use (EMR + HIT). The outcome variable is the average time spent on a patient visit (visit length). The data for this study came from the 2007 and 2010 National Physician Surveys. A log-linear model was used to analyze our visit length outcome.

Results: The average time worked per week was found to be in the neighbourhood of 36 hours in both 2007 and 2010, but users of EMR and EMR + HIT were undertaking fewer patient visits per week relative to NO users. Multivariable analysis showed that EMR and EMR + HIT were associated with longer average time spent per patient visit by about 7.7% (p<0.05) and 6.7% (p<0.01), respectively, compared to NO users in 2007. In 2010, EMR was not statistically significant and EMR + HIT was associated with a 4% (p<0.1) increased visit length. A variety of practice-related variables such as the mode of remuneration, work setting and interprofessional practice influenced visit length in the expected direction.

Conclusion: Use of HIT is found to be associated with fewer patient visits and longer visit length among family physicians in Canada relative to NO users, but this association weakened in the multivariable analysis of 2010.

A strong primary healthcare system has been shown to improve population health and buffer the health effects of socio-economic disparities at a lower cost than in health systems that rely more extensively on secondary or tertiary care (Macinko et al. 2003, 2007; Starfield et al. 2005). Despite this recognition, the lack of access to a family physician or general practitioner is becoming increasingly an issue in most developed nations. As elsewhere, there are growing pressures in Canada to improve quality of care, reduce wait times and increase access to the services of family physicians. It is widely believed that the introduction of health information technology (HIT) at the family physician's practice level and its appropriate diffusion/integration within the healthcare system is one way to improve the overall efficiency of the healthcare system and address the challenges of providing better access with improved quality of care (Bates et al. 2003; Blumenthal 2009; McInnes et al. 2006).

Adoption of HIT in physicians' practice can be seen as an important strategy to increase productivity and, therefore, efficiency of the Canadian healthcare system for several reasons. First, family physicians are the first point of contact for most Canadians and gatekeepers to the healthcare system; thus, adoption of HIT would enable effective coordination of patient care within the healthcare system (Schoen et al. 2009).

Second, HIT provides an opportunity to enhance quality of healthcare through sharing health information across care settings (Bryan and Boren 2008; Thomas et al. 1999; Van Der Kam et al. 2000). Many individuals seeking health services often visit multiple providers who are not directly connected with one another, resulting in an incomplete view of the available information that, in turn, could compromise the quality of care the patient receives. Having a complete picture of patients' interactions with the healthcare system would minimize medical errors and other adverse events.

Third, effective adoption of HIT could enable improved patient compliance with recommended care and physician compliance with recommended preventive care and disease management guidelines (Nilasena and Lincoln 1995). Given the increasing incidence of chronic diseases and a growing demand for management of them, there is an ongoing need to deliver preventive care, especially to the aging population (Canada Health Infoway 2012). Thus, it is important that detailed information about patients' health is transmitted electronically in real time and shared among the healthcare providers across primary, secondary and tertiary sectors.

Finally, from an information management perspective, the paper-based system creates a burden for physicians and payers owing to the large volume of transactions. Adoption of HIT would facilitate coordination and management of critical information across the healthcare system (Hillestad et al. 2005).

Although adoption of HIT has the potential to improve quality of healthcare, patient safety and efficiency at a lower cost (Blumenthal et al. 2008; Buntin et al. 2011; Hillestad et al. 2005), empirical findings from the literature are ambiguous at best (Black et al. 2011). Many positive benefits are observed among the early adopters of HIT, but dissatisfaction with the HIT system among physicians remains a major obstacle in realizing its full potential benefits (Buntin et al. 2011). Thus, it is interesting to investigate the association between HIT adoption and practice patterns of family physicians.

Why Do Practice Patterns Matter?

Family physicians generally spend the vast majority of their time on direct patient care activities (Sarma et al. 2010a,b; Sarma et al. 2011). Arguably, adoption of HIT in their practices would decrease time spent on diagnosis and treatment, and subsequently decrease time spent on patient visits. Familiarity with the HIT system can improve workflow among physicians by significantly reducing the time from diagnosis to treatment (Breil et al. 2009). The delivery of electronic results with the support of electronic documentation to follow up results may allow quick turnover of patients with effective treatment (Callen et al. 2010). Indeed, a cross-sectional study found that physicians using an electronic clinical management and reporting system in multiple HIV clinics saved a significant amount of time with an average of 16.1 minutes per visit (Magnus et al. 2009). In their time-motion study, Pizziferri and colleagues (2005) also found a decreased overall time spent per patient (i.e., direct patient care, indirect patient care and administrative work) after implementation of HIT.

The time saved can be allocated to other aspects of physician work, such as providing more medical or clinical services or treating more patients. EMRs that are linked to laboratory information systems can improve communication and reduce medical errors. Aside from error reduction, HIT systems that have automated laboratory processes can increase productivity of the practice (Uettwiller-Geiger 2005).

Although adoption of HIT has the potential to improve time efficiency, there may be certain unintended consequences on the part of physicians in the short run owing to (a) difficulties with using the new technology and an inadequate support system and (b) more in-depth interactions regarding patients' medications, referrals and laboratory tests. Adopting HIT in practice requires an investment of resources and time; for physicians, the time to learn how to use the system usually translates into time taken away from direct patient care activities (Audet et al. 2004). Indeed, physicians using EMRs reported that they spent more time per patient for a period of months or even years after EMR implementation (Miller and Sim 2004). Most providers do report spending more time than usual when HIT is implemented initially, and some practices experienced substantial financial losses (Miller et al. 2005). This may be because physicians who use an EMR system may take on a more active role in clarifying information, encouraging questions and ensuring completeness at the end of the visit compared to those who do not use EMR (Arar et al. 2004; Makoul et al. 2001). A review study by Poissant and colleagues (2005) on documentation time by physicians found that the use of bedside or point-of-care computer systems has increased documentation time of physicians by 17.5%.

A large literature on HIT deals with policies, adoption of and barriers to implementation, and medical errors, but very little attention has been paid to the resulting practice patterns of physicians. Family physicians are undoubtedly the main actors in adopting information technology in Canada's healthcare system. Thus, understanding the impact of HIT on visit length is clearly important.

Data and Methods

This study utilized individual-level data from the 2007 and 2010 National Physician Surveys (NPS) conducted collaboratively by the College of Family Physicians of Canada, the Canadian Medical Association and the Royal College of Physicians and Surgeons of Canada. The NPS is a census survey of all practising physicians in Canada and is designed to generate nationally representative estimates of family physicians (FPs) or general practitioners (GPs) and specialists.

The 2007 NPS consists of two versions of a FP/GP questionnaire: the shorter version contains some core questions and the longer version contains in-depth detailed questions on many topics relevant for FP/GPs. A census was attempted for the 2007 NPS main questions in the shorter version of the questionnaire. A stratified sampling technique was used to determine who would receive the shorter questionnaire and who would receive the detailed questionnaire. All the content from the core questionnaire is also captured on the detailed questionnaire. We utilized the longer version of the questionnaire completed by FPs, which had a detailed module on HIT use in their practice (CFPC et al. 2012). The 2010 NPS was a census survey of all physicians in Canada.1 The overall response rates for FPs in the 2007 NPS long version and 2010 NPS were 32.07% and 18.97%, respectively. Demographic characteristics of respondents and total FP population are reported in the Appendix A. Physicians aged 55 years and over represent a slightly larger proportion of the NPS respondents compared to the total family physician population, while physicians in the age group 35–44 years are relatively less represented. In both surveys, there are proportionately more female respondents than the total FP population (41.04% vs. 37.49% in 2007 and 44.06% vs. 39.95%). To account for survey non-response bias, sampling weights were developed by the NPS team for use in all analysis. The non-response adjustments were performed at the province level by physician type, age group and gender using the calibration method based on the NPS Masterfile on all licensed FPs in Canada.2 All analyses were weighted using the sampling weights relevant for the longer version of the questionnaire in 2007 and 2010 sampling weights provided by NPS data to represent the FP/GP population in Canada.

The outcome variable, visit length, is a continuous variable, which is defined as the average time in minutes spent on a patient visit. This variable was created by dividing the average amount of time (in hours) that the physician reported spending with patients in direct patient care each week by the average number of patients the physician reported seeing each week, then multiplying by 60. This variable reflects the efficiency of providing patient care and can be used to establish whether the adoption of HIT has an association with time spent on a patient visit.

In our analysis, HIT is defined as a comprehensive tool for management of health information, and it deals with storage, retrieval, sharing and use of information for delivering patient care. Based on 11 identical items listed for capturing HIT in the 2007 and 2010 surveys, we organized these into three meaningful types of HIT adoption to assess its association with visit length (Table 1). Four types of HIT reflecting the extent of HIT adoption are as follows: no HIT use (NO), basic HIT use without electronic medical record (HIT), basic HIT use along with electronic medical record system (EMR) and advanced HIT use (EMR + HIT). The HIT type variable mentioned above was constructed in two steps. First, we classified each EMR aspect into one of the three categories: (1) the common functions in the practice such as electronic billing, online access and e-mailing were classified as HIT; (2) use of electronic functions such as entering or retrieving patient notes, recommended patient care, warning systems/drug interactions by FPs were classified as EMR; and (3) the sharing of information outside of physicians' practice was classified as EMR + HIT. In the second step, we constructed the type of HIT into one of the four groups as described above and presented in Table 2.


TABLE 1. List of NPS health information technology items and their assigned classifications
HIT Item Classification
E-mail HIT
Online access to journals, clinical practice guidelines, medical databases (e.g., MEDLINE) HIT
Electronic patient appointment/scheduling system HIT
Electronic billing HIT
Electronic records to enter and retrieve clinical patient notes EMR
Electronic reminder systems for recommended patient care EMR
Electronic warning systems for adverse prescribing and/or drug interactions EMR
Electronic interface to external pharmacy/pharmacist EMR + HIT
Electronic interface to external laboratory/diagnostic imaging EMR + HIT
Electronic interface to other external systems (e.g., hospitals, other clinics) for accessing or sharing patient information EMR + HIT
Telemedicine/webcasting/videoconferencing EMR + HIT

 


TABLE 2. Categorical coding of HIT items among HIT users
    HIT
    No Yes
EMR No NO HIT
Yes EMR EMR + HIT

 

We controlled for characteristics of physicians, characteristics of the practice and type of patient population served in the practice in all regression models. Practice characteristics in the literature review suggested that variables such as remuneration scheme (Devlin and Sarma 2008; Sarma et al. 2010a; Shelton et al. 2009), work setting (Ammenwerth and Spötl 2009; Anderson et al. 2007), geographical jurisdiction (Hutten-Czapski et al. 2004; Van Den Berg et al. 2009) and practice organization (Friedberg et al. 2007; Hutten-Czapski et al. 2004) are associated with the outcome variable of interest. Additionally, the province variable is also included in our regression because provincial healthcare systems differ in terms of fee schedules, health policies and physician densities (Hutten-Czapski et al. 2004).

The physician demographics such as sex (French et al. 2006; Sarma et al. 2010a,b; Sarma et al. 2011; Watson et al. 2006) and age (Hutten-Czapski et al. 2004; Watson et al. 2006) are included in the regression model. The patient population is also included in our regression to control for the characteristics of the patients of the physician's practice. The variables included are elderly (Nabalamba and Millar 2007; Watson et al. 2005), mental illness (Wilson and Childs 2002), diabetes (Cherry et al. 2008; Pohar and Johnson 2007), obesity (Pearson et al. 2009), hypertension (Cherry et al. 2008) and heart conditions (Mackie et al. 2007).

Physicians who do not report patient care hours and those who practise in the territories were excluded. In order to avoid the influence of outliers, we restricted the sample to those physicians who provide between 15 and 80 hours in direct patient care (with or without teaching) per week and undertake at least 15 patient visits per week, and we restricted the sample to a maximum of less than 50 minutes per patient visit. The average visit length in minutes was estimated by taking the average number of hours spent on direct patient care divided by the average number of patient visits per week and multiplied by 60. Our inclusion/exclusion criteria yielded a sample size of 2,459 in 2007 and 4,003 in 2010 after excluding missing and inappropriate records. Table 3 reports definitions and descriptive statistics of all variables used in the analyses.


TABLE 3. Definitions and descriptive statistics of variables
  Definition 2007
Mean
(SD)
2010
Mean
(SD)
Continuous Variables
   Hours worked (H) Hours worked/week on direct patient care activities with or without teaching (15 ≥hours worked ≤80) 36.29 (11.25) 35.68 (10.93)
   Patient visits (Q) Patient visits/week to general practitioner/ family physician's office (ca. 15 visits) 124.25 (58.51) 118.54 (58.29)
   Visit length Average time spent on a patient visit measured in minutes (maximum visit length ≤50) 20.24 (8.63) 21.12 (9.16)
   ln(H) Natural logarithmic of H 3.54 (0.32) 3.53 (0.32)
   Outcome Variables
   ln(Visit length) Natural logarithmic of visit length 2.93 (0.40) 2.97 (0.42)
   ln(Q) Natural logarithmic of Q 4.71 (0.49) 4.66 (0.50)
Continuous Variables
   Explanatory Variables
   HIT Type
     NO (ref.) 1 if the physician is a NO HIT and EMR user, 0 otherwise 17% 07%
     HIT 1 if the physician is a HIT user, 0 otherwise 30% 21%
     EMR 1 if the physician is a EMR user, 0 otherwise 10% 11%
     EMR + HIT 1 if the physician is a EMR + HIT user, 0 otherwise 43% 61%
  100% 100%
   Remuneration Scheme
     Fee-for-service (ref.) 1 if the physician received 90% or more of his/her professional income from a fee-for-service remuneration, 0 otherwise 54% 47%
     Blended payment 1 if the physician's primary professional income comes from at least two or more variety of sources (e.g. fee-for-service, salary, capitation, sessional/per diem/hourly, service contract, incentives and premiums) and none of the sources constitutes 90% or more, 0 otherwise 35% 40%
     Other payments 1 if the physician received 90% or more of his/her professional income from sources other than fee-for-service or blended 11% 13%
   Work Setting
     Private clinic (ref.) 1 if the physician works in a private clinic or free-standing walk-in clinic, 0 otherwise 73% 71%
     Community clinic 1 if the physician works in a community clinic or community health centre, 0 otherwise 8% 8%
     Academic health sciences centre 1 if the physician works in an academic health sciences centre, 0 otherwise 4% 4%
     Community hospital 1 if the physician works in a community hospital or other hospital, 0 otherwise 12% 6%
     Other settings 1 if the physician works in other settings, 0 otherwise 4% 10%
  100% 100%
   Province
     Ontario (ref.) 1 if the physician is practising in Ontario, 0 otherwise 37% 36%
     Newfoundland and Labrador 1 if the physician is practising in Newfoundland, 0 otherwise 2% 2%
     Prince Edward Island 1 if the physician is practising in PEI, 0 otherwise 04% 03%
     Nova Scotia 1 if the physician is practising in Nova Scotia, 0 otherwise 4% 3%
     New Brunswick 1 if the physician is practising in New Brunswick, 0 otherwise 3% 3%
     Quebec 1 if the physician is practising in Quebec, 0 otherwise 21% 23%
     Manitoba 1 if the physician is practising in Manitoba, 0 otherwise 4% 3%
     Saskatchewan 1 if the physician is practising in Saskatchewan, 0 otherwise 3% 3%
     Alberta 1 if the physician is practising in Alberta, 0 otherwise 11% 12%
     British Columbia 1 if the physician is practising in British Columbia, 0 otherwise 16% 15%
  100% 100%
   Practice Organization
     Solo practice (ref.) 1 if the physician is a solo practitioner, 0 otherwise 23% 23%
     Group practice 1 if the physician is practising in a group, 0 otherwise 56% 53%
     Interprofessional practice 1 if the physician is practising in an interprofessional setting, 0 otherwise 21% 20%
     Other practice 1 if other practices (e.g., research unit, nursing home, etc.), 0 otherwise 5%
   Age
     <35 (ref.) 1 if age of the physician is <35, 0 otherwise 10% 9%
     35–44 1 if age of the physician is 35–44, 0 otherwise 27% 26%
     45–54 1 if age of the physician is 45–54, 0 otherwise 36% 33%
     55–64 1 if age of the physician is 55–64, 0 otherwise 22% 25%
     ≥65 1 if age of the physician is ≥65, 0 otherwise 5% 7%
  100% 100%
   Sex
     Male (ref.) 1 if the physician is male, 0 female 62% 59%
     Female 1 if the physician is female, 0 male 38% 41%
    100% 100%
   Region
     City (ref.) 1 if the physician's main patient care setting is located in an inner city, suburban or urban, 0 otherwise 66% 64%
     Small Town 1 if the physician's main patient care setting is located in a small town, 0 otherwise 19% 19%
     Rural 1 if the physician's main patient care setting is located in a rural area, 0 otherwise 14% 15%
     Undefined location 1 if the location is other than city, small town and rural, 0 otherwise 2% 2%
    100% 100%
   Patient Population
     Elderly 1 if more than 10% of the physician patients are elderly, 0 otherwise 44% 34%
     Hypertension 1 if more than 10% of the physician patients have high blood pressure, 0 otherwise 29% 26%
     Diabetes 1 if more than 10% of the physician patients are diabetic, 0 otherwise 26% 24%
     Heart disease 1 if more than 10% of the physician patients have been diagnosed with heart disease, 0 otherwise 23% 19%
     Obesity 1 if more than 10% of the physician patients are obese, 0 otherwise 20% 16%
     Mental illness 1 if more than 10% of the physician patients have been diagnosed with mental health problems, 0 otherwise 13% 16%
Number of Observations   2,459 4,003
ref.: reference category in regression analysis; SD = standard deviations reported in parentheses

 

Because visit length was skewed (skewness of visit length = 1.21 in 2007 and 1.1 in 2010), we transformed the visit length to natural logarithms to reduce the skewness of our outcome variable (skewness of ln(visit length) = 0.14 in 2007 and 0.03 in 2010). Aside from statistical considerations, it is also common practice to allow a non-linear relationship between patient visits and practice characteristics by specifying a log-linear model while analyzing physician throughputs (e.g., Devlin and Sarma 2008). When the dependent variable is log-transformed, it is commonly called a log-linear model (Stock and Watson 2003). Thus, a log-linear model was used in the analysis of the visit length in minutes. Similarly, for the patient visits outcome measure, we used natural logs of patient visits instead of levels owing to the skewed nature of the data (skewness of patient visits = 1.11 in 2007 and in 1.2 in 2010; skewness of ln(patient visits) = –0.41 in 2007 and –0.33 in 2010). In the log-linear model (i.e., lnYi = a + βXi + ei), the interpretation of the estimated coefficient β, say b, is that a one-unit increase X will lead to an expected increase in log Y of exp(b) units. The percentage interpretation of a dummy variable in the context of a log-linear model is: %ΔY = [exp(b)–1]*100 (Halvorsen and Palmquist 1980). We used this interpretation to explain our results in this paper. As per the requirement of the NPS survey methodology (CFPC et al. 2012), relevant sampling weights were applied to all regression analyses to ensure the representativeness of the data for family physicians in Canada. We corrected the unknown form of heteroscedasticity in our regression analyses to obtain precise standard errors for inference analysis.

Results

Our data exhibit some general trends in the demographics of Canadian FPs and their changing practice patterns. Between 2007 and 2010, the proportion of physicians adopting EMR + HIT increased from 43% to 61%. The proportion of physicians aged 55 years and older increased, and the proportion of female physicians increased over the two survey years. The proportion of family physicians obtaining at least 90% of their professional income from a fee-for-service (FFS) remuneration scheme declined from 54% to 47% during this period. The average hours worked per week remained in the neighbourhood of 36 hours in both 2007 and 2010, but fewer patient visits were undertaken in 2007. Table 4 reports average hours worked per week and average weekly patient visits in 2007 and 2010 across all HIT categories. It is seen from Table 4 that the average hours worked per week vary little across HIT categories, but differences in the patient visits were observed. Indeed, the Kruskal-Wallis rank test statistics showed that average weekly patient visits were significant at the 1% level while the hours worked per week were statistically insignificant. Particularly, physicians using EMR were undertaking about 5 fewer patient visits per week and those using EMR + HIT were undertaking in the range of 9 to 11 fewer patient visits compared to NO users.


TABLE 4. Average patient visits and hours worked per week across HIT categories
  2007 NPS 2010 NPS
  Average Patient Visits (SD) Average Hours Worked (SD) Average Patient Visit (SD) Average Hours
NO 128.20 (58.12) 36.43 (11.41) 126.27 (65.04) 35.87 (10.70)
HIT 129.30 (60.17) 36.20 (11.03) 124.75 (62.63) 35.50 (10.89)
EMR 123.54 (60.50) 36.86 (12.30) 121.60 (61.43) 35.79 (11.07)
EMR + HIT 119.36 (56.65) 36.17 (11.10) 115.05 (55.06) 35.71 (10.95)
Kruskal-Wallis Rank Test 17.314*** 1.241 17.084*** 0.087

SD: Standard deviations reported in parentheses

*** p<0.01


 

Table 5 reports the estimated coefficients showing the association between HIT adoption and average visit length. Results demonstrate that EMR and EMR + HIT adopters were associated with an increase of 7.7% (p<0.05) and 6.7% (p<0.01), respectively, in their expected mean time spent per patient visit compared to NO users in 2007. However, HIT variables were not statistically significant in 2010 with the exception of EMR + HIT, which is statistically significant at the 10% level.


TABLE 5. Association between HIT use and family physicians' average visit lenght: log-linear model results
Variable 2007 NPS
Estimated Coefficient
(Standard Error)
2010 NPS
Estimated Coefficient
(Standard Error)
HIT Type
   HIT 0.039*
(0.023)
0.007
(0.027)
   EMR 0.074**
(0.032)
0.023
(0.030)
   EMR + HIT 0.065***
(0.022)
0.041*
(0.025)
Age
   35–44 –0.044
(0.029)
–0.055**
(0.022)
   45–54 –0.001
(0.028)
–0.076***
(0.021)
   55–64 –0.011
(0.030)
–0.081***
(0.022)
   ≥65 –0.018
(0.043)
–0.065**
(0.031)
Sex
   Female 0.096***
(0.016)
0.104***
(0.013)
Remuneration
   Blended payment 0.078***
(0.018)
0.101***
(0.014)
   Other payments 0.151***
(0.029)
0.113***
(0.021)
Practice Organization
   Group 0.004
(0.020)
0.007
(0.016)
   Interprofessional 0.052*
(0.027)
0.096***
(0.021)
   Other practice   0.047
(0.032)
Work Setting
   Community clinic 0.223***
(0.033)
0.209***
(0.026)
   Academic health sciences centre 0.131***
(0.050)
0.298***
(0.036)
   Hospital 0.215***
(0.029)
0.288***
(0.031)
   Other setting 0.207***
(0.043)
0.084***
(0.023)
Patient Population
   Elderly 0.011
(0.019)
0.026*
(0.015)
   Hypertension –0.067**
(0.032)
0.016
(0.024)
   Diabetes 0.109***
(0.032)
–0.067***
(0.025)
   Heart disease –0.080***
(0.031)
–0.025
(0.024)
   Obesity 0.013
(0.025)
0.018
(0.020)
   Mental illness 0.023
(0.027)
0.045**
(0.019)
Region
   Town 0.047**
(0.020)
0.047***
(0.015)
   Rural 0.077***
(0.024)
0.114***
(0.018)
   Other 0.061
(0.063)
0.123**
(0.053)
Province
   Newfoundland and Labrador –0.040
(0.058)
–0.066
(0.042)
   Prince Edward Island 0.023
(0.069)
–0.097
(0.067)
   Nova Scotia 0.023
(0.029)
0.045
(0.031)
   New Brunswick –0.086**
(0.041)
–0.005
(0.038)
   Quebec 0.242***
(0.022)
0.254***
(0.017)
   Manitoba 0.035
(0.033)
0.064*
(0.034)
   Saskatchewan –0.003
(0.042)
–0.016
(0.040)
   Alberta 0.019
(0.028)
0.092***
(0.021)
   British Columbia 0.013
(0.023)
0.014
(0.018)
   Constant 2.669***
(0.039)
2.718***
(0.036)
   Number of observations 2,459 4,003
   R-squared 0.241 0.263

Heteroscedastic corrected standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1


 

The results for other covariates revealed that the type of remuneration scheme affects mean time spent per patient visit; physicians who are remunerated by blended/mixed payment schemes were found to spend more time on a patient visit compared to physicians who received a FFS payment (8.1% in 2007 and 10.6% in 2010). Similarly, as compared to FFS, other types of payments were positively associated with greater visit length (16% in 2007 and 12% in 2010). Work setting is another factor that explains the variation in average visit length among physicians. According to the results, physicians working in a community clinic, an academic setting, a community health or other setting spent more time on each patient visit than those physicians working in private clinics.

Province of practice is another factor that affected visit length. While physicians working in Quebec spent more time on each patient visit compared to those practising in Ontario in 2007 and 2010, the average length of patient visit was relatively less among physicians working in New Brunswick in 2007 and greater among physicians working in Alberta in 2010. The results also indicated the influence of geographic factors on patient visit length. As can be seen, physicians working in a small town, a rural area or other undefined remote area spent more time in patient visits, a finding that may reflect the special funding mechanisms put forward by the provinces to motivate physicians to work in rural areas. Among the physician demographic variables, being female was found to be associated with a greater amount of time spent on a patient visit.

Discussion

The objective of our study was to describe the association between HIT and visit length among FPs in Canada after controlling for physician and practice characteristics. In the literature, visit length is considered an important aspect of understanding physician practice patterns. Utilizing nationally representative data sets from the 2007 and 2010 National Physician Surveys, we found that adoption of HIT is associated with fewer patient visits in both 2007 and 2010. Our multivariable analysis showed that HIT adoption is associated with a longer visit length among EMR and EMR + HIT users compared to NO HIT users in 2007. However, this association weakened in our multivariable analysis of 2010 data. There are several possible explanations for fewer patient visits associated with advanced HIT use. From a patient's perspective, use of advanced HIT by family physicians may suggest a patient-centred approach, as some evidence suggests that longer visit lengths have been associated with dialogues initiated by the patient (Martin et al. 1999). As noted in the literature review, the use of HIT in the practice may encourage patients to ask more questions of their physicians and thus contribute to the patient-centred approach to care delivery (Arar et al. 2004; Makoul et al. 2001). Also, it has been noted that longer consultations provided by the physician may be associated with greater patient satisfaction, higher quality of care and increased health promotion (Wilson and Childs 2002).

Although longer visit length may be beneficial to the patient–physician interaction following HIT adoption, this may not be the complete story. From a physician's perspective, an alternative explanation could be that the longer visits may be due to the fact that physicians may be in the early phase of HIT adoption (i.e., switching from writing on a paper chart to entering data onto multiple computer screens). Thus, it may not be unreasonable to assume that longer visit length could also be attributed to physicians struggling to put the relevant patient data into the correct place in the EMR system. The results were corroborated when the log of patient visit was considered our outcome variable after adjusting for the log of hours worked and all other covariates.

The policy implications of this study are that the promising effect of HIT on time efficiency on the part of physicians may not have been fully realized in the Canadian context. It has been argued by many researchers that a fully integrated, interoperable HIT system adopted in FPs' practice that is linked to other clinics, hospitals and care providers across the continuum of the healthcare system will likely have a positive impact on healthcare delivery systems (e.g., Blumenthal 2009; Bryan and Boren 2008; Buntin et al. 2011; Hillestad et al. 2005; Schoen et al. 2009). Thus, support programs to teach physicians the effective use of HIT in their practice could reduce the physicians' time costs and help to realize the benefits of HIT without compromising the number of patients being treated in their day-to-day practice. Our study also underscored the importance of type of remuneration and the organization of practice when considering policies to promote the use of HIT in primary care practices.

Limitations

Our study has several limitations. First, it relies on self-reported hours worked and patient visits per week. Some evidence suggests that physicians tend to overestimate the number of hours spent on direct patient care (Casey et al. 2005). The number of patient visits per week may have been subject to measurement error. Another limitation is that a causal interpretation is limited given the cross-sectional nature of the study. In other words, there could be a selection bias such that certain types of physician may be more attracted to adopting advanced HIT in their practice quite rapidly. Although use of a rich set of controls in our regression models minimized the selection bias, it certainly does not eliminate it.

Although non-response bias is accounted for by applying respective sampling weights, the low overall response rates of 32.07% in 2007 and 18.97% in 2010 remained a concern. The results of this study warrant further investigation into the organization of physicians' practice, particularly the mode of remuneration, group practice and collaboration with other healthcare professionals and the effects on using HIT and patient outcomes. The unintended consequences of implementing HIT can exacerbate the problem of access to primary healthcare services in the short run if more time is spent per patient visit. Whether adoption of HIT has a positive or negative effect on the quality of care that patients receive and patient outcomes in Canada are topics for future research.

 


 

Relation entre l'adoption des technologies de l'information sur la santé et les schémas de pratique des médecins de famille au Canada : données provenant des sondages nationaux des médecins de 2007 et de 2010

Objectif: Décrire la relation entre l'adoption des technologies d'information sur la santé (TIS) et la durée des consultations chez les médecins de famille au Canada, après avoir contrôlé les caractéristiques des médecins et des pratiques.

Méthode: L'adoption des TIS se définit en fonction de quatre types d'usage des TIS : aucun usage des TIS [NO], un usage rudimentaire des TIS sans système de dossiers médicaux informatisés (DMI) [TIS], un usage rudimentaire des TIS avec système de DMI [DMI] et un usage étendu des TIS [DMI+TIS]. La variable dépendante est la moyenne du temps de consultation (durée de consultation). Les données de cette étude proviennent des sondages nationaux des médecins de 2007 et de 2010. Un modèle log-linéaire a été employé pour analyser le résultat des durées de consultation.

Résultats: Le temps de travail moyen est d'environ 36 heures par semaine tant en 2007 qu'en 2010, mais les usagers DMI et DMI+TIS effectuent moins de consultations par semaine comparativement aux usagers NO. L'analyse multivariable montre que les usagers DMI et DMI+TIS sont associés à un plus long temps moyen de consultation de l'ordre de 7,7 % (p<0,05) et 6,7 % (p<0,01), respectivement, comparativement aux usagers NO en 2007. Les données de 2010 sur les usagers DMI ne sont pas statistiquement significatives, alors que celles des usagers DMI+TIS sont associées à une plus grande durée de consultation de l'ordre de 4 % (p<0,1). Un certain nombre de variables liées à la pratique, telles que le mode de rémunération, le cadre de travail et la pratique interprofessionnelle influencent la durée de consultation dans le sens escompté.

Conclusion: Il semble y avoir, chez les médecins de famille au Canada, un lien entre l'usage des TIS et un moindre nombre de consultations par semaine ainsi qu'une plus grande durée de consultation, comparativement aux usagers NO, mais ce lien est moins marqué dans l'analyse multivariable des données de 2010.

 


 

APPENDIX A


Demographic characteristics of respondents and total physician population
  2007 NPS (Family Physicians) 2010 NPS (Family Physicians)
  Respondents (n) Total Eligible Population (N) Respondents (n) Total Eligible Population (N)
Total 32.07% (10,270) 100% (32,026) 18.97% (6,602) 100% (34,810)
Age Group
   <35 8.70% (894) 8.60% (2,753) 9.19% (607) 8.57% (2,983)
   35–44 23.51% (2,414) 26.04% (8,339) 20.22% (1,335) 23.03% (8,016)
   45–54 33.62% (3,453) 32.43% (10,387) 30.43% (2,009) 30.90% (10,756)
   55–64 24.55% (2,521) 22.99% (7,364) 27.14% (1,792) 24.72% (8,606)
   65+ 8.72% (896) 8.42% (2,695) 11.35% (749) 10.27% (3,574)
   Unknown 0.90% (92) 1.52% (488) 1.67% (110) 2.51% (875)
Total 100% 100% 100% 100%
Sex
   Male 57.96% (5,952) 62.51% (20,019) 55.62% (3,672) 59.64% (20,761)
   Female 42.04% (4,317) 37.49% (12,006) 44.06% (2,909) 39.95% (13,905)
   Unknown 0.01% (1) 0.00% (1) 0.32% (21) 0.41% (144)
Total 100% 100% 100% 100%
Sources: https://nationalphysiciansurvey.ca/surveys/2007-survey/2007-results/;
https://nationalphysiciansurvey.ca/surveys/2010-survey/2010-results/.

 

APPENDIX B


Association between HIT use and family physicians' weekly patient visits: log–linear results
Variables 2007 NPS
Estimated Coefficient
(Standard Error)
2010 NPS
Estimated Coefficient
(Standard Error)
   ln(Hours worked) 0.689***
(0.027)
0.711***
(0.021)
   HIT Type
   HIT –0.045**
(0.022)
–0.013
(0.026)
   EMR –0.077**
(0.03)
–0.024
(0.029)
   EMR + HIT –0.065***
(0.021)
–0.042*
(0.024)
Age
   35–44 0.027
(0.028)
0.036*
(0.021)
   45–54 –0.004
(0.027)
0.059***
(0.02)
   55–64 –0.001
(–0.029)
0.065***
(–0.022)
   ≥65 –0.027
(0.042)
0.009
(0.031)
Sex
   Female –0.151***
(0.016)
–0.151***
(0.013)
Remuneration
   Blended payment –0.095***
(0.017)
–0.116***
(0.014)
   Other payments –0.184***
(0.028)
–0.135***
(0.02)
Practice Organization
   Group –0.019
(0.019)
–0.022
(0.016)
   Interprofessional –0.073***
(0.026)
–0.120***
(0.021)
   Other practice –0.062**
(0.031)
Work Setting
   Community clinic –0.249***
(0.033)
–0.240***
(0.026)
   Academic health sciences centre –0.174***
(0.05)
–0.326***
(0.035)
   Hospital –0.211***
(0.028)
–0.289***
(0.03)
   Other setting –0.266***
(0.045)
–0.106***
(0.022)
Patient Population
   Elderly –0.007
(0.018)
–0.024*
(0.014)
   Hypertension 0.048
(0.031)
–0.019
(0.023)
   Diabetes –0.095***
(0.032)
0.068***
(0.024)
   Heart disease 0.083***
(0.031)
0.033
(0.023)
   Obesity –0.008
(0.024)
–0.015
(0.02)
   Mental illness –0.018
(0.026)
–0.048**
(0.019)
Region
   Town –0.040**
(0.019)
–0.034**
(0.015)
   Rural –0.064***
(0.023)
–0.087***
(0.017)
   Other –0.061
(0.066)
–0.151***
(0.051)
Province
   Newfoundland and Labrador 0.046
(0.055)
0.066
(0.043)
   Prince Edward Island –0.002
(0.073)
0.096
(0.071)
   Nova Scotia –0.027
(0.028)
–0.042
(0.03)
   New Brunswick –0.084**
(0.039)
0.002
(0.036)
   Quebec –0.251***
(0.021)
–0.266***
(0.017)
   Manitoba –0.024
(0.032)
–0.053
(0.033)
   Saskatchewan 0.012
(0.04)
0.036
(0.039)
   Alberta –0.016
(0.028)
–0.098***
(0.02)
   British Columbia –0.024
(0.022)
–0.025
(0.018)
   Constant 2.586***
(0.108)
2.460***
(0.087)
   Number of observations 2,459 4,003
   R-squared 0.523 0.524

Heteroscedastic corrected standard errors in parentheses; *** p<0.01, ** p<0.05, * p<0.1
Note: The coefficient on ln(Hours worked) should be interpreted as the (percentage change in Q/percentage change in H). Our results show that a 10% increase in hours worked per week leads to about a 7% increase in patient visits per week.


 

About the Author

Sisira Sarma, MPhil, PhD, Assistant Professor, Department of Epidemiology and Biostatistics, University of Western Ontario, London, ON

Mohammad Hajizadeh, MS, PhD, Post Doctoral Fellow, Department of Epidemiology and Biostatistics, University of Western Ontario, London, ON

Amardeep Thind, MD, PhD, Associate Professor, Department of Epidemiology and Biostatistics, Centre for Studies in Family Medicine, University of Western Ontario, London, ON

Rick Chan, MSc, Graduate Student, Department of Epidemiology and Biostatistics, University of Western Ontario, London, ON

Correspondence may be directed to: Dr. Sisira Sarma, Department of Epidemiology and Biostatistics, Kresge Building, Room K201, University of Western Ontario, London, ON N6A 5C1; e-mail: ssarma2@uwo.ca.

Acknowledgment

We thank three anonymous reviewers for their thoughtful comments and suggestions, which have improved the paper. This study utilizes the 2007 and 2010 National Physician Surveys (NPS) micro data, which are available under a special agreement for this research. The NPS project is co-led by the College of Family Physicians of Canada (CFPC), the Canadian Medical Association (CMA) and the Royal College of Physicians and Surgeons of Canada (RCPSC) and is supported by the Canadian Institute for Health Information (CIHI) and Health Canada. The views expressed in this paper are those of the authors and do not necessarily reflect the views of any affiliated organization.

References

Ammenwerth, E. and H.-P. Spötl. 2009. "The Time Needed for Clinical Documentation versus Direct Patient Care." Methods of Information in Medicine 48(1): 84–91.

Anderson, D.M., L.M. Asher and E.A. Wilson. 2007. "Physician Computer Skills: A Prerequisite to the Future in Healthcare Services." Journal of the Kentucky Medical Association 105(2): 67–71.

Arar, N., J. McGrath, J. Rosales and J. Pugh. 2004. "The Role of Electronic Medical Records in Improving Patient-Centred Care in Outpatient Encounters." Journal on Information Technology in Healthcare 2: 187–202.

Audet, A.-M., M.M. Doty, J. Peugh, J. Shamasdin, K. Zapert and S. Schoenbaum. 2004. "Information Technologies: When Will They Make It into Physicians' Black Bags?" Medscape General Medicine 6(4): 2.

Bates, D.W., M. Ebell, E. Gotlieb, J. Zapp and H.C. Mullins. 2003. "A Proposal for Electronic Medical Records in US Primary Care." Journal of the American Medical Informatics Association 10(1): 1–10. doi: 10.1197/jamia.M1097.

Black, A.D., J. Car, C. Pagliari, C. Anandan, K. Cresswell, T. Bokun et al. 2011. "The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview." PLoS Medicine 8(1): 16.

Blumenthal, D. 2009. "Stimulating the Adoption of Health Information Technology." New England Journal of Medicine 105(3): 1477–79.

Blumenthal, D., C. DesRoches, K. Donelan, T. Ferris, A. Jha, R. Kaushal et al. 2008. Health Information Technology in the United States: Where We Stand, 2008. Retrieved June 10, 2013. <https://folio.iupui.edu/bitstream/handle/10244/784/hitreport.pdf>.

Breil, B., A. Semjonow and M. Dugas. 2009. "HIS-Based Electronic Documentation Can Significantly Reduce the Time from Biopsy to Final Report for Prostate Tumours and Supports Quality Management as Well as Clinical Research." BMC Medical Informatics and Decision Making 9(1): 5.

Bryan, C. and S.A. Boren. 2008. "The Use and Effectiveness of Electronic Clinical Decision Support Tools in the Ambulatory/Primary Care Setting: A Systematic Review of the Literature." Informatics in Primary Care 16(2): 79–91.

Buntin, M.B., M.F. Burke, M.C. Hoaglin and D. Blumenthal. 2011. "The Benefits of Health Information Technology: A Review of the Recent Literature Shows Predominantly Positive Results." Health Affairs 30(3): 464–71.

Callen, J., R. Paoloni, A. Georgiou, M. Prgomet and J. Westbrook. 2010. "The Rate of Missed Test Results in an Emergency Department: An Evaluation Using an Electronic Test Order and Results Viewing System." Methods of Information in Medicine 49(1): 37–43.

Canada Health Infoway. 2012. Vision 2015: Advancing Canada's Next Generation of Health Care. Ottawa: Author.

Casey, C., S. Senapati, C.B. White, L.D. Gruppen and M.M. Hammoud. 2005. "Medical Students' Self-Reported Work Hours: Perception versus Reality." American Journal of Obstetrics and Gynecology 193(5): 1780–84.

Cherry, D.K., E. Hing, D.A. Woodwell and E.A. Rechtsteiner. 2008. "National Ambulatory Medical Care Survey: 2006 Summary." National Health Statistics Reports 3(3): 1–39.

College of Family Physicians of Canada (CFPC), Canadian Medical Association (CMA) and Royal College of Physicians and Surgeons of Canada (RCPSC). 2012. 2007 National Physician Survey (NPS) Methodology and Comparability between the Total Eligible Physician Population, Survey Respondents and Non-Respondents. Can be accessed at: http://nationalphysiciansurvey.ca/surveys/2007-survey/2007-methodologies/

Devlin, R.A. and S. Sarma. 2008. "Do Physician Remuneration Schemes Matter? The Case of Canadian Family Physicians." Journal of Health Economics 27(5): 1168–81.

French, F., J. Andrew, M. Awramenko, H. Coutts, L. Leighton-Beck, J. Mollison et al. 2006. "Why Do Work Patterns Differ between Men and Women GPs?" Journal of Health Organization and Management 20(2–3): 163–72.

Friedberg, M.W., K.L. Coltin, S.D. Pearson, K.P. Kleinman, J. Zheng, J.A. Singer et al. 2007. "Does Affiliation of Physician Groups with One Another Produce Higher Quality Primary Care?" Journal of General Internal Medicine 22(10): 1385–92.

Halvorsen, R. and R. Palmquist. 1980. "The Interpretation of Dummy Variables in Semilogarithmic Equations." American Economic Review 70(3): 474–75.

Hillestad, R., J. Bigelow, A. Bower, F. Girosi, R. Meili, R. Scoville et al. 2005. "Can Electronic Medical Record Systems Transform Health Care? Potential Health Benefits, Savings and Costs." Health Affairs 24(5): 1103–17.

Hutten-Czapski, P., R. Pitblado and S. Slade. 2004. "Short Report: Scope of Family Practice in Rural and Urban Settings." Canadian Family Physician 50: 1548–50.

Macinko, J., B. Starfield and L. Shi. 2003. "The Contribution of Primary Care Systems to Health Outcomes within Organization for Economic Cooperation and Development (OECD) Countries, 1970–1998." Health Services Research 38(3): 831–65.

Macinko, J., B. Starfield and L. Shi. 2007. "Quantifying the Health Benefits of Primary Care Physician Supply in the United States." International Journal of Health Services 37(1): 111–26.

Mackie, A.S., L. Pilote, R. Ionescu-Ittu, E. Rahme and A.J. Marelli. 2007. "Health Care Resource Utilization in Adults with Congenital Heart Disease." American Journal of Cardiology 99(6): 839–43.

Magnus, M., J. Herwehe, L. Andrews, L. Gibson, N. Daigrepont, J.M. De Leon et al. 2009. "Evaluating Health Information Technology: Provider Satisfaction with an HIV-Specific, Electronic Clinical Management and Reporting System." AIDS Patient Care and STDs 23(2): 85–91.

Makoul, G., R.H. Curry and P.C. Tang. 2001. "The Use of Electronic Medical Records: Communication Patterns in Outpatient Encounters." Journal of the American Medical Informatics Association 8(6): 610–15.

Martin, C.M., C.L. Banwell, D.H. Broom and M. Nisa. 1999. "Consultation Length and Chronic Illness Care in General Practice: A Qualitative Study." Medical Journal of Australia 171(2): 77–81.

McInnes, D.K., D.C. Saltman and M.R. Kidd. 2006. "General Practitioners' Use of Computers for Prescribing and Electronic Health Records: Results from a National Survey." Medical Journal of Australia 185(2): 88–91.

Miller, R.H. and I. Sim. 2004. "Physicians' Use of Electronic Medical Records: Barriers and Solutions." Health Affairs 23(2): 116–26.

Miller, R.H., C. West, T.M. Brown, I. Sim and C. Ganchoff. 2005. "The Value of Electronic Health Records in Solo or Small Group Practices." Health Affairs 24(5): 1127–37.

Nabalamba, A. and W.J. Millar. 2007. "Going to the Doctor." Health Reports 18(1): 23–35.

Nilasena, D.S. and M.J. Lincoln. 1995. "A Computer-Generated Reminder System Improves Physician Compliance with Diabetes Preventive Care Guidelines." Proceedings of the Annual Symposium on Computer Application in Medical Care 280(23): 640–45.

Pearson, W.S., K. Bhat-Schelbert, E.S. Ford and A.H. Mokdad. 2009. "The Impact of Obesity on Time Spent with the Provider and Number of Medications Managed During Office-Based Physician Visits Using a Cross-Sectional, National Health Survey." BMC Public Health 9: 436.

Pizziferri, L., A.F. Kittler, L.A. Volk, M.M. Honour, S. Gupta, S. Wang et al. 2005. "Primary Care Physician Time Utilization Before and After Implementation of an Electronic Health Record: A Time-Motion Study." Journal of Biomedical Informatics 38(3): 176–88.

Pohar, S.L. and J.A. Johnson. 2007. "Health Care Utilization and Costs in Saskatchewan's Registered Indian Population with Diabetes." BMC Health Services Research 7: 126.

Poissant, L., J. Pereira, R. Tamblyn and Y. Kawasumi. 2005. "The Impact of Electronic Health Records on Time Efficiency of Physicians and Nurses: A Systematic Review." Journal of the American Medical Informatics Association 12(5): 505–16.

Sarma, S., R.A. Devlin, B. Belhadji and A. Thind. 2010a. "Does the Way Physicians Are Paid Influence the Way They Practice? The Case of Canadian Family Physicians' Work Activity." Health Policy 1 98(2–3): 203–17.

Sarma, S., R.A. Devlin and W. Hogg. 2010b. "Physicians' Production of Primary Care in Ontario, Canada." Health Economics 19(1): 14–30.

Sarma, S., A. Thind and M.-K. Chu. 2011. "Do New Cohorts of Family Physicians Work Less Compared to Their Older Predecessors? The Evidence from Canada." Social Science and Medicine 72(12): 2049–58.

Schoen, C., R. Osborn, M.M. Doty, D. Squires, J. Peugh and S. Applebaum. 2009. "A Survey of Primary Care Physicians in Eleven Countries, 2009: Perspectives on Care, Costs and Experiences." Health Affairs 28(6): w1171–83.

Shelton, R.C., E. Puleo, G.G. Bennett, L.H. McNeill, G. Sorensen and K.M. Emmons. 2009. "The Association between Racial and Gender Discrimination and Body Mass Index among Residents Living in Lower-Income Housing." Ethnicity and Disease 19(3): 251–57.

Starfield, B., L. Shi and J. Macinko. 2005. "Contribution of Primary Care to Health Systems and Health." Milbank Quarterly 83(3): 457–502.

Stock, J.H. and M.W. Watson. 2003. Introduction to Econometrics. New York: Addison Wesley.

Thomas, K.W., C.S. Dayton and M.W. Peterson. 1999. "Evaluation of Internet-Based Clinical Decision Support Systems." Journal of Medical Internet Research 1(2): E6.

Uettwiller-Geiger, D.L. 2005. "A Lab's Strategy to Reduce Errors Depends on Automation." Medical Laboratory Observer 37(12): 26, 28–29.

Van Den Berg, M.J., D.H. De Bakker, G.P. Westert, J. Van Der Zee and P.P. Groenewegen. 2009. "Do List Size and Remuneration Affect GPs' Decisions about How They Provide Consultations?" BMC Health Services Research 9(1): 39.

Van Der Kam, W.J., P.W. Moorman and M.J. Koppejan-Mulder. 2000. "Effects of Electronic Communication in General Practice." International Journal of Medical Informatics 60(1): 59–70.

Watson, D.E., P. Heppner, R. Reid, B. Bogdanovic and N. Roos. 2005. "Use of Physician Services by Older Adults: 1991/1992 to 2000/2001." Canadian Journal on Aging 24(Suppl. 1): 29–36.

Watson, D.E., S. Slade, L. Buske and J. Tepper. 2006. "Intergenerational Differences in Workloads among Primary Care Physicians: A Ten-Year, Population-Based Study." Health Affairs 25(6): 1620–28.

Wilson, A. and S. Childs. 2002. "The Relationship between Consultation Length, Process and Outcomes in General Practice: A Systematic Review." British Journal of General Practice 52(485): 1012–20.

Footnotes

1. Both versions of the 2007 NPS questionnaires are available online at http://nationalphysiciansurvey.ca/surveys/2007-survey/2007-results/. The 2010 NPS questionnaire is available at http://nationalphysiciansurvey.ca/surveys/2010-survey/2010-questionnaires/.

2. The detailed methodology regarding non-response bias adjustment through weights can be found at http://nationalphysiciansurvey.ca/wp-content/uploads/2012/05/2010-complete-methods-en.pdf.

Comments

Be the first to comment on this!

Note: Please enter a display name. Your email address will not be publically displayed