BRIEF REPORT
Patient-Centered Medical Home Implementation and Primary Care Provider Turnover Philip W. Sylling, MA,* Edwin S. Wong, PhD,*w Chuan-Fen Liu, MPH, PhD,*w Susan E. Hernandez, MPA,*w Adam J. Batten, BA,* Christian D. Helfrich, MPH, PhD,*w Karin Nelson, MD, MSHS,*zy Stephan D. Fihn, MD, MPH,wzy8 and Paul L. Hebert, PhD*w
Background: The Veterans Health Administration (VHA) began implementing a patient-centered medical home (PCMH) model of care delivery in April 2010 through its Patient Aligned Care Team (PACT) initiative. PACT represents a substantial system reengineering of VHA primary care and its potential effect on primary care provider (PCP) turnover is an important but unexplored relationship. This study examined the association between a system-wide PCMH implementation and PCP turnover. Methods: This was a retrospective, longitudinal study of VHA-employed PCPs spanning 29 calendar quarters before PACT and eight quarters of PACT implementation. PCP employment periods were identified from administrative data and turnover was defined by an indicator on the last quarter of each uncensored period. An interrupted time series model was used to estimate the association between PACT and turnover, adjusting for secular trend and seasonality, provider and job characteristics, and local unemployment. We calculated average marginal effects (AME), which reflected the change in turnover probability associated with PACT implementation. Results: The quarterly rate of PCP turnover was 3.06% before PACT and 3.38% after initiation of PACT. In adjusted analysis, PACT was associated with a modest increase in turnover (AME = 4.0 additional PCPs per 1000 PCPs per quarter, P = 0.004). Models with interaction terms suggested that the PACT-related change in turnover was increasing in provider age and experience. Conclusions: PACT was associated with a modest increase in PCP turnover, concentrated among older and more experienced providers, during initial implementation. Our findings suggest that From the *Northwest HSR&D Center for Innovation, VA Puget Sound Healthcare System; wDepartment of Health Services, School of Public Health, University of Washington; zGeneral Internal Medicine Service, VA Puget Sound Healthcare System; yDepartment of Medicine, School of Medicine, University of Washington; and 8VHA Office of Analytics and Business Intelligence, VA Puget Sound Healthcare System, Seattle, WA. Supported by the Veterans Health Administration Patient Centered Medical Home Demonstration Laboratory Coordination Center (XVA-61-041). A subset of these findings was presented in an oral presentation at the 2014 SGIM Annual Meeting, April 26, San Diego, CA, and in a poster presentation at the Academy Health Annual Research Meeting, June 8, San Diego, CA. The authors declare no conflict of interest. Reprints: Philip W. Sylling, MA, Northwest HSR&D Center for Innovation, VA Puget Sound Healthcare System, 1100 Olive Way, Suite 1400, Seattle, WA 98101. E-mail: [email protected]. Copyright r 2014 by Lippincott Williams & Wilkins ISSN: 0025-7079/14/5212-1017
Medical Care
Volume 52, Number 12, December 2014
policymakers should evaluate potential workforce effects when implementing PCMH. Key Words: primary care provider, turnover, patient-centered medical home, Veterans Health Administration (Med Care 2014;52: 1017–1022)
BACKGROUND The Veterans Health Administration (VHA) is one of the largest integrated health systems in the United States, providing care to over 5 million patients at >900 sites of care annually. In April 2010, the VHA began implementing a patient-centered medical home (PCMH) model of primary care through its Patient Aligned Care Team (PACT) initiative.1,2 PACT represents a substantial system reengineering of VHA primary care and its potential effect on primary care provider (PCP) turnover is an important but unexplored area of study. Key organizational change components of PACT include a transition to team-based care in which a PCP collaborates closely with other PACT team members (registered nurse care manager, licensed practical nurse, and administrative clerk) and an expansion of non-face-to-face visits (eg, telephone, email) to enhance access to care. In the behavioral science and management literature, organizational change has been associated with employee turnover3 and attitudes about change related to intention to quit.4,5 Qualitative studies of early PCMH initiatives suggested that the magnitude of organizational transition during implementation could induce change fatigue resulting in clinician turnover.6,7 Anecdotal evidence has also highlighted potential resistance to PCMH transition.8 This study examined the association between PACT implementation and PCP turnover. Although prior research has studied workforce outcomes including burnout and job satisfaction in relation to PCMH,9–12 no study to our knowledge has specifically examined whether PCMH is associated with turnover. Understanding this relationship is important because turnover directly affects patient-provider continuity, which is a core principle of PCMH,13,14 and also burdens health systems with high costs.15
METHODS Research Design and Sample This was a retrospective, longitudinal study at the PCP level spanning 29 calendar quarters before PACT (January www.lww-medicalcare.com |
1017
Medical Care
Sylling et al
Volume 52, Number 12, December 2014
PCP has assigned patients and VHA payroll earnings during any quarter from January 2003 –September 2012. Step 1: 13,739 PCPs, 242,463 observations Exclude residents and PCPs with intermittent schedules (1,777 PCPs) Step 2 : 11,962 PCPs, 218,417 observations
For 1,386 PCPs with simultaneous primary care positions, retain observations from the longest-held position. (Exclude 6,754 observations) Step 3: 11,962 PCPs, 211,663 observations Exclude PCPs with missing covariate data (588 PCPs) Step 4: 11,374 PCPs, 204,083 observations
To account for censoring of the dependent variable, exclude final two quarters (April 2012 –September 2012) from analysis. (11,317 observations)
Step 5: 11,180 PCPs, 192,766 observations
FIGURE 1. Flow diagram of sample construction.
2003–March 2010) and eight quarters of PACT implementation (April 2010–March 2012). For each quarter, we used administrative data to identify PCPs with assigned patients and VHA payroll earnings, which resulted in 13,739 PCPs employed during the study period. We excluded providers with temporary positions such as intermittent assignments and medical residents (N = 1777). For PCPs with >1 primary care position during the same quarter (N = 1386), we retained observations from the longest held position. Remaining providers with missing covariate data were excluded (N = 588). To account for censoring of the dependent variable (see below), we collected data up to September 2012 but excluded the final 2 quarters (April 2012–September 2012) from regression analyses. The final sample consisted of 11,180 PCPs employed in 759 sites of care that were either affiliated with or located in 1 of 129 VHA medical centers (Fig. 1).
Turnover Measure For each quarter, we identified all providers present in our data and defined a dichotomous measure of turnover indicating absence from our data for at least 2 consecutive quarters directly following the quarter under consideration, which was defined as the end of the PCP’s employment period. Thus, the turnover variable was coded 1 in the last quarter of the period and coded zero for all prior quarters of
1018 | www.lww-medicalcare.com
the period. To allow leaves of absence, a one-quarter absence was treated as a missing observation and the employment period continued in the following quarter. To account for censoring, we collected data through September 2012, identified turnover until March 2012, and excluded observations from April 2012–September 2012 from analyses. Our sample of 11,180 PCPs recorded 11,654 employment periods due to some providers’ turnover and sample reentry. Transfers between primary care positions at different VHA facilities, without a two-quarter absence, were not counted as turnover but transfers to nonprimary care VHA positions were considered turnover.
Explanatory Variables The explanatory variable of interest was a PACT indicator variable coded 1 for all observations beginning with the quarter starting April 2010 and coded zero for all prior quarters. We adjusted for the secular trend in turnover with a linear time variable (integer values 1–37) and for seasonality with quarter-of-year indicators. Following conceptual models of employee turnover,16,17 we adjusted for PCP-level covariates that are transferable between positions, job-level covariates that are characteristics of VHA employment, and environment variables that reflect opportunity costs of VHA employment. PCP-level covariates included sex, profession [Physician (MD), Nurse Practitioner r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 12, December 2014
PCMH Implementation and PCP Turnover
(NP), or Physician Assistant (PA)], age (categorized as under 45, 45 to 55, and over 55), and years of VHA experience. Joblevel covariates included full-time equivalent (FTE) salary rate, appointment class (0 = Career or Career Conditional, 1 = Term), full-time or part-time VHA employee, type of facility (hospital-based medical center or community-based outpatient clinic [CBOC]), and fixed effects for the 129 VHA medical centers with which each facility is affiliated. Term appointment duration is expected to exceed 1 year but not exceed 4 years, whereas Career or Career Conditional appointments have no duration limitations. Full-time employees may regularly divide worktime between primary care and other functions, such as administration. However, we did not observe PCPs’ proportion of FTE devoted to primary care. We accounted for opportunity costs of VHA employment in 2 ways. First, we inflation-adjusted salary rates to 2003 dollars using the annual, site-level Medicare wage index. Second, we adjusted for the overall unemployment rate in providers’ VHA market areas to account for outside-VHA job opportunities. The VHA delineates 81 market areas for planning purposes based on the geographical distribution of Veterans, location of VHA and non-VHA facilities, state and county borders, geographical barriers, and travel time to
VHA sites. Using monthly, county-level unemployment rates, we averaged the 3 observations within each countyquarter and then calculated the population-weighted average unemployment rate for each VHA market area’s constituent counties in the quarter.
Data The VHA’s Corporate Data Warehouse (CDW) and Personnel and Accounting Integrated Data (PAID) were used to identify PCPs and obtain provider-level and job-level covariates. County-level unemployment rates were obtained from the Bureau of Labor Statistics18 and the VHA’s Health Economics Resource Center,19 provided the Medicare wage index matched with VHA sites.
Statistical Analysis The dependent variable (0 = no turnover, 1 = turnover) was modeled using logistic regression, as performed in prior studies.20,21 Our analysis represented an interrupted time series (ITS) in which the PACT indicator variable captured the discrete change in turnover following PACT implementation after accounting for a constant secular trend during the study period.22 Changes in turnover over time have also been examined using
TABLE 1. Characteristics of Primary Care Providers, Local Unemployment Rates, and Quarterly Turnover Before and After Initiation of PACT
Sample size (N) Observations Unique providers Provider characteristics Female (%) Profession (%) Physician Nurse practitioner Physician assistant Age group (%) Under 45 45–55 > 55 VHA experience [mean (SD)] (y) Job characteristics Salary [mean (SD)] (in 2003 dollars) Position type (%) Career or career conditional (vs. term) Full-time employee (vs. part-time) Clinic type (%) Hospital based (VHA Medical Center) Community-based outpatient clinic Local unemployment rate Percentage points [mean (SD)] Quarterly turnover rate (%) Overall Physician Nurse Practitioner Physician assistant
Pre-PACT (%)
PACT (%)
January 2003–March 2010 (29 Quarters)w
April 2010–March 2012 (8 Quarters)w
147,225 9794
45,541 7094
50.85
54.00
0.001
69.26 20.83 9.91
69.65 21.30 9.05
0.636 0.528 0.111
29.70 39.95 30.35 8.94 (7.49)
22.78 35.32 41.91 9.40 (7.87)
< 0.001 < 0.001 < 0.001 0.001
$101,622 ($15,913)
$108,702 ($14,936)
< 0.001
88.58 87.55
93.51 88.90
< 0.001 0.015
59.43 40.57
52.86 47.14
< 0.001 < 0.001
6.12 (2.13)
9.18 (1.82)
< 0.001
3.06 2.86 3.34 3.91
3.38 3.26 3.56 3.93
P*
0.001 0.001 0.333 0.938
P-values determined from 1000 iterations of a block bootstrap process. *Test of equality between the respective pre-PACT and PACT periods. w Statistics are calculated across all provider-quarter observations. PACT indicates Patient Aligned Care Team; VHA, Veterans Health Administration.
r
2014 Lippincott Williams & Wilkins
www.lww-medicalcare.com |
1019
Medical Care
Sylling et al
TABLE 2. Adjusted Analysis From Logistic Regression Average Marginal Effectw
Predictor Variablesy PACT indicator Linear time trend Quarter of year January–March April–June July–September October–December Female Profession Physician Nurse practitioner Physician assistant Age group Under 45 45–55 > 55 VHA experience (per year) Salary ratez (per $10,000) Term position (vs. career or career conditional) Part-time employee (vs. full-time) CBOC (vs. VHA Medical Center) Local unemployment rate Observations
SE
0.004** 0.0002**
0.0014 0.0001
0.0035** 0.001 0.0062** Reference 0.0027**
0.0012 0.0012 0.0011 0.001
Reference 0.0055** 0.0084**
0.0014 0.0016
Reference 0.0049** 0.0055** 0.0005** 0.0011* 0.0108**
0.0011 0.0012 0.0001 0.0004 0.0014
0.0073** 0.0062** 0.0011** 192,766
0.0014 0.001 0.0003
*, **Significantly different from zero at the 0.05 and 0.01 levels, respectively. w Marginal effects reflect the change in the probability of PCP turnover associated with a unit increase in an explanatory variable. z Annual salary rate in tens of thousands of inflation-adjusted 2003 dollars. y Model includes fixed effects for the VHA stations (n = 129) with which each site of care is affiliated. CBOC indicates community-based outpatient clinic; PACT, Patient Aligned Care Team; VHA, Veterans Health Administration.
time-period dummy variables.23 Following estimation of our base model, which included all explanatory variables described above, we examined differences in the association between PACT implementation and turnover across PCP characteristics by interacting the PACT indicator with provider’s sex, profession (MD, NP, or PA), age, and experience. To report results, we calculated average marginal effects (AME), which reflected the change in turnover probability associated with a unit increase in an explanatory variable. For models with interactions, AMEs were calculated following Karaca-Mandic et al.24 PCPlevel cluster-robust standard errors were calculated using the Huber-White estimator. Statistical models were fit using the STATA Statistical Software (Version 13, College Station, TX).
RESULTS Descriptive statistics (Table 1) indicated that, before PACT, PCPs were evenly split between men and women, largely middle aged, and mostly MDs. In the PACT implementation period, providers were more likely to be female (P = 0.001), older (P < 0.001), and more experienced (P = 0.001). Inflation-adjusted salary (P < 0.001), local unemployment (P < 0.001), and the quarterly turnover rate (3.38% vs. 3.06%, P = 0.001) were also higher after PACT was initiated.
1020 | www.lww-medicalcare.com
Volume 52, Number 12, December 2014
In adjusted analysis (Table 2), PACT implementation was associated with a modest but statistically significant increase in turnover (AME = 0.004, P = 0.004). This implies turnover of 4.0 additional PCPs per 1000 PCPs per quarter after initiation of PACT. Turnover also exhibited an upward secular trend (constant change each quarter) over the study period (AME = 0.0002, P = 0.002). PCP-level, job-level, and environment variables were associated with turnover. Female providers were less likely to leave VHA primary care than males (AME = 0.0027, P = 0.005). Turnover was higher for NPs (AME = 0.0055, P < 0.001) and PAs (AME = 0.0084, P < 0.001) compared with MDs. Turnover exhibited a U-shaped relationship with respect to provider age, being lowest in the 45–55 age group. An additional $10,000 in salary rate (AME = 0.0011, P = 0.011) or 1 year of experience (AME = 0.0005, P < 0.001) were each associated with lower turnover. Providers were less likely to leave CBOC positions than those in VHA medical centers (AME = 0.0062, P < 0.001) and turnover was inversely related to local unemployment (AME = 0.0011, P < 0.001). In interaction analyses (Table 3), there were no significant differences in the association between PACT implementation and turnover across PCP sex or profession. However, compared with PCPs under age 45, the association between PACT and turnover was significantly greater for PCPs aged 45–55 (P = 0.023) and for PCPs over age 55 (P = 0.003). Similarly, the change in turnover associated with PACT was increasing in VHA experience. For example, the AME of PACT was greater at 10 years of experience relative to 5 years of experience (P < 0.001).
TABLE 3. Average Marginal Effects (AME) of PACT for Selected PCP Characteristic Groups Interaction With PACT Indicatorz Provider sex Male Female Profession Physician Nurse practitioner Physician assistant Age group Under 45 45–55 > 55 VHA Experience (y) 5 10 15 20 No. observations in all models
AMEw of PACT
SE
0.0061** 0.0024
0.0019 0.0017
0.0048** 0.003 0.0014
0.0016 0.0026 0.0036
0.0008 0.0046*,y 0.0069**,y
0.0021 0.0018 0.0022
0.0019 0.0051**,8 0.008**,8 0.0106**,8 192,766
0.0016 0.0015 0.0017 0.0021
*, **Significantly different from zero at the 0.05 and 0.01 levels, respectively. w Average marginal effects reflect the change in the probability of PCP turnover associated with the PACT indicator variable for specified subgroups. z Models included all covariates from model in Table 2. y Significantly different from under 45 age group. 8 Significantly different from next lowest experience level. PACT indicates Patient Aligned Care Team; PCP, primary care provider; VHA, Veterans Health Administration.
r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 12, December 2014
DISCUSSION At the system-wide level, we found that PACT implementation was associated with a modest increase in turnover of 4.0 additional PCPs per 1000 PCPs per quarter. For reference, average quarterly turnover was 30.6 PCPs per 1000 PCPs before PACT. Assuming a VHA workforce of 6000 providers, this would represent a loss of 192 additional PCPs during the PACT implementation period we studied (April 2010–March 2012), or about 3% of the workforce. On an annualized basis, the turnover rate is similar to clinician turnover in non-VHA settings.25–27 Interaction analyses showed that the increased turnover was concentrated among older and more experienced PCPs. We are unaware of other studies of age-related changes in turnover among the health care workforce, but changes in organizational culture have been related to higher turnover among older employees in the technology industry.3 In contrast, we found that younger, less experienced providers did not leave primary care at a higher rate during the beginning of PACT implementation. We found other predictors of turnover. There was higher turnover among NPs and PAs compared with MDs. We are unaware of prior studies that have compared turnover across PCP professions, although, in the related area of job satisfaction, NPs and PAs have reported relatively high satisfaction.28,29 Consistent with previous findings,30 we found that turnover was inversely related to the unemployment rate in providers’ market areas. This study extends the literature on the relationship between PCMH implementation and health care workforce outcomes by measuring changes in PCP turnover during a large PCMH implementation. Our findings also contribute to the turnover literature by examining PCP-level factors of turnover using an extensive, administrative database. There are several limitations to this study. First, because PACT was a system-wide initiative, our study lacked a control group of VHA sites that did not implement PACT. To address this limitation, we applied an ITS model to account for the secular trend in turnover over a 9-year period. Second, we could not observe whether site-level PACT implementation began at a later date than the April 2010, system-wide date. Third, we could not discriminate between voluntary and involuntary turnover. However, terminations in VHA are rare and we have no reason to believe that PACT would affect involuntary terminations. Fourth, we used the overall unemployment rate as a proxy for opportunity costs of VHA employment rather than statistics specific to health care providers because only the former were reliably available at the market level on a quarterly basis. Fifth, we did not account for provider panel sizes because we did not have accurate historical data on the proportion of PCP-level FTE devoted to primary care, which is a determinant of VHA panel size. Finally, our implementation period is limited to eight quarters, whereas the PACT initiative is still evolving and may not be completely implemented in all sites. Thus, our results may only reflect a short-term effect. To summarize, we found a modest increase in PCP turnover during the first 2 years of implementation of a PCMH and found that changes in turnover were increasing in provider age r
2014 Lippincott Williams & Wilkins
PCMH Implementation and PCP Turnover
and experience. These findings provide useful insights for policymakers considering potential workforce effects when evaluating the overall costs and benefits of implementing PCMH.
REFERENCES 1. Rosland A, Nelson K, Sun H, et al. The patient-centered medical home in the Veterans Health Administration. Am J Manag Care. 2013; 19:263–272. 2. Klein S. The Veterans Health Administration: implementing patientcentered medical homes in the nation’s largest integrated delivery system. Commonwealth Fund, 2011. Available at: http://www.commonwealthfund. org/B/media/Files/Publications/case%20Study/2011/Sep/1537_Klein_ veterans_hlt_admin_case%20study.pdf. Accessed July, 2013. 3. Baron JN, Hannan MT, Burton MD. Labor pains: change in organizational models and employee turnover in young, high tech firms. Am J Sociol. 2001;106:960–1012. 4. Wanberg CR, Banas JT. Predictors and outcomes of openness to changes in a reorganizing workplace. J Appl Psychol. 2000;85:132–142. 5. Rafferty AE, Griffin MA. Perceptions of organizational change: a stress and coping perspective. J Appl Psychol. 2006;91:1154–1162. 6. Nutting PA, Miller WL, Crabtree BF, et al. Initial lessons from the first national demonstration project on practice transformation to a patientcentered medical home. Ann Fam Med. 2009;7:254–260. 7. Nutting PA, Crabtree BF, Miller WL, et al. Journey to the patientcentered medical home: a qualitative analysis of the experiences of practices in the National Demonstration Project. Ann Fam Med. 2010;8(suppl 1):S45–S56. S92. 8. Meyer H. Report from the field. Group Health’s move to the medical home: for doctors, it’s often a hard journey. Health Aff (Millwood). 2010;29:844–851. 9. Reid RJ, Coleman K, Johnson EA, et al. The Group Health medical home at year two: cost savings, higher patient satisfaction, and less burnout for providers. Health Aff (Millwood). 2010;29:835–843. 10. Reid RJ, Fishman PA, Yu O, et al. Patient-centered medical home demonstration: a prospective, quasi-experimental, before and after evaluation. Am J Manag Care. 2009;15:e71–e87. 11. Lewis SE, Nocon RS, Tang H, et al. Patient-centered medical home characteristics and staff morale in safety net clinics. Arch Intern Med. 2012;172:23–31. 12. Helfrich C, Dolan ED, Simonetti J, et al. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. J Gen Intern Med. 2014;29: 659–666. 13. Patient-Centered Primary Care Collaborative (PCPCC). Joint principles of the patient-centered medical home. Available at: http://www.aafp. org/dam/AAFP/documents/practice_management/pcmh/initiatives/ PCMHJoint.pdf. Accessed July, 2013. 14. Ferrante JM, Balasubramanian BA, Hudson SV, et al. Principles of the patient-centered medical home and preventive services delivery. Ann Fam Med. 2010;8:108–116. 15. Buchbinder SB, Wilson M, Melick CF, et al. Estimates of costs of primary care physician turnover. Am J Manag Care. 1999;5: 1431–1438. 16. Farrell D, Rusbult CE. Exchange variables as predictors of job satisfaction, job commitment, and turnover: The impact of rewards, costs, alternatives, and investments. Organ Behav Hum Perform. 1981;28:78–95. 17. Iverson RD. An event history analysis of employee turnover: the case of hospital employees in Australia. Hum Resource Manag Rev. 2000; 9:397–418. 18. Bureau of Labor Statistics. Local area unemployment statistics. Available at: http://www.bls.gov/lau/data.htm. Accessed July, 2013. 19. Health Economics Resource Center. Medicare wage index for VA facilities: 2000-2013. Available at: http://www.herc.research.va.gov/ resources/faq_l02.asp. Accessed July, 2013. 20. Sjo¨berg A, Sverke M. The interactive effect of job involvement and organizational commitment on job turnover revisited: a note on the mediating role of turnover intention. Scand J Psychol. 2000;41: 247–252.
www.lww-medicalcare.com |
1021
Sylling et al
21. Chen H-C, Chu C-I, Wang Y-H, et al. Turnover factors revisited: a longitudinal study of Taiwan-based staff nurses. Int J Nurs Stud. 2008;45:277–285. 22. Ramsay CR, Matowe L, Grilli R, et al. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int J Technol Assess Health Care. 2003;19: 613–623. 23. Light A, Ureta M. Panel estimates of male and female job turnover behavior: can female nonquitters be identified? J Labor Econ. 1992; 10:156–181. 24. Karaca-Mandic P, Norton EC, Dowd B. Interaction terms in nonlinear models. Health Serv Res. 2012;47(pt 1):255–274. 25. Buchbinder SB, Wilson M, Melick CF, et al. Primary care physician job satisfaction and turnover. Am J Manag Care. 2001;7:701–713.
1022 | www.lww-medicalcare.com
Medical Care
Volume 52, Number 12, December 2014
26. Misra-Hebert AD, Kay R, Stoller JK. A review of physician turnover: rates, causes, and consequences. Am J Med Qual. 2004;19:56–66. 27. Plomondon ME, Magid DJ, Steiner JF, et al. Primary care provider turnover and quality in managed care organizations. Am J Manag Care. 2007;13:465–472. 28. Kacel B, Millar M, Norris D. Measurement of nurse practitioner job satisfaction in a Midwestern state. J Am Acad Nurse Pract. 2005; 17:27–32. 29. Freeborn DK, Hooker RS. Satisfaction of physician assistants and other nonphysician providers in a managed care setting. Public Health Rep. 1995;110:714–719. 30. Carsten JM, Spector PE. Unemployment, job satisfaction, and employee turnover: a meta-analytic test of the Muchinsky model. J Appl Psychol. 1987;72:3.
r
2014 Lippincott Williams & Wilkins
Patient-Centered Medical Home Implementation and Primary Care Provider Turnover Philip W. Sylling, MA,* Edwin S. Wong, PhD,*w Chuan-Fen Liu, MPH, PhD,*w Susan E. Hernandez, MPA,*w Adam J. Batten, BA,* Christian D. Helfrich, MPH, PhD,*w Karin Nelson, MD, MSHS,*zy Stephan D. Fihn, MD, MPH,wzy8 and Paul L. Hebert, PhD*w
Background: The Veterans Health Administration (VHA) began implementing a patient-centered medical home (PCMH) model of care delivery in April 2010 through its Patient Aligned Care Team (PACT) initiative. PACT represents a substantial system reengineering of VHA primary care and its potential effect on primary care provider (PCP) turnover is an important but unexplored relationship. This study examined the association between a system-wide PCMH implementation and PCP turnover. Methods: This was a retrospective, longitudinal study of VHA-employed PCPs spanning 29 calendar quarters before PACT and eight quarters of PACT implementation. PCP employment periods were identified from administrative data and turnover was defined by an indicator on the last quarter of each uncensored period. An interrupted time series model was used to estimate the association between PACT and turnover, adjusting for secular trend and seasonality, provider and job characteristics, and local unemployment. We calculated average marginal effects (AME), which reflected the change in turnover probability associated with PACT implementation. Results: The quarterly rate of PCP turnover was 3.06% before PACT and 3.38% after initiation of PACT. In adjusted analysis, PACT was associated with a modest increase in turnover (AME = 4.0 additional PCPs per 1000 PCPs per quarter, P = 0.004). Models with interaction terms suggested that the PACT-related change in turnover was increasing in provider age and experience. Conclusions: PACT was associated with a modest increase in PCP turnover, concentrated among older and more experienced providers, during initial implementation. Our findings suggest that From the *Northwest HSR&D Center for Innovation, VA Puget Sound Healthcare System; wDepartment of Health Services, School of Public Health, University of Washington; zGeneral Internal Medicine Service, VA Puget Sound Healthcare System; yDepartment of Medicine, School of Medicine, University of Washington; and 8VHA Office of Analytics and Business Intelligence, VA Puget Sound Healthcare System, Seattle, WA. Supported by the Veterans Health Administration Patient Centered Medical Home Demonstration Laboratory Coordination Center (XVA-61-041). A subset of these findings was presented in an oral presentation at the 2014 SGIM Annual Meeting, April 26, San Diego, CA, and in a poster presentation at the Academy Health Annual Research Meeting, June 8, San Diego, CA. The authors declare no conflict of interest. Reprints: Philip W. Sylling, MA, Northwest HSR&D Center for Innovation, VA Puget Sound Healthcare System, 1100 Olive Way, Suite 1400, Seattle, WA 98101. E-mail: [email protected]. Copyright r 2014 by Lippincott Williams & Wilkins ISSN: 0025-7079/14/5212-1017
Medical Care
Volume 52, Number 12, December 2014
policymakers should evaluate potential workforce effects when implementing PCMH. Key Words: primary care provider, turnover, patient-centered medical home, Veterans Health Administration (Med Care 2014;52: 1017–1022)
BACKGROUND The Veterans Health Administration (VHA) is one of the largest integrated health systems in the United States, providing care to over 5 million patients at >900 sites of care annually. In April 2010, the VHA began implementing a patient-centered medical home (PCMH) model of primary care through its Patient Aligned Care Team (PACT) initiative.1,2 PACT represents a substantial system reengineering of VHA primary care and its potential effect on primary care provider (PCP) turnover is an important but unexplored area of study. Key organizational change components of PACT include a transition to team-based care in which a PCP collaborates closely with other PACT team members (registered nurse care manager, licensed practical nurse, and administrative clerk) and an expansion of non-face-to-face visits (eg, telephone, email) to enhance access to care. In the behavioral science and management literature, organizational change has been associated with employee turnover3 and attitudes about change related to intention to quit.4,5 Qualitative studies of early PCMH initiatives suggested that the magnitude of organizational transition during implementation could induce change fatigue resulting in clinician turnover.6,7 Anecdotal evidence has also highlighted potential resistance to PCMH transition.8 This study examined the association between PACT implementation and PCP turnover. Although prior research has studied workforce outcomes including burnout and job satisfaction in relation to PCMH,9–12 no study to our knowledge has specifically examined whether PCMH is associated with turnover. Understanding this relationship is important because turnover directly affects patient-provider continuity, which is a core principle of PCMH,13,14 and also burdens health systems with high costs.15
METHODS Research Design and Sample This was a retrospective, longitudinal study at the PCP level spanning 29 calendar quarters before PACT (January www.lww-medicalcare.com |
1017
Medical Care
Sylling et al
Volume 52, Number 12, December 2014
PCP has assigned patients and VHA payroll earnings during any quarter from January 2003 –September 2012. Step 1: 13,739 PCPs, 242,463 observations Exclude residents and PCPs with intermittent schedules (1,777 PCPs) Step 2 : 11,962 PCPs, 218,417 observations
For 1,386 PCPs with simultaneous primary care positions, retain observations from the longest-held position. (Exclude 6,754 observations) Step 3: 11,962 PCPs, 211,663 observations Exclude PCPs with missing covariate data (588 PCPs) Step 4: 11,374 PCPs, 204,083 observations
To account for censoring of the dependent variable, exclude final two quarters (April 2012 –September 2012) from analysis. (11,317 observations)
Step 5: 11,180 PCPs, 192,766 observations
FIGURE 1. Flow diagram of sample construction.
2003–March 2010) and eight quarters of PACT implementation (April 2010–March 2012). For each quarter, we used administrative data to identify PCPs with assigned patients and VHA payroll earnings, which resulted in 13,739 PCPs employed during the study period. We excluded providers with temporary positions such as intermittent assignments and medical residents (N = 1777). For PCPs with >1 primary care position during the same quarter (N = 1386), we retained observations from the longest held position. Remaining providers with missing covariate data were excluded (N = 588). To account for censoring of the dependent variable (see below), we collected data up to September 2012 but excluded the final 2 quarters (April 2012–September 2012) from regression analyses. The final sample consisted of 11,180 PCPs employed in 759 sites of care that were either affiliated with or located in 1 of 129 VHA medical centers (Fig. 1).
Turnover Measure For each quarter, we identified all providers present in our data and defined a dichotomous measure of turnover indicating absence from our data for at least 2 consecutive quarters directly following the quarter under consideration, which was defined as the end of the PCP’s employment period. Thus, the turnover variable was coded 1 in the last quarter of the period and coded zero for all prior quarters of
1018 | www.lww-medicalcare.com
the period. To allow leaves of absence, a one-quarter absence was treated as a missing observation and the employment period continued in the following quarter. To account for censoring, we collected data through September 2012, identified turnover until March 2012, and excluded observations from April 2012–September 2012 from analyses. Our sample of 11,180 PCPs recorded 11,654 employment periods due to some providers’ turnover and sample reentry. Transfers between primary care positions at different VHA facilities, without a two-quarter absence, were not counted as turnover but transfers to nonprimary care VHA positions were considered turnover.
Explanatory Variables The explanatory variable of interest was a PACT indicator variable coded 1 for all observations beginning with the quarter starting April 2010 and coded zero for all prior quarters. We adjusted for the secular trend in turnover with a linear time variable (integer values 1–37) and for seasonality with quarter-of-year indicators. Following conceptual models of employee turnover,16,17 we adjusted for PCP-level covariates that are transferable between positions, job-level covariates that are characteristics of VHA employment, and environment variables that reflect opportunity costs of VHA employment. PCP-level covariates included sex, profession [Physician (MD), Nurse Practitioner r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 12, December 2014
PCMH Implementation and PCP Turnover
(NP), or Physician Assistant (PA)], age (categorized as under 45, 45 to 55, and over 55), and years of VHA experience. Joblevel covariates included full-time equivalent (FTE) salary rate, appointment class (0 = Career or Career Conditional, 1 = Term), full-time or part-time VHA employee, type of facility (hospital-based medical center or community-based outpatient clinic [CBOC]), and fixed effects for the 129 VHA medical centers with which each facility is affiliated. Term appointment duration is expected to exceed 1 year but not exceed 4 years, whereas Career or Career Conditional appointments have no duration limitations. Full-time employees may regularly divide worktime between primary care and other functions, such as administration. However, we did not observe PCPs’ proportion of FTE devoted to primary care. We accounted for opportunity costs of VHA employment in 2 ways. First, we inflation-adjusted salary rates to 2003 dollars using the annual, site-level Medicare wage index. Second, we adjusted for the overall unemployment rate in providers’ VHA market areas to account for outside-VHA job opportunities. The VHA delineates 81 market areas for planning purposes based on the geographical distribution of Veterans, location of VHA and non-VHA facilities, state and county borders, geographical barriers, and travel time to
VHA sites. Using monthly, county-level unemployment rates, we averaged the 3 observations within each countyquarter and then calculated the population-weighted average unemployment rate for each VHA market area’s constituent counties in the quarter.
Data The VHA’s Corporate Data Warehouse (CDW) and Personnel and Accounting Integrated Data (PAID) were used to identify PCPs and obtain provider-level and job-level covariates. County-level unemployment rates were obtained from the Bureau of Labor Statistics18 and the VHA’s Health Economics Resource Center,19 provided the Medicare wage index matched with VHA sites.
Statistical Analysis The dependent variable (0 = no turnover, 1 = turnover) was modeled using logistic regression, as performed in prior studies.20,21 Our analysis represented an interrupted time series (ITS) in which the PACT indicator variable captured the discrete change in turnover following PACT implementation after accounting for a constant secular trend during the study period.22 Changes in turnover over time have also been examined using
TABLE 1. Characteristics of Primary Care Providers, Local Unemployment Rates, and Quarterly Turnover Before and After Initiation of PACT
Sample size (N) Observations Unique providers Provider characteristics Female (%) Profession (%) Physician Nurse practitioner Physician assistant Age group (%) Under 45 45–55 > 55 VHA experience [mean (SD)] (y) Job characteristics Salary [mean (SD)] (in 2003 dollars) Position type (%) Career or career conditional (vs. term) Full-time employee (vs. part-time) Clinic type (%) Hospital based (VHA Medical Center) Community-based outpatient clinic Local unemployment rate Percentage points [mean (SD)] Quarterly turnover rate (%) Overall Physician Nurse Practitioner Physician assistant
Pre-PACT (%)
PACT (%)
January 2003–March 2010 (29 Quarters)w
April 2010–March 2012 (8 Quarters)w
147,225 9794
45,541 7094
50.85
54.00
0.001
69.26 20.83 9.91
69.65 21.30 9.05
0.636 0.528 0.111
29.70 39.95 30.35 8.94 (7.49)
22.78 35.32 41.91 9.40 (7.87)
< 0.001 < 0.001 < 0.001 0.001
$101,622 ($15,913)
$108,702 ($14,936)
< 0.001
88.58 87.55
93.51 88.90
< 0.001 0.015
59.43 40.57
52.86 47.14
< 0.001 < 0.001
6.12 (2.13)
9.18 (1.82)
< 0.001
3.06 2.86 3.34 3.91
3.38 3.26 3.56 3.93
P*
0.001 0.001 0.333 0.938
P-values determined from 1000 iterations of a block bootstrap process. *Test of equality between the respective pre-PACT and PACT periods. w Statistics are calculated across all provider-quarter observations. PACT indicates Patient Aligned Care Team; VHA, Veterans Health Administration.
r
2014 Lippincott Williams & Wilkins
www.lww-medicalcare.com |
1019
Medical Care
Sylling et al
TABLE 2. Adjusted Analysis From Logistic Regression Average Marginal Effectw
Predictor Variablesy PACT indicator Linear time trend Quarter of year January–March April–June July–September October–December Female Profession Physician Nurse practitioner Physician assistant Age group Under 45 45–55 > 55 VHA experience (per year) Salary ratez (per $10,000) Term position (vs. career or career conditional) Part-time employee (vs. full-time) CBOC (vs. VHA Medical Center) Local unemployment rate Observations
SE
0.004** 0.0002**
0.0014 0.0001
0.0035** 0.001 0.0062** Reference 0.0027**
0.0012 0.0012 0.0011 0.001
Reference 0.0055** 0.0084**
0.0014 0.0016
Reference 0.0049** 0.0055** 0.0005** 0.0011* 0.0108**
0.0011 0.0012 0.0001 0.0004 0.0014
0.0073** 0.0062** 0.0011** 192,766
0.0014 0.001 0.0003
*, **Significantly different from zero at the 0.05 and 0.01 levels, respectively. w Marginal effects reflect the change in the probability of PCP turnover associated with a unit increase in an explanatory variable. z Annual salary rate in tens of thousands of inflation-adjusted 2003 dollars. y Model includes fixed effects for the VHA stations (n = 129) with which each site of care is affiliated. CBOC indicates community-based outpatient clinic; PACT, Patient Aligned Care Team; VHA, Veterans Health Administration.
time-period dummy variables.23 Following estimation of our base model, which included all explanatory variables described above, we examined differences in the association between PACT implementation and turnover across PCP characteristics by interacting the PACT indicator with provider’s sex, profession (MD, NP, or PA), age, and experience. To report results, we calculated average marginal effects (AME), which reflected the change in turnover probability associated with a unit increase in an explanatory variable. For models with interactions, AMEs were calculated following Karaca-Mandic et al.24 PCPlevel cluster-robust standard errors were calculated using the Huber-White estimator. Statistical models were fit using the STATA Statistical Software (Version 13, College Station, TX).
RESULTS Descriptive statistics (Table 1) indicated that, before PACT, PCPs were evenly split between men and women, largely middle aged, and mostly MDs. In the PACT implementation period, providers were more likely to be female (P = 0.001), older (P < 0.001), and more experienced (P = 0.001). Inflation-adjusted salary (P < 0.001), local unemployment (P < 0.001), and the quarterly turnover rate (3.38% vs. 3.06%, P = 0.001) were also higher after PACT was initiated.
1020 | www.lww-medicalcare.com
Volume 52, Number 12, December 2014
In adjusted analysis (Table 2), PACT implementation was associated with a modest but statistically significant increase in turnover (AME = 0.004, P = 0.004). This implies turnover of 4.0 additional PCPs per 1000 PCPs per quarter after initiation of PACT. Turnover also exhibited an upward secular trend (constant change each quarter) over the study period (AME = 0.0002, P = 0.002). PCP-level, job-level, and environment variables were associated with turnover. Female providers were less likely to leave VHA primary care than males (AME = 0.0027, P = 0.005). Turnover was higher for NPs (AME = 0.0055, P < 0.001) and PAs (AME = 0.0084, P < 0.001) compared with MDs. Turnover exhibited a U-shaped relationship with respect to provider age, being lowest in the 45–55 age group. An additional $10,000 in salary rate (AME = 0.0011, P = 0.011) or 1 year of experience (AME = 0.0005, P < 0.001) were each associated with lower turnover. Providers were less likely to leave CBOC positions than those in VHA medical centers (AME = 0.0062, P < 0.001) and turnover was inversely related to local unemployment (AME = 0.0011, P < 0.001). In interaction analyses (Table 3), there were no significant differences in the association between PACT implementation and turnover across PCP sex or profession. However, compared with PCPs under age 45, the association between PACT and turnover was significantly greater for PCPs aged 45–55 (P = 0.023) and for PCPs over age 55 (P = 0.003). Similarly, the change in turnover associated with PACT was increasing in VHA experience. For example, the AME of PACT was greater at 10 years of experience relative to 5 years of experience (P < 0.001).
TABLE 3. Average Marginal Effects (AME) of PACT for Selected PCP Characteristic Groups Interaction With PACT Indicatorz Provider sex Male Female Profession Physician Nurse practitioner Physician assistant Age group Under 45 45–55 > 55 VHA Experience (y) 5 10 15 20 No. observations in all models
AMEw of PACT
SE
0.0061** 0.0024
0.0019 0.0017
0.0048** 0.003 0.0014
0.0016 0.0026 0.0036
0.0008 0.0046*,y 0.0069**,y
0.0021 0.0018 0.0022
0.0019 0.0051**,8 0.008**,8 0.0106**,8 192,766
0.0016 0.0015 0.0017 0.0021
*, **Significantly different from zero at the 0.05 and 0.01 levels, respectively. w Average marginal effects reflect the change in the probability of PCP turnover associated with the PACT indicator variable for specified subgroups. z Models included all covariates from model in Table 2. y Significantly different from under 45 age group. 8 Significantly different from next lowest experience level. PACT indicates Patient Aligned Care Team; PCP, primary care provider; VHA, Veterans Health Administration.
r
2014 Lippincott Williams & Wilkins
Medical Care
Volume 52, Number 12, December 2014
DISCUSSION At the system-wide level, we found that PACT implementation was associated with a modest increase in turnover of 4.0 additional PCPs per 1000 PCPs per quarter. For reference, average quarterly turnover was 30.6 PCPs per 1000 PCPs before PACT. Assuming a VHA workforce of 6000 providers, this would represent a loss of 192 additional PCPs during the PACT implementation period we studied (April 2010–March 2012), or about 3% of the workforce. On an annualized basis, the turnover rate is similar to clinician turnover in non-VHA settings.25–27 Interaction analyses showed that the increased turnover was concentrated among older and more experienced PCPs. We are unaware of other studies of age-related changes in turnover among the health care workforce, but changes in organizational culture have been related to higher turnover among older employees in the technology industry.3 In contrast, we found that younger, less experienced providers did not leave primary care at a higher rate during the beginning of PACT implementation. We found other predictors of turnover. There was higher turnover among NPs and PAs compared with MDs. We are unaware of prior studies that have compared turnover across PCP professions, although, in the related area of job satisfaction, NPs and PAs have reported relatively high satisfaction.28,29 Consistent with previous findings,30 we found that turnover was inversely related to the unemployment rate in providers’ market areas. This study extends the literature on the relationship between PCMH implementation and health care workforce outcomes by measuring changes in PCP turnover during a large PCMH implementation. Our findings also contribute to the turnover literature by examining PCP-level factors of turnover using an extensive, administrative database. There are several limitations to this study. First, because PACT was a system-wide initiative, our study lacked a control group of VHA sites that did not implement PACT. To address this limitation, we applied an ITS model to account for the secular trend in turnover over a 9-year period. Second, we could not observe whether site-level PACT implementation began at a later date than the April 2010, system-wide date. Third, we could not discriminate between voluntary and involuntary turnover. However, terminations in VHA are rare and we have no reason to believe that PACT would affect involuntary terminations. Fourth, we used the overall unemployment rate as a proxy for opportunity costs of VHA employment rather than statistics specific to health care providers because only the former were reliably available at the market level on a quarterly basis. Fifth, we did not account for provider panel sizes because we did not have accurate historical data on the proportion of PCP-level FTE devoted to primary care, which is a determinant of VHA panel size. Finally, our implementation period is limited to eight quarters, whereas the PACT initiative is still evolving and may not be completely implemented in all sites. Thus, our results may only reflect a short-term effect. To summarize, we found a modest increase in PCP turnover during the first 2 years of implementation of a PCMH and found that changes in turnover were increasing in provider age r
2014 Lippincott Williams & Wilkins
PCMH Implementation and PCP Turnover
and experience. These findings provide useful insights for policymakers considering potential workforce effects when evaluating the overall costs and benefits of implementing PCMH.
REFERENCES 1. Rosland A, Nelson K, Sun H, et al. The patient-centered medical home in the Veterans Health Administration. Am J Manag Care. 2013; 19:263–272. 2. Klein S. The Veterans Health Administration: implementing patientcentered medical homes in the nation’s largest integrated delivery system. Commonwealth Fund, 2011. Available at: http://www.commonwealthfund. org/B/media/Files/Publications/case%20Study/2011/Sep/1537_Klein_ veterans_hlt_admin_case%20study.pdf. Accessed July, 2013. 3. Baron JN, Hannan MT, Burton MD. Labor pains: change in organizational models and employee turnover in young, high tech firms. Am J Sociol. 2001;106:960–1012. 4. Wanberg CR, Banas JT. Predictors and outcomes of openness to changes in a reorganizing workplace. J Appl Psychol. 2000;85:132–142. 5. Rafferty AE, Griffin MA. Perceptions of organizational change: a stress and coping perspective. J Appl Psychol. 2006;91:1154–1162. 6. Nutting PA, Miller WL, Crabtree BF, et al. Initial lessons from the first national demonstration project on practice transformation to a patientcentered medical home. Ann Fam Med. 2009;7:254–260. 7. Nutting PA, Crabtree BF, Miller WL, et al. Journey to the patientcentered medical home: a qualitative analysis of the experiences of practices in the National Demonstration Project. Ann Fam Med. 2010;8(suppl 1):S45–S56. S92. 8. Meyer H. Report from the field. Group Health’s move to the medical home: for doctors, it’s often a hard journey. Health Aff (Millwood). 2010;29:844–851. 9. Reid RJ, Coleman K, Johnson EA, et al. The Group Health medical home at year two: cost savings, higher patient satisfaction, and less burnout for providers. Health Aff (Millwood). 2010;29:835–843. 10. Reid RJ, Fishman PA, Yu O, et al. Patient-centered medical home demonstration: a prospective, quasi-experimental, before and after evaluation. Am J Manag Care. 2009;15:e71–e87. 11. Lewis SE, Nocon RS, Tang H, et al. Patient-centered medical home characteristics and staff morale in safety net clinics. Arch Intern Med. 2012;172:23–31. 12. Helfrich C, Dolan ED, Simonetti J, et al. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. J Gen Intern Med. 2014;29: 659–666. 13. Patient-Centered Primary Care Collaborative (PCPCC). Joint principles of the patient-centered medical home. Available at: http://www.aafp. org/dam/AAFP/documents/practice_management/pcmh/initiatives/ PCMHJoint.pdf. Accessed July, 2013. 14. Ferrante JM, Balasubramanian BA, Hudson SV, et al. Principles of the patient-centered medical home and preventive services delivery. Ann Fam Med. 2010;8:108–116. 15. Buchbinder SB, Wilson M, Melick CF, et al. Estimates of costs of primary care physician turnover. Am J Manag Care. 1999;5: 1431–1438. 16. Farrell D, Rusbult CE. Exchange variables as predictors of job satisfaction, job commitment, and turnover: The impact of rewards, costs, alternatives, and investments. Organ Behav Hum Perform. 1981;28:78–95. 17. Iverson RD. An event history analysis of employee turnover: the case of hospital employees in Australia. Hum Resource Manag Rev. 2000; 9:397–418. 18. Bureau of Labor Statistics. Local area unemployment statistics. Available at: http://www.bls.gov/lau/data.htm. Accessed July, 2013. 19. Health Economics Resource Center. Medicare wage index for VA facilities: 2000-2013. Available at: http://www.herc.research.va.gov/ resources/faq_l02.asp. Accessed July, 2013. 20. Sjo¨berg A, Sverke M. The interactive effect of job involvement and organizational commitment on job turnover revisited: a note on the mediating role of turnover intention. Scand J Psychol. 2000;41: 247–252.
www.lww-medicalcare.com |
1021
Sylling et al
21. Chen H-C, Chu C-I, Wang Y-H, et al. Turnover factors revisited: a longitudinal study of Taiwan-based staff nurses. Int J Nurs Stud. 2008;45:277–285. 22. Ramsay CR, Matowe L, Grilli R, et al. Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies. Int J Technol Assess Health Care. 2003;19: 613–623. 23. Light A, Ureta M. Panel estimates of male and female job turnover behavior: can female nonquitters be identified? J Labor Econ. 1992; 10:156–181. 24. Karaca-Mandic P, Norton EC, Dowd B. Interaction terms in nonlinear models. Health Serv Res. 2012;47(pt 1):255–274. 25. Buchbinder SB, Wilson M, Melick CF, et al. Primary care physician job satisfaction and turnover. Am J Manag Care. 2001;7:701–713.
1022 | www.lww-medicalcare.com
Medical Care
Volume 52, Number 12, December 2014
26. Misra-Hebert AD, Kay R, Stoller JK. A review of physician turnover: rates, causes, and consequences. Am J Med Qual. 2004;19:56–66. 27. Plomondon ME, Magid DJ, Steiner JF, et al. Primary care provider turnover and quality in managed care organizations. Am J Manag Care. 2007;13:465–472. 28. Kacel B, Millar M, Norris D. Measurement of nurse practitioner job satisfaction in a Midwestern state. J Am Acad Nurse Pract. 2005; 17:27–32. 29. Freeborn DK, Hooker RS. Satisfaction of physician assistants and other nonphysician providers in a managed care setting. Public Health Rep. 1995;110:714–719. 30. Carsten JM, Spector PE. Unemployment, job satisfaction, and employee turnover: a meta-analytic test of the Muchinsky model. J Appl Psychol. 1987;72:3.
r
2014 Lippincott Williams & Wilkins
