Predictors of Capacity in Public Health, Environmental, and Agricultural Laboratories Angela J. Beck, PhD, MPH; Matthew L. Boulton, MD, MPH rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr
Objectives: Ensuring adequate capacity to address population health concerns has challenged public health for decades. Organizational and workforce characteristics are theorized to contribute to organizational capacity. This article considers 2 possible quantitative measures of organizational capacity using public health, environmental, and agricultural laboratories (PHEALs) as the unit of interest and tests their associations with workforce and human resources variables. Design: The National Laboratory Capacity Assessment was developed by the University of Michigan Center of Excellence in Public Health Workforce Studies and the Association of Public Health Laboratories. Online data collection took place from July to September 2011. All statistical analyses were performed in 2013. Setting: US PHEALs were invited to participate in the study. All study participants were Association of Public Health Laboratories members. Participants: The Association of Public Health Laboratories distributed the National Laboratory Capacity Assessment survey to 105 PHEAL directors in all 50 states, the District of Columbia, and Puerto Rico, including 50 state public health laboratories, 41 local public health laboratories, 8 environmental laboratories, and 6 agricultural laboratories. Main Outcome Measures: Logistic regression analyses were performed to assess relationships between outcome measures of overall capacity and averaged program capacity and variables representing characteristics of PHEALs and their workforce, including number of workers, proportion of scientists, education, experience, training, and equipment quality. Results: The survey achieved a 76% response rate. Both capacity models showed that PHEALs offering an array of training opportunities are 4 times more likely to report higher capacity scores. One model showed a positive association between workforce size and
J Public Health Management Practice, 2014, 20(6), 654–661 C 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Copyright
capacity. Worker education and equipment quality were negatively associated with capacity in both models. Conclusions: The findings of this study provide empirical evidence that some workforce factors may influence organizational capacity of PHEALs. Techniques used to measure capacity and workforce factors must be improved to produce consistent findings across public health organizational data sets. KEY WORDS: laboratories, public health practice, public health
workforce
Ensuring adequate capacity to address population health concerns has been a public health challenge for decades. Deficiencies in public health system capacity at the organizational and workforce levels have been theorized to negatively affect sustainability of public health programs and interventions.1,2 Strengthening infrastructure has been noted by many as a primary strategy for improving capacity at the organizational and system levels,3-5 and a strong workforce is considered a key component of effective public health organizations and the foundation of public health infrastructure.6-10 Although public health services and systems research (PHSSR) has shown that organizational factors, Author Affiliations: Departments of Health Management and Policy (Dr Beck) and Epidemiology, Health Management and Policy, and Preventive Medicine (Dr Boulton), and Center of Excellence in Public Health Workforce Studies (Drs Beck and Boulton), University of Michigan School of Public Health, Ann Arbor. Funding for this project was provided by the Centers for Disease Control and Prevention through a cooperative agreement with the Public Health Foundation. The authors acknowledge the work of the Association of Public Health Laboratories staff and members of their Workforce Research and Pipeline Subcommittee in the development and implementation of this study. The authors declare no conflicts of interest. Correspondence: Angela J. Beck, PhD, MPH, Department of Health Management and Policy and Center of Excellence in Public Health Workforce Studies, University of Michigan School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109 ([email protected]). DOI: 10.1097/PHH.0000000000000050
654 Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
including leadership and financial resources, impact performance in achieving objectives related to population health outcomes,11-16 no published studies address how characteristics of the workforce, such as size, composition, training, and education, may impact the capacity of public health organizations to deliver services. “Capacity” is a broad term with varied definitions and contexts in the literature,2,17-20 making it difficult to measure and operationalize. Despite increasing interest in measuring capacity, standardized measurement tools for assessing organizational capacity do not yet exist for the public health system. The organizational systems literature shows common themes that are theorized to be associated with organizational capacity,21 including workforce competence, adaptability, durability,22 external and human resources,23 proper planning, effective leadership, networks, and specialized skills.24 Organizational capacity for PHSSR has been described as a “predictor of process and performance and resultant health outcomes” that occurs at the system, organizational, community, and individual levels and includes capabilities, knowledge, and resources.25 A conceptual model recently developed for PHSSR organizational capacity includes constructs of fiscal and economic resources, workforce and human resources, physical infrastructure, interorganizational relations, informational resources, system boundaries and size, governance and decision-making structure, and organizational culture.25 A 2005 study analyzed some of these organizational capacity themes in survey modules targeted to social service agencies serving homeless populations, including internal organizational characteristics, internal activities, external resources (eg, collaborations), technical assistance, and program activities.21 Workforce factors addressed number of workers, worker status (eg, full-/part-time), educational background, and staff training. Study findings were mixed but provide a basis for considering the many dimensions of organizational capacity. In public health, few attempts have been made to measure organizational capacity to meet program objectives or deliver Essential Public Health Services (EPHS). The Council of State and Territorial Epidemiologists last attempted to assess epidemiology program capacity of state health departments through qualitative measures reported by state epidemiologists in 2009; however, the workforce and organizational variables used in the study did not significantly predict capacity ratings.26 This article uses data collected in the 2011 National Laboratory Capacity Assessment (NLCA) study conducted by the Association of Public Health Laboratories (APHL) and the University of Michigan Center of Excellence in Public Health Workforce Studies to examine the PHSSR organizational capacity construct of
❘ 655
workforce and human resources using laboratories as the organizational unit of interest. The NLCA collected organizational and workforce data through a survey of directors of US public health, environmental, and agricultural laboratories (PHEALs), which are an essential component of the public health system often located in local, state, and federal government agencies. Public health laboratories are typically affiliated with health departments, whereas environmental and agricultural laboratories sometimes are not, although they constitute a part of the public health system and often work closely with health departments. PHEALs are unique because of the specificity and scope of their work, which tends to be highly routinized for laboratory aides/technicians and standardized with similar functions overall, regardless of the type or setting of the laboratory.27 For example, laboratories may vary in the number and type of tests they perform; however, the equipment and basic skills needed by the workforce to perform the tests are generally quite similar. PHEALs share characteristics of other public health department units in that their work is guided by the EPHS. However, they have distinctive characteristics related to their workforce and organizational structure.28 Although the NLCA did not collect data for all possible workforce and human resources variables related to PHSSR organizational capacity, the following suggested measures25 were included: number of full-time employees, staff education, experience, training, and expertise. In addition, PHEAL directors were asked to rate their laboratory’s capacity to perform necessary activities and services in 2 different ways. One measure is a summary rating of the laboratory’s overall capacity to perform activities and services in all program areas. The directors were also asked to rate capacity in 19 separate laboratory program areas. This study applies aspects of the PHSSR organizational capacity framework to answer 3 research questions: (1) Do PHEAL directors report the same level of capacity for their laboratory for both capacity measures? (2) Do organizational and individual level factors related to workforce and human resources significantly predict organizational capacity in PHEALs? and (3) Which capacity variable is a more useful measure of organizational capacity in PHEALs?
● Methods The APHL and the University of Michigan Center of Excellence in Public Health Workforce Studies jointly developed the NLCA to assess organizational capacity of PHEALs, as well as the size, composition, and characteristics of the PHEAL workforce. The survey was piloted in 2011 with 4 laboratory directors in North Dakota, New Mexico, Michigan, and Vermont.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
656 ❘ Journal of Public Health Management and Practice Cognitive interviews were performed with all pilot testers to obtain feedback on survey design, including the wording of survey questions and response choices, and revise the survey instrument. The APHL developed the online survey questionnaire using the mrInterview platform and distributed it to 105 PHEAL directors in all 50 states, the District of Columbia, and Puerto Rico, including 50 state public health laboratories, 41 local public health laboratories, 8 environmental laboratories, and 6 agricultural laboratories. It is possible that a small number of non-APHL member PHEALs exist in the US; these laboratories were not eligible for inclusion in the study. Data collection took place from July to September 2011. APHL staff followed up with laboratory directors by e-mail and phone throughout the data collection period to encourage additional responses.28
Measures Two measures of PHEAL capacity were used as outcome variables in separate analyses. The first model uses overall capacity, which is a summary rating of the laboratory’s capacity to perform necessary activities and services in all program areas. Laboratory directors used a 6-point rating scale of none (0% capacity to perform), minimal capacity (1%-24%), partial capacity (25%-49%), substantial capacity (50%-74%), almost full capacity (75%-99%), and full capacity (100%). On the basis of the frequency distribution of this variable, it was dichotomized so that all PHEALs reporting full capacity were coded as “1” and all others were coded “0.” Averaged program capacity was computed by using the same 6-point response scale to score 19 laboratory program areas: agricultural chemistry, agricultural microbiology, bacteriology, clinical chemistry/hematology, education/training, emergency preparedness, environmental microbiology, administration/operation, quality assurance and/or continuing quality improvement, regulation and inspection, safety and/or security, molecular biology, mycology, newborn screening, parasitology, serology/immunology, toxicology, and virology. Laboratories were also given the option to respond “not applicable” for program areas they did not offer, which were removed from their averaged program capacity score. The frequency distribution for this capacity variable was much different from the overall capacity variable. The variable was dichotomized so that PHEALs reporting substantial, almost full, or full capacity were coded as “1,” whereas others were coded “0.” Six predictor variables were used in both models. Number of workers is a summation of full-time equivalent (FTE) workers in each of the 19 laboratory pro-
gram areas. The scientists variable represents the proportion of workers classified in one of the following 4 positions: laboratory scientist, laboratory scientistsupervisor, laboratory scientist-manager, and laboratory developmental scientist. Full descriptions of these job classifications have been reported by the APHL and the University of Michigan Center of Excellence in Public Health Workforce Studies.28 The denominator used for all proportional variables reflects the total number of workers reported by job classification, which varies slightly from the total FTEs reported for the number of workers variable. The education variable represents the proportion of workers whose highest degree obtained was a bachelor’s or graduate degree in any field of study. Minimum required experience represents the minimum number of years of laboratory experience the PHEAL requires for entry-level laboratory scientists. Having no experience requirement was the referent category, coded 0, whereas having a minimum experience requirement was coded 1. The training support variable is a composite summation of responses to questions related to the following continuing education and professional development opportunities offered by the PHEAL: financial support for courses; internal training opportunities; training for laboratory partners; reimbursement for dues/memberships to professional societies; support staff positions responsible for monitoring or providing internal training; time off to attend classes; time off to participate in on-site or online trainings; and tuition reimbursement. PHEALs supporting at least 6 of the 8 types of training opportunities were coded 1; the rest were coded 0. Finally, equipment quality was used as a control variable, given its potential impact on whether a laboratory can adequately carry out its activities and services. PHEAL directors rated the general quality of the instrumentation and equipment in their laboratory on a 5-point Likert scale of “very poor,” “poor,” “fair,” “good,” and “very good.” A dichotomous variable was created where “1” denotes good/very good and “0” denotes all other responses.
Statistical procedures and analysis All analyses were performed using SPSS (version 19). All variables reflected an approximately normal distribution with the exception of the number of workers variable, which exhibited some positive skewness. A t test was performed to compare the means of the overall capacity variables and averaged program capacity variables before the 2 variables were dichotomized. After dichotomization, logistic regression analyses were performed for each PHEAL capacity variable. The University of Michigan institutional review board reviewed the study and ruled it exempt from ongoing review.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
● Results The survey achieved a 76% (80/105) response rate from PHEALs, representing 49 states; the District of Columbia and Puerto Rico did not respond to the survey. Respondents included 49 public health, 4 agricultural, and 3 environmental laboratories, with the other 24 PHEALs reported as various combinations of the 3 laboratory types. Descriptive statistics were examined for all variables. Before dichotomization, the outcome variables of averaged program capacity and overall capacity were moderately correlated (r = 0.619) and their means differed significantly (P < .001), with laboratory directors reporting a mean score of 4.15 for averaged program capacity and 5.36 for overall capacity. Approximately 68% of the PHEALs (30/44) that reported full overall capacity also reported having substantial-to-full averaged program capacity (χ 2 = 14.777; P < .0001). Agricultural chemistry, agricultural microbiology, and clinical chemistry/hematology were among the program areas most likely to be reported as “not applicable” by PHEALs (Table 1). The total number of laboratorians employed by the responding PHEALs was 5561 (mean = 69.5, SD = 90.08; range, 4-635). Approximately 86% of the workforce has obtained a college degree (SD = 13.82; range, 9%-100%). Thirty-seven PHEALs (46%) do not have a minimum years of experience requirement, 12 (15%) require less than 1 year of job experience for entry-level scientist positions, 26 (33%) require 1 to 2 years of experience, and 5 (6%) require more than 2 years of experience. Approximately 77% of the workforce is employed in a scientist classification (SD = 13.44; range, 33%-98%). Overall, almost half of PHEALs (46%) provide at least 6 of the 8 continuing education/professional development provisions surveyed, with all PHEALs offering internal training opportunities for staff (Figure). All directors rated the quality of their equipment and instrumentation as fair (41/80; 51%), good (35/80; 44%), or very good (4/80; 5%).
Model 1: Overall capacity An omnibus test of model coefficients was statisti2 = 29.469; P < .0001), indicating cally significant (χ6,80 that the predictors, as a set, reliably distinguished between full capacity and less-than-full capacity at the α = .05 level. Full capacity was significantly predicted by training opportunities offered by the PHEAL, equipment quality, and education. The results suggest that the odds of reporting full capacity increases by 4.243 (95% confidence interval [CI], 1.230-14.640; P = .022) if the PHEAL provides at least 6 types of training opportunities. However, the odds of reporting full capacity decreases by 0.120 (95% CI, 0.035-0.408; P = .001) if the
❘ 657
PHEAL reported having good/very good equipment and decreases by 0.924 (95% CI, 0.870-0.983; P = .012) if the PHEAL reported a higher proportion of workers with college degrees. Overall, this model is predicting 73.8% capacity scores and explained 41.1% of the variance, according to the Nagelkerke R2 (Table 2).
Model 2: Averaged program capacity The omnibus test of model coefficients was also statis2 = 22.395; tically significant in the second model (χ6,80 P = .001), indicating that the predictors, as a set, reliably distinguished between having substantial, almost full, or full capacity and having no, minimal, or partial capacity. Having substantial or better capacity was significantly predicted by training opportunities offered by the PHEAL (odds ratio [OR] = 4.887; 95% CI, 1.53914.991; P = .006), suggesting that PHEALs that provide more training opportunities are 4 times as likely to report a substantial or better capacity score. In addition, the odds of reporting a substantial or better capacity score increases with more PHEAL workers (OR = 1.014; 95% CI, 1.000-1.027). Similar to the first model, the odds of reporting substantial or better capacity decreases by 0.324 (95% CI, 0.106-0.994; P = .049) if the PHEAL reported having good/very good equipment and decreases by 0.947 (95% CI, 0.897-0.999; P = .047) if the PHEAL reported a higher proportion of workers with college degrees. Overall, the model is predicting 75% of capacity scores and explained 32.7% of the variance (Table 2).
● Discussion The findings of this study provide empirical evidence that some organizational and individual level factors related to workforce and human resources can influence the capacity of PHEALs to deliver public health services. Across both models, higher laboratory capacity was associated with PHEALs that provide at least 6 of the 8 surveyed workforce development opportunities. Interestingly, when used as individual variables in the models, none of the workforce development opportunities were significant predictors of PHEAL capacity on their own (data not presented). It appears that offering an array of professional development opportunities is an important indicator of organizational capacity in PHEALs. National efforts to better train the public health workforce have increased over the past decade. Although no studies directly show that worker training results in improved organizational capacity or performance, the findings of this study provide empirical support for laboratory policies that offer a range of professional development opportunities and may
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
658 ❘ Journal of Public Health Management and Practice
TABLE 1 ● Number (%) and Mean Score of Laboratories Reporting Overall Capacity and Averaged Program Capacity, by Program Area: 2011 National Laboratory Capacity Assessment qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
Variable
None (1)
Minimal (2)
Partial (3)
Substantial (4)
Almost Full (5)
Full (6)
N/A
Mean Score
Overall capacity Averaged program capacitya Agricultural chemistry Agricultural microbiology Bacteriology Clinical chemistry/hematology Education/training Emergency preparedness and response Environmental microbiology Environmental chemistry Laboratory administration/operation Laboratory quality assurance Laboratory regulation and inspection Laboratory safety and/or security Molecular biology Mycology Newborn screening Parasitology Serology/immunology Toxicology Virology
0 (0%) 10 (12%) 25 (31%) 28 (35%) 1 (1%) 23 (29%)
0 (0%) 6 (7%) 9 (11%) 5 (6%) 2 (3%) 9 (11%)
1 (1%) 7 (9%) 6 (8%) 10 (13%) 4 (5%) 0 (0%)
8 (10%) 12 (16%) 2 (3%) 4 (5%) 15 (19%) 5 (6%)
32 (40%) 16 (19%) 2 (3%) 1 (1%) 25 (31%) 5 (6%)
39 (49%) 18 (22%) 6 (8%) 6 (8%) 29 (36%) 5 (6%)
... 12 (15%) 30 (38%) 26 (33%) 4 (5%) 33 (41%)
5.36 4.15 2.30 2.31 4.95 2.47
0 (0%) 1 (1%)
22 (28%) 1 (1%)
19 (24%) 7 (9%)
16 (20%) 20 (25%)
5 (6%) 30 (28%)
13 (16%) 21 (26%)
5 (6%) 0 (0%)
3.57 4.75
3 (4%) 11 (14%) 0 (0%)
8 (10%) 5 (6%) 3 (4%)
17 (21%) 5 (6%) 8 (10%)
17 (21%) 11 (14%) 15 (19%)
15 (19%) 23 (29%) 22 (28%)
16 (20%) 11 (14%) 32 (40%)
4 (5%) 14 (18%) 0 (0%)
4.07 3.95 4.90
2 (3%)
2 (3%)
10 (13%)
9 (11%)
28 (35%)
29 (36%)
0 (0%)
4.83
6 (8%)
0 (0%)
7 (9%)
13 (16%)
13 (16%)
25 (31%)
16 (20%)
4.59
0 (0%)
2 (3%)
5 (6%)
15 (19%)
25 (31%)
31 (39%)
2 (3%)
5.00
1 (1%) 21 (26%) 26 (33%) 13 (16%) 4 (5%) 15 (19%) 6 (8%)
3 (4%) 8 (10%) 2 (3%) 12 (15%) 2 (3%) 8 (10%) 5 (6%)
2 (3%) 2 (3%) 0 (0%) 5 (6%) 5 (6%) 13 (16%) 8 (10%)
25 (31%) 9 (11%) 2 (3%) 12 (15%) 20 (25%) 6 (8%) 20 (25%)
25 (31%) 10 (13%) 6 (8%) 10 (13%) 25 (31%) 7 (9%) 18 (23%)
19 (24%) 16 (20%) 21 (26%) 21 (26%) 17 (21%) 6 (8%) 17 (21%)
5 (6%) 14 (18%) 23 (29%) 7 (9%) 7 (9%) 25 (31%) 6 (8%)
4.69 3.41 3.40 3.78 4.52 3.00 4.22
a Numbers
represent average of the 19 program area scores.
FIGURE ● Training Support Provided by PHEALs for Workers: 2011 National Laboratory Capacity Assessment (n = 80)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
Abbreviation: PHEAL, public health, environmental, and agricultural laboratories.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
❘ 659
TABLE 2 ● Regression Weights and Model Summary Statistics for Overall Capacity and Averaged Program Capacity
Models: 2011 National Laboratory Capacity Assessment qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq Model 1: Overall Capacity Variables Constant Number of workers Scientists, % Education, % Experience Training support Equipment quality Model summary aP bP
B
SE
Exp(b)
6.351 2.665 572.86 0.005 0.003 1.005 0.010 0.024 1.010 − 0.079 0.031 0.924 − 1.091 0.586 0.336 1.445 0.632 4.243 − 2.120 0.624 0.120 Cox and Snell R2 = 0.308 −2 log likelihood = 81.39
95% CI
Model 2: Averaged Program Capacity
P
.017 0.998-1.011 .134 0.963-1.059 .679 0.870-0.983 .012a 0.107-1.059 .063 1.230-14.640 .022a 0.035-0.408 .001b Nagelkerke R2 = 0.411 2 = 29.469b χ6,80
B
SE
Exp(b)
4.199 2.352 66.596 0.014 0.007 1.014 − 0.007 0.023 0.993 − 0.055 0.028 0.947 0.270 0.578 1.310 1.587 0.572 4.887 − 1.127 0.572 0.324 Cox and Snell R2 = 0.244 −2 log likelihood = 87.71
95% CI
P
.074 1.000-1.027 .049a 0.949-1.040 .775 0.897-0.999 .047a 0.422-4.071 .640 1.593-14.991 .006b 0.106-.994 .049a Nagelkerke R2 = 0.327 2 χ6,80 = 22.395b
< .05. < .01.
encourage administrators to expand continuing education options for laboratory staff.29 The results of the averaged program capacity model showed a positive association between the number of FTE workers employed by the PHEAL and capacity, indicating that the size of the workforce impacts ability to deliver services. More research is needed to determine the specific composition and characteristics of the workforce necessary to ensure optimal capacity in PHEALs. Surprisingly, in both models, education and equipment quality were negatively associated with capacity, in that having a higher proportion of college-educated workers and having good or very good equipment reduced the odds of reporting full capacity in the first model and substantial or better capacity in the second model. It would be expected that a highly educated workforce yields workers of higher competence, who would then contribute to higher capacity of the organization to deliver EPHS. A workforce profile of PHEALs reveals that nearly three-fourths of the PHEAL workforce is educated at the bachelor’s degree level or below, a substantially different finding from most other public health professional areas, which typically call for a master’s degree.28 Laboratory aides/technicians comprise 16% of the workforce; nearly 70% of them are trained at the associate’s degree (21%) or high school or equivalent (46%) level. Their job tasks include processing specimens/samples, performing moderate-tohigh complexity testing, and reporting test results.28 The negative association between a college-educated workforce and laboratory capacity may imply that the aides/technicians, although a small part of the workforce, play a substantial role in the overall delivery of EPHS by virtue of their job tasks. It is also possible that laboratory capacity relies more heavily on worker training than education. Laboratory tasks tend to be
more procedurally oriented than tasks performed by epidemiologists, for example, which may rely on more critical thinking and analytic skills. Laboratory tasks are complex, but repetitive, fixed, and procedurally rigid in how they are executed, perhaps making on-thejob training a more effective way to improve laboratory capacity than advanced degree attainment. The negative association between laboratory equipment and both measures of organizational capacity may reflect the limitations of using a summative measure to comprehensively evaluate innumerable pieces of equipment, which likely vary with regard to impact on capacity. For example, a low-quality piece of equipment with limited utility may have a lesser impact on laboratory capacity than a low-quality piece of equipment in constant use. This summative measure of equipment quality does not allow for these differences to be examined. This study tested 2 types of capacity models, which had different cutoff points based on the frequency distribution of the variables. The overall capacity model, which looked at associations between workforce/organizational variables and having full capacity (ie, 100% capacity to perform necessary activities and services), and the averaged program capacity model, which looked at associations between workforce/organizational variables and having substantialto-full capacity (ie, at least 50% capacity to perform necessary activities and services), both yielded some positive empirical findings, although this study shows that improvements are needed in the way capacity is assessed in the NLCA. First, although the 2 quantitative measures used the same response scale, PHEAL directors scored the capacity of their laboratories quite differently. For example, averaged program capacity yielded a significantly lower mean score from PHEAL directors
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
660 ❘ Journal of Public Health Management and Practice than did overall capacity, perhaps indicating that the process of scoring each program area individually may yield a more “honest” assessment of capacity than a single summary measure. Forty-four PHEAL directors reported full capacity on the summary measure; only 30 of those same directors reported having substantialto-full capacity for all 19 laboratory program areas. This paradox suggests that these measures, despite stemming from similarly phrased survey questions and using the same response scale, were interpreted differently by PHEAL directors. There may be a need to better define capacity during future data collection efforts. It is difficult to determine which outcome variable represents the more useful measure of capacity, as both measures have benefits and drawbacks. The overall capacity model may provide PHEAL directors with the ability to include more areas and activities in their assessment. For example, all administrative, managerial, or technical work performed by the laboratories may not be adequately captured in the averaged program capacity variable. However, the overall capacity measure appears to yield inflated scores. The natural cutoff point for this variable was full capacity versus non–full capacity; this cutoff point would not have been useful with the averaged program capacity variable because the number of PHEALs reporting an average score equivalent to full capacity was very low. This makes the 2 measures difficult to compare directly, as the directors reported scores for these variables differently. The averaged program capacity model produced some significant associations, representing an important improvement over the program capacity results in the 2009 Council of State and Territorial Epidemiologists study.26 This may be a result of differences in how program capacity was measured in the 2 studies, having more variables to test in the APHL data set than in the Council of State and Territorial Epidemiologists data set, or may imply that this measure works slightly better in a laboratory setting where program area tasks may be more standardized than in a state health department epidemiology bureau, for example. Although these findings are specific to these predictor and outcome variables, future surveys may find it beneficial to continue to include a program area capacity assessment for the purpose of monitoring change over time, as well as a summative measure of capacity, which may provide an additional estimate of the organization’s ability to deliver services. There are several study limitations to consider. First, it is possible that laboratories not responding to the NLCA were significantly different from respondents in terms of workforce factors and capacity scores. Survey responses included 49 of 50 state public health laboratories; nonrespondents were primarily environmental/agricultural laboratories. We believe we captured a significant majority of these laboratories nationally;
however, no national database of laboratories exists to verify this assumption. Second, self-rated subjective measures such as those used to measure capacity can be limited by bias. It is unknown whether the designated official from each PHEAL who completed the survey interpreted “capacity” in the same way, due to the abstractness of the term itself, although the use of a quantitative scale (ie, 0%-100% capacity) was intended to provide some consistency to how this measure was interpreted by respondents. In addition, the use of cognitive interviewing during pilot testing to get feedback from respondents on survey design issues, such as how the capacity scale was constructed, was intended to minimize reporting bias as much as possible. Third, organizational capacity has been theorized to include numerous factors, primarily organizational characteristics. These characteristics are not tested in these models because of lack of data. Therefore, it is possible that control variables are not included in this model that could alter the findings of the study. Finally, caution should be taken when generalizing these study findings to nonlaboratory settings.
● Conclusion This is the first analysis to attempt to identify workforce and organizational factors that predict capacity of PHEALs to perform necessary services and activities. Although the findings of the study and its attendant research questions are mixed, they do provide some guidance on how to structure future studies. Provision of training and professional development activities seems to be an important component of laboratory capacity. Overall, the study validates some previous epidemiology capacity study findings30 and suggests that current measures to estimate program and organizational capacity are difficult to use in statistical analyses and sometimes produce counterintuitive results. Use of a subjective summary measure may be too simplistic to adequately estimate an organization’s capacity. As noted, literature in PHSSR and other fields have identified several factors that potentially contribute to organizational capacity. No scale has received consensus endorsement; however, it would be beneficial to test these measures in organizational settings. Future research devoted to constructing a scale incorporating elements of organizational and workforce characteristics, workforce development activities, program activities, and partnerships and networks and testing its usability in public health organizations would be valuable. Finally, this study looked specifically at variables associated with capacity in PHEALs. Larger studies could potentially collect data from health departments and attempt to replicate or improve the model presented in
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
this study to identify correlates of capacity for state and local agencies. Overall, the techniques used to measure capacity and related workforce factors must be improved to produce consistent findings across public health organizational data sets. The field currently lacks a systematic method for collecting detailed workforce and organizational level data on an ongoing basis. Improving measures and data collection methods is a critical next step to identify associations between workforce characteristics and organizational capacity that could aid public health organizations in strengthening their ability to deliver services and improve population health.
REFERENCES 1. Hawe P, Noort M, King L, Jordens C. Multiplying health gains: the critical role of capacity building within health promotion programs. Health Policy. 1997;39(1):29-42. 2. Schwartz R, Smith C, Speers MA, et al. Capacity building and resource needs of state health agencies to implement community-based cardiovascular disease programs. J Public Health Policy. 1993;14(4):480-493. 3. Institute of Medicine. The Future of Public Health. Washington, DC: National Academies Press; 1988. 4. Roper WL, Baker EL, Dyal W, Nicola RM. Strengthening the public health system. Public Health Rep. 1992;107(6): 609-615. 5. Baker EL, Potter MA, Jones DL, et al. Public health infrastructure and our nation’s health. Annu Rev Public Health. 2005;26:303-318. 6. Centers for Disease Control and Prevention. Public Health’s Infrastructure: A Status Report. Atlanta, GA: Centers for Disease Control and Prevention; 2001. 7. Gebbie KM, Merrill J, Tilson HH. The public health workforce. Health Aff. 2002;21(6):57-67. 8. Lichtveld MY, Cioffi J, Henderson J, Sage M, Steele L. Editorial. People protected—public health prepared through a competent workforce. J Public Health Manag Pract. 2003;9(5):340-343. 9. Popovic T. Workforce science: a critical component to ensuring future of health. J Public Health Manag Pract. 2009;15(6):S3S4. 10. Tilson H, Gebbie KM. The public health workforce. Ann Rev Public Health. 2004;25:341-356. 11. Novick LF, Morrow CB, Mays GP. eds. Public Health Administration: Principles for Population-Based Management. 2nd ed. Sudbury, MA: Jones & Bartlett; 2008. 12. Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract. 2003;9(3): 183-187. 13. Honore PA, Simoes EJ, Jones WJ, Moonesinghe R. Practices in public health finance: an investigation of jurisdiction funding patterns and performance. J Public Health Manag Pract. 2004;10(5):444-450.
❘ 661
14. Schenck SE, Miller CA, Richards TB. Public health performance related to selected health status and risk measures. Am J Prev Med. 1995;11(6)(suppl):55-57. 15. Kanarek N, Stanley J, Bialek R. Local public health agency performance and community health status. J Public Health Manag Pract. 2006;12(6):522-527. 16. Erwin PC, Greene SB, Mays GP, Ricketts TC, Davis MV. The association of changes in local health department resources with changes in state-level health outcomes. Am J Public Health. 2011;101(4):609-615. 17. Meissner HI, Bergner L, Marconi KM. Developing cancer control capacity in state and local public health agencies. Public Health Rep. 1992;107(1):15-23. 18. Crisp BR, Swerissen H, Duckett SJ. Four approaches to capacity building in health: consequences for measurement and accountability. Health Promot Int. 2000;15(2):99-107. 19. Rissel C, Finnegan J, Bracht N. Evaluating quality and sustainability: issues and insights from the Minnesota Heart Health Program. Health Promot Int. 1995;10(3):199-207. 20. Goodman RM, Speers MA, McLeroy K, et al. Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Educ Behav. 1998;25(3):258-278. 21. White MD, Fisher C, Hadfield K, Saunders J, Williams L. Measuring organizational capacity among agencies serving the poor: implications for achieving organizational effectiveness. Justice Policy J. 2005;2:1-39. http://www.cjcj.org/uploads/ cjcj/documents/measuring_organizational.pdf. Accessed December 18, 2013. 22. Eisinger P. Organizational capacity and organizational effectiveness among street level food assistance programs. Nonprofit Voluntary Sector Q. 2002;31(1):115-130. 23. Rowe WE, Jacobs NF, Grant H. Facilitating development of organizational productive capacity: a role for empowerment evaluation. Can J Program Eval. 1999;(special issue):69-92. 24. Walker C, Weinhimer M. Community Development in the 1990s. Washington, DC: Urban Institute; 1998. 25. Meyer A, Davis M, Mays GP. Defining organizational capacity for public health services and systems research. J Public Health Manag Pract. 2012;18(6):535-544. 26. Boulton ML, Hadler J, Beck AJ, Ferland L, Lichtveld M. Assessment of epidemiology capacity in state health departments, 2004-2009. Public Health Rep. 2011;126(1):84-93. 27. University of Michigan Center of Excellence in Public Health Workforce Studies/Association of Public Health Laboratories. 2011 National Laboratory Capacity Assessment. Ann Arbor, MI: University of Michigan; 2012. 28. Association of Public Health Laboratories. The Core Functions of State Public Health Laboratories. Silver Spring, MD: Association of Public Health Laboratories; 2010. http:// www.aphl.org/aboutaphl/publications/documents/com_ 2010_corefunctionsphls.pdf. Accessed February 18, 2013. 29. DeBoy JM, Beck AJ, Boulton ML, Kim DH, Wichman MD, Leudtke PF. Core courses in public health laboratory science and practice: findings from 2006 and 2011 surveys. Public Health Rep. 2013;128(suppl 2):105-114. 30. Boulton ML, Lemmings J, Beck AJ. Assessment of epidemiology capacity in state health departments, 2001-2006. J Public Health Manag Pract. 2009;15(4):328-336.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Objectives: Ensuring adequate capacity to address population health concerns has challenged public health for decades. Organizational and workforce characteristics are theorized to contribute to organizational capacity. This article considers 2 possible quantitative measures of organizational capacity using public health, environmental, and agricultural laboratories (PHEALs) as the unit of interest and tests their associations with workforce and human resources variables. Design: The National Laboratory Capacity Assessment was developed by the University of Michigan Center of Excellence in Public Health Workforce Studies and the Association of Public Health Laboratories. Online data collection took place from July to September 2011. All statistical analyses were performed in 2013. Setting: US PHEALs were invited to participate in the study. All study participants were Association of Public Health Laboratories members. Participants: The Association of Public Health Laboratories distributed the National Laboratory Capacity Assessment survey to 105 PHEAL directors in all 50 states, the District of Columbia, and Puerto Rico, including 50 state public health laboratories, 41 local public health laboratories, 8 environmental laboratories, and 6 agricultural laboratories. Main Outcome Measures: Logistic regression analyses were performed to assess relationships between outcome measures of overall capacity and averaged program capacity and variables representing characteristics of PHEALs and their workforce, including number of workers, proportion of scientists, education, experience, training, and equipment quality. Results: The survey achieved a 76% response rate. Both capacity models showed that PHEALs offering an array of training opportunities are 4 times more likely to report higher capacity scores. One model showed a positive association between workforce size and
J Public Health Management Practice, 2014, 20(6), 654–661 C 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Copyright
capacity. Worker education and equipment quality were negatively associated with capacity in both models. Conclusions: The findings of this study provide empirical evidence that some workforce factors may influence organizational capacity of PHEALs. Techniques used to measure capacity and workforce factors must be improved to produce consistent findings across public health organizational data sets. KEY WORDS: laboratories, public health practice, public health
workforce
Ensuring adequate capacity to address population health concerns has been a public health challenge for decades. Deficiencies in public health system capacity at the organizational and workforce levels have been theorized to negatively affect sustainability of public health programs and interventions.1,2 Strengthening infrastructure has been noted by many as a primary strategy for improving capacity at the organizational and system levels,3-5 and a strong workforce is considered a key component of effective public health organizations and the foundation of public health infrastructure.6-10 Although public health services and systems research (PHSSR) has shown that organizational factors, Author Affiliations: Departments of Health Management and Policy (Dr Beck) and Epidemiology, Health Management and Policy, and Preventive Medicine (Dr Boulton), and Center of Excellence in Public Health Workforce Studies (Drs Beck and Boulton), University of Michigan School of Public Health, Ann Arbor. Funding for this project was provided by the Centers for Disease Control and Prevention through a cooperative agreement with the Public Health Foundation. The authors acknowledge the work of the Association of Public Health Laboratories staff and members of their Workforce Research and Pipeline Subcommittee in the development and implementation of this study. The authors declare no conflicts of interest. Correspondence: Angela J. Beck, PhD, MPH, Department of Health Management and Policy and Center of Excellence in Public Health Workforce Studies, University of Michigan School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109 ([email protected]). DOI: 10.1097/PHH.0000000000000050
654 Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
including leadership and financial resources, impact performance in achieving objectives related to population health outcomes,11-16 no published studies address how characteristics of the workforce, such as size, composition, training, and education, may impact the capacity of public health organizations to deliver services. “Capacity” is a broad term with varied definitions and contexts in the literature,2,17-20 making it difficult to measure and operationalize. Despite increasing interest in measuring capacity, standardized measurement tools for assessing organizational capacity do not yet exist for the public health system. The organizational systems literature shows common themes that are theorized to be associated with organizational capacity,21 including workforce competence, adaptability, durability,22 external and human resources,23 proper planning, effective leadership, networks, and specialized skills.24 Organizational capacity for PHSSR has been described as a “predictor of process and performance and resultant health outcomes” that occurs at the system, organizational, community, and individual levels and includes capabilities, knowledge, and resources.25 A conceptual model recently developed for PHSSR organizational capacity includes constructs of fiscal and economic resources, workforce and human resources, physical infrastructure, interorganizational relations, informational resources, system boundaries and size, governance and decision-making structure, and organizational culture.25 A 2005 study analyzed some of these organizational capacity themes in survey modules targeted to social service agencies serving homeless populations, including internal organizational characteristics, internal activities, external resources (eg, collaborations), technical assistance, and program activities.21 Workforce factors addressed number of workers, worker status (eg, full-/part-time), educational background, and staff training. Study findings were mixed but provide a basis for considering the many dimensions of organizational capacity. In public health, few attempts have been made to measure organizational capacity to meet program objectives or deliver Essential Public Health Services (EPHS). The Council of State and Territorial Epidemiologists last attempted to assess epidemiology program capacity of state health departments through qualitative measures reported by state epidemiologists in 2009; however, the workforce and organizational variables used in the study did not significantly predict capacity ratings.26 This article uses data collected in the 2011 National Laboratory Capacity Assessment (NLCA) study conducted by the Association of Public Health Laboratories (APHL) and the University of Michigan Center of Excellence in Public Health Workforce Studies to examine the PHSSR organizational capacity construct of
❘ 655
workforce and human resources using laboratories as the organizational unit of interest. The NLCA collected organizational and workforce data through a survey of directors of US public health, environmental, and agricultural laboratories (PHEALs), which are an essential component of the public health system often located in local, state, and federal government agencies. Public health laboratories are typically affiliated with health departments, whereas environmental and agricultural laboratories sometimes are not, although they constitute a part of the public health system and often work closely with health departments. PHEALs are unique because of the specificity and scope of their work, which tends to be highly routinized for laboratory aides/technicians and standardized with similar functions overall, regardless of the type or setting of the laboratory.27 For example, laboratories may vary in the number and type of tests they perform; however, the equipment and basic skills needed by the workforce to perform the tests are generally quite similar. PHEALs share characteristics of other public health department units in that their work is guided by the EPHS. However, they have distinctive characteristics related to their workforce and organizational structure.28 Although the NLCA did not collect data for all possible workforce and human resources variables related to PHSSR organizational capacity, the following suggested measures25 were included: number of full-time employees, staff education, experience, training, and expertise. In addition, PHEAL directors were asked to rate their laboratory’s capacity to perform necessary activities and services in 2 different ways. One measure is a summary rating of the laboratory’s overall capacity to perform activities and services in all program areas. The directors were also asked to rate capacity in 19 separate laboratory program areas. This study applies aspects of the PHSSR organizational capacity framework to answer 3 research questions: (1) Do PHEAL directors report the same level of capacity for their laboratory for both capacity measures? (2) Do organizational and individual level factors related to workforce and human resources significantly predict organizational capacity in PHEALs? and (3) Which capacity variable is a more useful measure of organizational capacity in PHEALs?
● Methods The APHL and the University of Michigan Center of Excellence in Public Health Workforce Studies jointly developed the NLCA to assess organizational capacity of PHEALs, as well as the size, composition, and characteristics of the PHEAL workforce. The survey was piloted in 2011 with 4 laboratory directors in North Dakota, New Mexico, Michigan, and Vermont.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
656 ❘ Journal of Public Health Management and Practice Cognitive interviews were performed with all pilot testers to obtain feedback on survey design, including the wording of survey questions and response choices, and revise the survey instrument. The APHL developed the online survey questionnaire using the mrInterview platform and distributed it to 105 PHEAL directors in all 50 states, the District of Columbia, and Puerto Rico, including 50 state public health laboratories, 41 local public health laboratories, 8 environmental laboratories, and 6 agricultural laboratories. It is possible that a small number of non-APHL member PHEALs exist in the US; these laboratories were not eligible for inclusion in the study. Data collection took place from July to September 2011. APHL staff followed up with laboratory directors by e-mail and phone throughout the data collection period to encourage additional responses.28
Measures Two measures of PHEAL capacity were used as outcome variables in separate analyses. The first model uses overall capacity, which is a summary rating of the laboratory’s capacity to perform necessary activities and services in all program areas. Laboratory directors used a 6-point rating scale of none (0% capacity to perform), minimal capacity (1%-24%), partial capacity (25%-49%), substantial capacity (50%-74%), almost full capacity (75%-99%), and full capacity (100%). On the basis of the frequency distribution of this variable, it was dichotomized so that all PHEALs reporting full capacity were coded as “1” and all others were coded “0.” Averaged program capacity was computed by using the same 6-point response scale to score 19 laboratory program areas: agricultural chemistry, agricultural microbiology, bacteriology, clinical chemistry/hematology, education/training, emergency preparedness, environmental microbiology, administration/operation, quality assurance and/or continuing quality improvement, regulation and inspection, safety and/or security, molecular biology, mycology, newborn screening, parasitology, serology/immunology, toxicology, and virology. Laboratories were also given the option to respond “not applicable” for program areas they did not offer, which were removed from their averaged program capacity score. The frequency distribution for this capacity variable was much different from the overall capacity variable. The variable was dichotomized so that PHEALs reporting substantial, almost full, or full capacity were coded as “1,” whereas others were coded “0.” Six predictor variables were used in both models. Number of workers is a summation of full-time equivalent (FTE) workers in each of the 19 laboratory pro-
gram areas. The scientists variable represents the proportion of workers classified in one of the following 4 positions: laboratory scientist, laboratory scientistsupervisor, laboratory scientist-manager, and laboratory developmental scientist. Full descriptions of these job classifications have been reported by the APHL and the University of Michigan Center of Excellence in Public Health Workforce Studies.28 The denominator used for all proportional variables reflects the total number of workers reported by job classification, which varies slightly from the total FTEs reported for the number of workers variable. The education variable represents the proportion of workers whose highest degree obtained was a bachelor’s or graduate degree in any field of study. Minimum required experience represents the minimum number of years of laboratory experience the PHEAL requires for entry-level laboratory scientists. Having no experience requirement was the referent category, coded 0, whereas having a minimum experience requirement was coded 1. The training support variable is a composite summation of responses to questions related to the following continuing education and professional development opportunities offered by the PHEAL: financial support for courses; internal training opportunities; training for laboratory partners; reimbursement for dues/memberships to professional societies; support staff positions responsible for monitoring or providing internal training; time off to attend classes; time off to participate in on-site or online trainings; and tuition reimbursement. PHEALs supporting at least 6 of the 8 types of training opportunities were coded 1; the rest were coded 0. Finally, equipment quality was used as a control variable, given its potential impact on whether a laboratory can adequately carry out its activities and services. PHEAL directors rated the general quality of the instrumentation and equipment in their laboratory on a 5-point Likert scale of “very poor,” “poor,” “fair,” “good,” and “very good.” A dichotomous variable was created where “1” denotes good/very good and “0” denotes all other responses.
Statistical procedures and analysis All analyses were performed using SPSS (version 19). All variables reflected an approximately normal distribution with the exception of the number of workers variable, which exhibited some positive skewness. A t test was performed to compare the means of the overall capacity variables and averaged program capacity variables before the 2 variables were dichotomized. After dichotomization, logistic regression analyses were performed for each PHEAL capacity variable. The University of Michigan institutional review board reviewed the study and ruled it exempt from ongoing review.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
● Results The survey achieved a 76% (80/105) response rate from PHEALs, representing 49 states; the District of Columbia and Puerto Rico did not respond to the survey. Respondents included 49 public health, 4 agricultural, and 3 environmental laboratories, with the other 24 PHEALs reported as various combinations of the 3 laboratory types. Descriptive statistics were examined for all variables. Before dichotomization, the outcome variables of averaged program capacity and overall capacity were moderately correlated (r = 0.619) and their means differed significantly (P < .001), with laboratory directors reporting a mean score of 4.15 for averaged program capacity and 5.36 for overall capacity. Approximately 68% of the PHEALs (30/44) that reported full overall capacity also reported having substantial-to-full averaged program capacity (χ 2 = 14.777; P < .0001). Agricultural chemistry, agricultural microbiology, and clinical chemistry/hematology were among the program areas most likely to be reported as “not applicable” by PHEALs (Table 1). The total number of laboratorians employed by the responding PHEALs was 5561 (mean = 69.5, SD = 90.08; range, 4-635). Approximately 86% of the workforce has obtained a college degree (SD = 13.82; range, 9%-100%). Thirty-seven PHEALs (46%) do not have a minimum years of experience requirement, 12 (15%) require less than 1 year of job experience for entry-level scientist positions, 26 (33%) require 1 to 2 years of experience, and 5 (6%) require more than 2 years of experience. Approximately 77% of the workforce is employed in a scientist classification (SD = 13.44; range, 33%-98%). Overall, almost half of PHEALs (46%) provide at least 6 of the 8 continuing education/professional development provisions surveyed, with all PHEALs offering internal training opportunities for staff (Figure). All directors rated the quality of their equipment and instrumentation as fair (41/80; 51%), good (35/80; 44%), or very good (4/80; 5%).
Model 1: Overall capacity An omnibus test of model coefficients was statisti2 = 29.469; P < .0001), indicating cally significant (χ6,80 that the predictors, as a set, reliably distinguished between full capacity and less-than-full capacity at the α = .05 level. Full capacity was significantly predicted by training opportunities offered by the PHEAL, equipment quality, and education. The results suggest that the odds of reporting full capacity increases by 4.243 (95% confidence interval [CI], 1.230-14.640; P = .022) if the PHEAL provides at least 6 types of training opportunities. However, the odds of reporting full capacity decreases by 0.120 (95% CI, 0.035-0.408; P = .001) if the
❘ 657
PHEAL reported having good/very good equipment and decreases by 0.924 (95% CI, 0.870-0.983; P = .012) if the PHEAL reported a higher proportion of workers with college degrees. Overall, this model is predicting 73.8% capacity scores and explained 41.1% of the variance, according to the Nagelkerke R2 (Table 2).
Model 2: Averaged program capacity The omnibus test of model coefficients was also statis2 = 22.395; tically significant in the second model (χ6,80 P = .001), indicating that the predictors, as a set, reliably distinguished between having substantial, almost full, or full capacity and having no, minimal, or partial capacity. Having substantial or better capacity was significantly predicted by training opportunities offered by the PHEAL (odds ratio [OR] = 4.887; 95% CI, 1.53914.991; P = .006), suggesting that PHEALs that provide more training opportunities are 4 times as likely to report a substantial or better capacity score. In addition, the odds of reporting a substantial or better capacity score increases with more PHEAL workers (OR = 1.014; 95% CI, 1.000-1.027). Similar to the first model, the odds of reporting substantial or better capacity decreases by 0.324 (95% CI, 0.106-0.994; P = .049) if the PHEAL reported having good/very good equipment and decreases by 0.947 (95% CI, 0.897-0.999; P = .047) if the PHEAL reported a higher proportion of workers with college degrees. Overall, the model is predicting 75% of capacity scores and explained 32.7% of the variance (Table 2).
● Discussion The findings of this study provide empirical evidence that some organizational and individual level factors related to workforce and human resources can influence the capacity of PHEALs to deliver public health services. Across both models, higher laboratory capacity was associated with PHEALs that provide at least 6 of the 8 surveyed workforce development opportunities. Interestingly, when used as individual variables in the models, none of the workforce development opportunities were significant predictors of PHEAL capacity on their own (data not presented). It appears that offering an array of professional development opportunities is an important indicator of organizational capacity in PHEALs. National efforts to better train the public health workforce have increased over the past decade. Although no studies directly show that worker training results in improved organizational capacity or performance, the findings of this study provide empirical support for laboratory policies that offer a range of professional development opportunities and may
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
658 ❘ Journal of Public Health Management and Practice
TABLE 1 ● Number (%) and Mean Score of Laboratories Reporting Overall Capacity and Averaged Program Capacity, by Program Area: 2011 National Laboratory Capacity Assessment qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
Variable
None (1)
Minimal (2)
Partial (3)
Substantial (4)
Almost Full (5)
Full (6)
N/A
Mean Score
Overall capacity Averaged program capacitya Agricultural chemistry Agricultural microbiology Bacteriology Clinical chemistry/hematology Education/training Emergency preparedness and response Environmental microbiology Environmental chemistry Laboratory administration/operation Laboratory quality assurance Laboratory regulation and inspection Laboratory safety and/or security Molecular biology Mycology Newborn screening Parasitology Serology/immunology Toxicology Virology
0 (0%) 10 (12%) 25 (31%) 28 (35%) 1 (1%) 23 (29%)
0 (0%) 6 (7%) 9 (11%) 5 (6%) 2 (3%) 9 (11%)
1 (1%) 7 (9%) 6 (8%) 10 (13%) 4 (5%) 0 (0%)
8 (10%) 12 (16%) 2 (3%) 4 (5%) 15 (19%) 5 (6%)
32 (40%) 16 (19%) 2 (3%) 1 (1%) 25 (31%) 5 (6%)
39 (49%) 18 (22%) 6 (8%) 6 (8%) 29 (36%) 5 (6%)
... 12 (15%) 30 (38%) 26 (33%) 4 (5%) 33 (41%)
5.36 4.15 2.30 2.31 4.95 2.47
0 (0%) 1 (1%)
22 (28%) 1 (1%)
19 (24%) 7 (9%)
16 (20%) 20 (25%)
5 (6%) 30 (28%)
13 (16%) 21 (26%)
5 (6%) 0 (0%)
3.57 4.75
3 (4%) 11 (14%) 0 (0%)
8 (10%) 5 (6%) 3 (4%)
17 (21%) 5 (6%) 8 (10%)
17 (21%) 11 (14%) 15 (19%)
15 (19%) 23 (29%) 22 (28%)
16 (20%) 11 (14%) 32 (40%)
4 (5%) 14 (18%) 0 (0%)
4.07 3.95 4.90
2 (3%)
2 (3%)
10 (13%)
9 (11%)
28 (35%)
29 (36%)
0 (0%)
4.83
6 (8%)
0 (0%)
7 (9%)
13 (16%)
13 (16%)
25 (31%)
16 (20%)
4.59
0 (0%)
2 (3%)
5 (6%)
15 (19%)
25 (31%)
31 (39%)
2 (3%)
5.00
1 (1%) 21 (26%) 26 (33%) 13 (16%) 4 (5%) 15 (19%) 6 (8%)
3 (4%) 8 (10%) 2 (3%) 12 (15%) 2 (3%) 8 (10%) 5 (6%)
2 (3%) 2 (3%) 0 (0%) 5 (6%) 5 (6%) 13 (16%) 8 (10%)
25 (31%) 9 (11%) 2 (3%) 12 (15%) 20 (25%) 6 (8%) 20 (25%)
25 (31%) 10 (13%) 6 (8%) 10 (13%) 25 (31%) 7 (9%) 18 (23%)
19 (24%) 16 (20%) 21 (26%) 21 (26%) 17 (21%) 6 (8%) 17 (21%)
5 (6%) 14 (18%) 23 (29%) 7 (9%) 7 (9%) 25 (31%) 6 (8%)
4.69 3.41 3.40 3.78 4.52 3.00 4.22
a Numbers
represent average of the 19 program area scores.
FIGURE ● Training Support Provided by PHEALs for Workers: 2011 National Laboratory Capacity Assessment (n = 80)
qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq
Abbreviation: PHEAL, public health, environmental, and agricultural laboratories.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
❘ 659
TABLE 2 ● Regression Weights and Model Summary Statistics for Overall Capacity and Averaged Program Capacity
Models: 2011 National Laboratory Capacity Assessment qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq Model 1: Overall Capacity Variables Constant Number of workers Scientists, % Education, % Experience Training support Equipment quality Model summary aP bP
B
SE
Exp(b)
6.351 2.665 572.86 0.005 0.003 1.005 0.010 0.024 1.010 − 0.079 0.031 0.924 − 1.091 0.586 0.336 1.445 0.632 4.243 − 2.120 0.624 0.120 Cox and Snell R2 = 0.308 −2 log likelihood = 81.39
95% CI
Model 2: Averaged Program Capacity
P
.017 0.998-1.011 .134 0.963-1.059 .679 0.870-0.983 .012a 0.107-1.059 .063 1.230-14.640 .022a 0.035-0.408 .001b Nagelkerke R2 = 0.411 2 = 29.469b χ6,80
B
SE
Exp(b)
4.199 2.352 66.596 0.014 0.007 1.014 − 0.007 0.023 0.993 − 0.055 0.028 0.947 0.270 0.578 1.310 1.587 0.572 4.887 − 1.127 0.572 0.324 Cox and Snell R2 = 0.244 −2 log likelihood = 87.71
95% CI
P
.074 1.000-1.027 .049a 0.949-1.040 .775 0.897-0.999 .047a 0.422-4.071 .640 1.593-14.991 .006b 0.106-.994 .049a Nagelkerke R2 = 0.327 2 χ6,80 = 22.395b
< .05. < .01.
encourage administrators to expand continuing education options for laboratory staff.29 The results of the averaged program capacity model showed a positive association between the number of FTE workers employed by the PHEAL and capacity, indicating that the size of the workforce impacts ability to deliver services. More research is needed to determine the specific composition and characteristics of the workforce necessary to ensure optimal capacity in PHEALs. Surprisingly, in both models, education and equipment quality were negatively associated with capacity, in that having a higher proportion of college-educated workers and having good or very good equipment reduced the odds of reporting full capacity in the first model and substantial or better capacity in the second model. It would be expected that a highly educated workforce yields workers of higher competence, who would then contribute to higher capacity of the organization to deliver EPHS. A workforce profile of PHEALs reveals that nearly three-fourths of the PHEAL workforce is educated at the bachelor’s degree level or below, a substantially different finding from most other public health professional areas, which typically call for a master’s degree.28 Laboratory aides/technicians comprise 16% of the workforce; nearly 70% of them are trained at the associate’s degree (21%) or high school or equivalent (46%) level. Their job tasks include processing specimens/samples, performing moderate-tohigh complexity testing, and reporting test results.28 The negative association between a college-educated workforce and laboratory capacity may imply that the aides/technicians, although a small part of the workforce, play a substantial role in the overall delivery of EPHS by virtue of their job tasks. It is also possible that laboratory capacity relies more heavily on worker training than education. Laboratory tasks tend to be
more procedurally oriented than tasks performed by epidemiologists, for example, which may rely on more critical thinking and analytic skills. Laboratory tasks are complex, but repetitive, fixed, and procedurally rigid in how they are executed, perhaps making on-thejob training a more effective way to improve laboratory capacity than advanced degree attainment. The negative association between laboratory equipment and both measures of organizational capacity may reflect the limitations of using a summative measure to comprehensively evaluate innumerable pieces of equipment, which likely vary with regard to impact on capacity. For example, a low-quality piece of equipment with limited utility may have a lesser impact on laboratory capacity than a low-quality piece of equipment in constant use. This summative measure of equipment quality does not allow for these differences to be examined. This study tested 2 types of capacity models, which had different cutoff points based on the frequency distribution of the variables. The overall capacity model, which looked at associations between workforce/organizational variables and having full capacity (ie, 100% capacity to perform necessary activities and services), and the averaged program capacity model, which looked at associations between workforce/organizational variables and having substantialto-full capacity (ie, at least 50% capacity to perform necessary activities and services), both yielded some positive empirical findings, although this study shows that improvements are needed in the way capacity is assessed in the NLCA. First, although the 2 quantitative measures used the same response scale, PHEAL directors scored the capacity of their laboratories quite differently. For example, averaged program capacity yielded a significantly lower mean score from PHEAL directors
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
660 ❘ Journal of Public Health Management and Practice than did overall capacity, perhaps indicating that the process of scoring each program area individually may yield a more “honest” assessment of capacity than a single summary measure. Forty-four PHEAL directors reported full capacity on the summary measure; only 30 of those same directors reported having substantialto-full capacity for all 19 laboratory program areas. This paradox suggests that these measures, despite stemming from similarly phrased survey questions and using the same response scale, were interpreted differently by PHEAL directors. There may be a need to better define capacity during future data collection efforts. It is difficult to determine which outcome variable represents the more useful measure of capacity, as both measures have benefits and drawbacks. The overall capacity model may provide PHEAL directors with the ability to include more areas and activities in their assessment. For example, all administrative, managerial, or technical work performed by the laboratories may not be adequately captured in the averaged program capacity variable. However, the overall capacity measure appears to yield inflated scores. The natural cutoff point for this variable was full capacity versus non–full capacity; this cutoff point would not have been useful with the averaged program capacity variable because the number of PHEALs reporting an average score equivalent to full capacity was very low. This makes the 2 measures difficult to compare directly, as the directors reported scores for these variables differently. The averaged program capacity model produced some significant associations, representing an important improvement over the program capacity results in the 2009 Council of State and Territorial Epidemiologists study.26 This may be a result of differences in how program capacity was measured in the 2 studies, having more variables to test in the APHL data set than in the Council of State and Territorial Epidemiologists data set, or may imply that this measure works slightly better in a laboratory setting where program area tasks may be more standardized than in a state health department epidemiology bureau, for example. Although these findings are specific to these predictor and outcome variables, future surveys may find it beneficial to continue to include a program area capacity assessment for the purpose of monitoring change over time, as well as a summative measure of capacity, which may provide an additional estimate of the organization’s ability to deliver services. There are several study limitations to consider. First, it is possible that laboratories not responding to the NLCA were significantly different from respondents in terms of workforce factors and capacity scores. Survey responses included 49 of 50 state public health laboratories; nonrespondents were primarily environmental/agricultural laboratories. We believe we captured a significant majority of these laboratories nationally;
however, no national database of laboratories exists to verify this assumption. Second, self-rated subjective measures such as those used to measure capacity can be limited by bias. It is unknown whether the designated official from each PHEAL who completed the survey interpreted “capacity” in the same way, due to the abstractness of the term itself, although the use of a quantitative scale (ie, 0%-100% capacity) was intended to provide some consistency to how this measure was interpreted by respondents. In addition, the use of cognitive interviewing during pilot testing to get feedback from respondents on survey design issues, such as how the capacity scale was constructed, was intended to minimize reporting bias as much as possible. Third, organizational capacity has been theorized to include numerous factors, primarily organizational characteristics. These characteristics are not tested in these models because of lack of data. Therefore, it is possible that control variables are not included in this model that could alter the findings of the study. Finally, caution should be taken when generalizing these study findings to nonlaboratory settings.
● Conclusion This is the first analysis to attempt to identify workforce and organizational factors that predict capacity of PHEALs to perform necessary services and activities. Although the findings of the study and its attendant research questions are mixed, they do provide some guidance on how to structure future studies. Provision of training and professional development activities seems to be an important component of laboratory capacity. Overall, the study validates some previous epidemiology capacity study findings30 and suggests that current measures to estimate program and organizational capacity are difficult to use in statistical analyses and sometimes produce counterintuitive results. Use of a subjective summary measure may be too simplistic to adequately estimate an organization’s capacity. As noted, literature in PHSSR and other fields have identified several factors that potentially contribute to organizational capacity. No scale has received consensus endorsement; however, it would be beneficial to test these measures in organizational settings. Future research devoted to constructing a scale incorporating elements of organizational and workforce characteristics, workforce development activities, program activities, and partnerships and networks and testing its usability in public health organizations would be valuable. Finally, this study looked specifically at variables associated with capacity in PHEALs. Larger studies could potentially collect data from health departments and attempt to replicate or improve the model presented in
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Predictors of Capacity in PHEALs
this study to identify correlates of capacity for state and local agencies. Overall, the techniques used to measure capacity and related workforce factors must be improved to produce consistent findings across public health organizational data sets. The field currently lacks a systematic method for collecting detailed workforce and organizational level data on an ongoing basis. Improving measures and data collection methods is a critical next step to identify associations between workforce characteristics and organizational capacity that could aid public health organizations in strengthening their ability to deliver services and improve population health.
REFERENCES 1. Hawe P, Noort M, King L, Jordens C. Multiplying health gains: the critical role of capacity building within health promotion programs. Health Policy. 1997;39(1):29-42. 2. Schwartz R, Smith C, Speers MA, et al. Capacity building and resource needs of state health agencies to implement community-based cardiovascular disease programs. J Public Health Policy. 1993;14(4):480-493. 3. Institute of Medicine. The Future of Public Health. Washington, DC: National Academies Press; 1988. 4. Roper WL, Baker EL, Dyal W, Nicola RM. Strengthening the public health system. Public Health Rep. 1992;107(6): 609-615. 5. Baker EL, Potter MA, Jones DL, et al. Public health infrastructure and our nation’s health. Annu Rev Public Health. 2005;26:303-318. 6. Centers for Disease Control and Prevention. Public Health’s Infrastructure: A Status Report. Atlanta, GA: Centers for Disease Control and Prevention; 2001. 7. Gebbie KM, Merrill J, Tilson HH. The public health workforce. Health Aff. 2002;21(6):57-67. 8. Lichtveld MY, Cioffi J, Henderson J, Sage M, Steele L. Editorial. People protected—public health prepared through a competent workforce. J Public Health Manag Pract. 2003;9(5):340-343. 9. Popovic T. Workforce science: a critical component to ensuring future of health. J Public Health Manag Pract. 2009;15(6):S3S4. 10. Tilson H, Gebbie KM. The public health workforce. Ann Rev Public Health. 2004;25:341-356. 11. Novick LF, Morrow CB, Mays GP. eds. Public Health Administration: Principles for Population-Based Management. 2nd ed. Sudbury, MA: Jones & Bartlett; 2008. 12. Kennedy VC. A study of local public health system performance in Texas. J Public Health Manag Pract. 2003;9(3): 183-187. 13. Honore PA, Simoes EJ, Jones WJ, Moonesinghe R. Practices in public health finance: an investigation of jurisdiction funding patterns and performance. J Public Health Manag Pract. 2004;10(5):444-450.
❘ 661
14. Schenck SE, Miller CA, Richards TB. Public health performance related to selected health status and risk measures. Am J Prev Med. 1995;11(6)(suppl):55-57. 15. Kanarek N, Stanley J, Bialek R. Local public health agency performance and community health status. J Public Health Manag Pract. 2006;12(6):522-527. 16. Erwin PC, Greene SB, Mays GP, Ricketts TC, Davis MV. The association of changes in local health department resources with changes in state-level health outcomes. Am J Public Health. 2011;101(4):609-615. 17. Meissner HI, Bergner L, Marconi KM. Developing cancer control capacity in state and local public health agencies. Public Health Rep. 1992;107(1):15-23. 18. Crisp BR, Swerissen H, Duckett SJ. Four approaches to capacity building in health: consequences for measurement and accountability. Health Promot Int. 2000;15(2):99-107. 19. Rissel C, Finnegan J, Bracht N. Evaluating quality and sustainability: issues and insights from the Minnesota Heart Health Program. Health Promot Int. 1995;10(3):199-207. 20. Goodman RM, Speers MA, McLeroy K, et al. Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Educ Behav. 1998;25(3):258-278. 21. White MD, Fisher C, Hadfield K, Saunders J, Williams L. Measuring organizational capacity among agencies serving the poor: implications for achieving organizational effectiveness. Justice Policy J. 2005;2:1-39. http://www.cjcj.org/uploads/ cjcj/documents/measuring_organizational.pdf. Accessed December 18, 2013. 22. Eisinger P. Organizational capacity and organizational effectiveness among street level food assistance programs. Nonprofit Voluntary Sector Q. 2002;31(1):115-130. 23. Rowe WE, Jacobs NF, Grant H. Facilitating development of organizational productive capacity: a role for empowerment evaluation. Can J Program Eval. 1999;(special issue):69-92. 24. Walker C, Weinhimer M. Community Development in the 1990s. Washington, DC: Urban Institute; 1998. 25. Meyer A, Davis M, Mays GP. Defining organizational capacity for public health services and systems research. J Public Health Manag Pract. 2012;18(6):535-544. 26. Boulton ML, Hadler J, Beck AJ, Ferland L, Lichtveld M. Assessment of epidemiology capacity in state health departments, 2004-2009. Public Health Rep. 2011;126(1):84-93. 27. University of Michigan Center of Excellence in Public Health Workforce Studies/Association of Public Health Laboratories. 2011 National Laboratory Capacity Assessment. Ann Arbor, MI: University of Michigan; 2012. 28. Association of Public Health Laboratories. The Core Functions of State Public Health Laboratories. Silver Spring, MD: Association of Public Health Laboratories; 2010. http:// www.aphl.org/aboutaphl/publications/documents/com_ 2010_corefunctionsphls.pdf. Accessed February 18, 2013. 29. DeBoy JM, Beck AJ, Boulton ML, Kim DH, Wichman MD, Leudtke PF. Core courses in public health laboratory science and practice: findings from 2006 and 2011 surveys. Public Health Rep. 2013;128(suppl 2):105-114. 30. Boulton ML, Lemmings J, Beck AJ. Assessment of epidemiology capacity in state health departments, 2001-2006. J Public Health Manag Pract. 2009;15(4):328-336.
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
