ORIGINAL ARTICLE
Beryllium Biobank 3: Considerations for Improving Chronic Beryllium Disease Screening Philip Harber, MD, MPH and Jing Su, MS
Objective: To optimize beryllium worker screening. Methods: Berylliumexposed persons are classified as beryllium-exposed, beryllium-sensitized (BeS), or chronic beryllium disease. Implications of defining BeS by two or more positive lymphocyte proliferation tests (LPTs) were investigated with a simple binomial model. The potential effect of adjusting the interval for repeated intensive testing to detect chronic beryllium disease among persons with BeS was assessed with a Markov model. Results: Accuracy of properly identifying BeS is reduced as the number of repeated tests increases. Markov simulation illustrates that adjusting second-stage screening intervals on the basis of personal risk may significantly affect cost-effectiveness. Conclusions: The criteria for classification as BeS should be adjusted depending on the number of LPTs performed. Modifying the interval for repeated intensive testing on the basis of each worker’s data can improve cost-effectiveness.
C
hronic beryllium disease (CBD) is an immunologically mediated disease, which can lead to severe pulmonary dysfunction or death. Secondary prevention is detection of illness in an early stage at which intervention can prevent worsening, and screening is a systematic program to identify cases earlier than would occur through typical diagnostic approaches. The utility of systematically identifying early CBD is favored for several reasons as follows: 1. The disease develops sequentially. Beryllium-exposed (BeE) workers may develop immunologic beryllium sensitization (BeS) without disease. Then, some with BeS develop actual CBD. Initially, early CBD is mild and largely asymptomatic. Nevertheless, many persons with early CBD will progress to advanced CBD. Subjects must pass through BeS before developing disease. 2. Treatment of early CBD with corticosteroids or other immunosuppressive agents can be effective in preventing progression to advanced symptomatic CBD. 3. A noninvasive procedure (blood lymphocyte proliferation test [LPT]) is useful for detecting BeS. Extensive experience has shown the utility of the test, and automation has made its cost reasonable. Studies have shown that screening effectively identifies individuals needing treatment.1–4 4. There is a large population at risk, estimated to include 300,000 persons in the United States.5 5. A large cohort may be specifically identified because many workers in the US nuclear industry, particularly in work associated with weapons production, have been exposed and are potentially at risk.6 6. An existing organizational structure (the US Department of Energy) can organize screening programs. In addition, a US law From the Community, Environment, and Policy Division, Mel and Enid Zuckerman College of Public Health, University of Arizona, Tucson. Sources of funding: None. Conflicts of interest: None declared. Address correspondence to: Philip Harber, MD, MPH, Community, Environment, and Policy Division, Mel and Enid Zuckerman College of Public Health, University of Arizona, 1656 E. Mabel St, Room 112, Tucson, AZ 85724 ([email protected]). C 2014 by American College of Occupational and Environmental Copyright Medicine DOI: 10.1097/JOM.0000000000000200
provides specific benefits encouraging identification of berylliumrelated diseases in current and past nuclear weapons workers (Energy Employees Occupational and Illness Compensation Program). Several factors make CBD of particular interest when considering design of screening and secondary prevention programs. These factors include the following: 1. Systematic efforts to identify early CBD must occur over many years because CBD can develop very soon after initiating exposure7–9 and also develop with very long latency, often many years after the most recent exposure.3 2. Systematic screening programs to identify early CBD may be considered a form of two-stage screening. In the first stage, relatively simple procedures such as the LPT are applied to large populations. Then, some population members, typically those with BeS, undergo more extensive testing to determine whether they have early CBD. This testing often includes invasive tests such as bronchoscopy and biopsy.10 This should be considered a form of “second-stage screening” when the extensive testing is performed for a patient derived from a screening program rather than individually referred because of symptoms or an identified radiographic abnormality. (Clinical judgment is, however, involved in both the first- and second-stage screening processes.) 3. Extensive data concerning screening and diagnosis have been systematically collected by federal agencies (eg, the US Department of Energy and National Institute for Occupational Safety and Health, Washington DC), academic clinics (eg, at the National Jewish Medical Center), and a novel data/tissue repository (Beryllium BioBank [BBB]).11 4. Because CBD may develop many years after an exposure, the primary screening test must be repeated periodically (eg, annually), even if an individual is no longer exposed. The invasive secondary testing may also be repeated, especially because of the potential for false-negative biopsy tests and progression from BeS to CBD after many years. 5. Nevertheless, there are significant opportunities for improvement in the screening process and a need to demonstrate costeffectiveness.12 The BBB11 is a clinical data and biologic specimen repository from five collaborating centers. Two related articles describe analyses concerning the occupational exposure history and LPT testing.13,14 This article describes the potential usefulness of relatively simple modeling techniques for potentially incrementally improving screening program design. Although this article focuses on BeE workers, the principles and findings are likely applicable to many other screening programs that involve repetitive testing over many years and/or a combination of a simple first-stage screening test to identify persons who should undergo more extensive and expensive secondary testing. For example, rapid computed tomographic scanning of smokers identifies those who need more extensive additional procedures. Similarly, several occupational asthma screening programs in Canada and the United States use questionnaires as a first stage screening to identify persons who then are evaluated in more detail.15,16
JOEM r Volume 56, Number 8, August 2014 Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
861
JOEM r Volume 56, Number 8, August 2014
Harber and Su
METHODS Two aspects of screening were considered—implications repetitive LPT testing on accuracy of classification as BeS and guiding selection of the optimal interval for extensive clinical testing of persons with BeS. The approaches are conceptual, but they have been suggested by the empirical analyses of exposure history and LPT data from the BBB.13,14
Repeated LPT Testing The main purpose of LPT testing in programs for worker surveillance is to identify individuals with BeS. Once an individual has been classified as having BeS, he or she is typically entered into the more intensive testing program, termed “second stage” described in the introductory paragraphs. According to the current practice, BeS is defined by having “positive” LPT tests on at least two different occasions. Each LPT is interpreted as positive or negative without regard to the history of all prior tests performed on the patient. The criterion for considering an individual test to positive generally has the goal of limiting false positives to 5%. Several metrics are commonly used to describe test performance in screening programs. The purpose of LPT is to identify truly sensitized persons (who have either BeS or CBD). A simplified test shows the effect of repetitive testing cycles on these metrics on the basis of the following three characteristics: (1) test specificity (proportion of true negatives found to be negative), (2) sensitivity (proportion of true positives reported as positive), and (3) prevalence (proportion of the target population with BeS or CBD). The proportion of persons who would have zero, one, or at least two positive results is calculated on the basis of the number of cycles of tests performed on the individual. Spreadsheet calculations (Microsoft Excel) implementing the binomial formula facilitated these calculations, which were done separately for the true negative (BeE) and true positive (BeS + CBD) persons. PSensitized =
n m
(Sensitivity)m (1 − Sensitivity)n−m
For n testing cycles, the probability of exactly m positive tests among individuals who are truly sensitized is determined by the formula; comparable calculations were performed for those who were truly not sensitized. The correct classifications are those in which the total number of positive tests is 2 or more among the true positives and less than 2 among the true negatives. The overall accuracy is determined by a weighted average of the accurate proportion between these two groups, with weights equal to the prevalence.
Selection of Testing Interval for Persons With BeE Clinicians and screening program directors must decide which persons classified as BeE should undergo the extensive testing necessary to determine whether they have early CBD. This testing must be repeated periodically because of potential false-negative evaluations and because of new cases developing over time. In this article, we consider the extensive testing of the individual detected solely because of the screening program to be part of the program, hence using the term “second-stage screening.” This does not apply to diagnostic evaluations performed for individual patients referred because of symptoms or radiographic abnormalities. Changes in disease status over time may be represented by Markov models. This approach uses a set of classes on the basis of disease status. An individual may move from one class to another over time. A Markov model was developed to illustrate potential effects of alternative second-stage screening strategies as illustrated in Fig. 1. Markov models represent transition between mutually exclusive states over time. The model incorporates the following five states: 862
FIGURE 1. Markov model for screening interval. The figure illustrates Markov model transitions. CBD, Advanced-CBD; BeNS, beryllium-nonsensitized; BeS, beryllium-sensitized; CBD, chronic beryllium disease; Detected, E-CBD cases detected by second-stage screening. The arrows represent potential state transitions. Calculations were performed for various screening intervals.
BeNS, BeS, Early-CBD, Advanced-CBD, and Detected. “EarlyCBD” corresponds to the state during which second-stage screening is most useful for detecting the disease before it is otherwise evident and while it is amenable to preventive therapy (ie, the goal of screening). “Detected” represents individuals whose early CBD was detected by a screening program. This classification differs from the clinical classification because it represents the “true status” of the individual, even if not known to the clinician. (For example, only those in the “Detected” group would be classified as CBD in a clinical program. Because the art of clinical diagnosis is not 100% accurate, some persons with true CBD may be clinically classified as BeS and conversely.) Transition probabilities express the likelihood of changing from one state to another in a unit of time (eg, from BeS to EarlyCBD). Calculations were performed using a commercially available decision-modeling system (TreeAge Pro Suite version 2012, Treeage Software, Inc, Williamstown, MA). Transition probabilities reflect the likelihood of changing states within any time period. State transitions are calculated 10 times per year (deci-years) over a 40-year (400 deci-years) horizon. Transition probabilities for BeS to Early-CBD and Early-CBD to Advanced-CBD were selected to reflect cross-sectional prevalence of the stages in the published population data, as discussed in an earlier publication.17 The following overall transition probabilities were used: BeS → Early (mild) CBD = 0.001 per deci-year and Early (mild) CBD → CBD Advanced = 0.025 per deci-year. For this simple illustration, second-stage screening is assumed to be specific (BeS → Detected = 0) and sensitive (Early [mild] CBD → Detected = 1.0 in years with screening, 0 otherwise). Because this illustration focuses on second-stage screening, transition to BeS is not including calculations, but NeNS is included in the figure to illustrate that a similar approach might be used for first-stage (eg, LPT) screening. The BeS population is considered to include the following two groups: persons at “typical risk” of progression to CBD and persons with “increased risk” of such progression. The influence of factors affecting personal risk (eg, LPT results and exposure characteristics) is expressed by the risk factor adjustment (RFA), which represents the ratio of the “increased risk” subjects’ transition probability to the “typical risk” subjects’ transition probability. The effect of personal risk factors (eg, LPT result and exposure) on likelihood of progression is modeled by multiplying the composite (whole population) transition probabilities #1 and #2 shown earlier by the RFA as appropriate. For “increased risk” subjects, RFA = 1.5; for “lower risk” subjects, RFA = 0.8. These values were chosen to illustrate the effect of relatively small differences in risk as might be achievable by examining factors such as the LPT results and exposure history. The selected RFA illustration is intentionally much lower than the effect of genetic factors; although such genetic factors may have
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 8, August 2014
odds ratios as high as 20,18 information about a BeS person’s genetic status is not available to clinicians deciding whether to perform a bronchoscopy or similar second-stage screening procedure. Screening of BeS persons is applied at several possible screening frequencies; when the stage is an even multiple of the screening frequency, the appropriate Early (mild) CBD → Detected transition probability is 1.0, but otherwise is zero. For simplicity of this illustration, second stage is assumed to be both sensitive and specific. Screening Detected and Advanced-CBD are considered absorbing states, from which no one leaves. Results are calculated over a 40-year horizon on the basis of 10-time slices (deci-years) per year. Outcomes of interest are the number of Advanced-CBD cases, the number of people successfully detected with screening (“Detected”), cost (on the basis of the number of screenings performed), and cost-effectiveness (cost divided by the number “Detected”). Both overall and incremental results are calculated. Outcomes are calculated for several different screening intervals for both “increased risk” and “typical risk” people. Then, the potential benefit of implementing personalized screening on the basis of personal risk factors is estimated using a weighted combination of less frequent screening for low-risk individuals and more frequent screening for high-risk individuals. “Saved” cases are based on a comparison to a 6-year screening interval with no risk differentiation. For descriptive purposes, model results are based on the initial cohort of 1000 persons without replacement of those who move out of the BeS state. Costs were accumulated over the 40-year time span, and total costs are used in calculating effect of calculations. “Cost” for second-stage screening varies considerably on the basis of regional cost differences and contractual agreements (eg, the payments for a spirometry or bronchoscopy procedure vary widely and the relationship between actual cost and charges for many health services is unknown in the United States). In addition, there is variation in
Beryllium Biobank 3
the proportion of second-stage examinations that include expensive procedures such as computerized axial tomography scans and bronchoscopy. This represents both temporal changes in pulmonary practices such as relying more on computerized axial tomography scans than on biannual bronchoscopy and the potential influence of symptoms on clinicians’ recommendations. For calculation purposes, an estimate of an average $2000 cost for radiography, clinical examination, and pulmonary function testing at each secondary screen and bronchoscopy for some is used. This value was selected on the basis of review of Medicare resource-based relative value scale payments by the Current Procedural Terminology code for procedures and on ad hoc discussion with practicing bronchoscopists. This is a lower bound for cost because it includes a hybrid of individuals who have only radiography and pulmonary function testing as well as persons undergoing bronchoscopy. The cost does not include any cost for nonmonetary consequences of second-stage screening (eg, potential complications of testing and discomfort from procedures), nor does it include indirect costs such as lost work time.
RESULTS Repeated Testing Using LPT The potential effects of repeated testing using LPT are shown in Fig. 2. The figure shows the overall accuracy using the traditional criterion that beryllium sensitization is present if the individual has two or more positive tests. Panels A to E each represent calculations for one set of parameters for sensitivity, specificity, and prevalence, and panel F compares the overall accuracy for the five sets of parameters. The true prevalence rate is specified at 5% in parameter sets A to D and 20% for set E. Panels A to E also show results separately for those who are truly sensitized and those who are truly not sensitized. Sets of illustrative parameters were chosen to reflect reasonable ranges expected in practice:
FIGURE 2. Effect of repeated LPT testing. The vertical axis represents the proportion of correct classifications, and the x axis represents the number of times the test is repeated. Parameters (sensitivity, specificity, and prevalence) are described in the Methods section. In panels A to E, the solid black line represents the overall percentage, dotted line represents the percentage of correct classification among persons who are truly not sensitized, and the dashed line represents the percentage of correct classification among those who are truly sensitized. Panel F summarizes results for the correct classification from each of the panels. LPT, lymphocyte proliferation test. C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
863
JOEM r Volume 56, Number 8, August 2014
Harber and Su
Parameters
A
B
C
D
E
Sensitivity Specificity Prevalence
95% 90% 5%
75% 90% 5%
80% 80% 5%
70% 70% 5%
75% 90% 20%
markedly worse cost-effectiveness than the other strategies. Figure 3 graphically displays the prevented cases of Advanced-CBD (when compared with a fixed 6-year screening interval) and program cost. Several of the hybrid screening strategies-–in which the secondary screening frequency is adjusted for personal risk-–have the greatest yield of Advanced-CBD cases saved per dollar invested.
DISCUSSION Repeating the tests increases accuracy initially, but the overall accuracy later declines as more cycles of testing are conducted. As long as the prevalence is less than 50%, extending testing will eventually reduce accuracy. This adverse effect is increasingly important with more testing cycles, lower prevalence, and lower specificity. With an optimal situation of 95% specificity and 80% sensitivity for the individual LPT, total accuracy remains more than 95% for as many as seven cycles. For an optimal situation with 95% specificity for each individual test, accuracy remains more than 95% for the first seven cycles. Nevertheless, even minor reduction of specificity to 90% significantly reduces overall accuracy after the test is done four times (panels A, B, and E). With significantly lower sensitivity and specificity (eg, 70% for each), only two thirds are accurate with for test cycles and only one third are accurate when testing is repeated seven times. Panels C and D may reflect situations compatible with empirical data.13,14
Screening programs should optimize the net benefit to individuals and maximize cost-effectiveness. Existing screening program methods have been effective for classifying cases into two well-accepted categories and for detection of early CBD. Nevertheless, this article suggests the importance of evaluating two possible enhancements to commonly used practices. First, the number of positive LPT tests needed to consider an individual to be sensitized (BeS)
Testing Interval for Diagnosing CBD Among Persons Classed as BeS The potential effect of alternative screening intervals and adjusting screening intervals considering personal factors associated with an increased risk of progression is summarized in Table 1. The estimated number of detected early cases (Detected), advanced cases (Advanced-CBD), and program costs over a 40-year horizon is shown. In addition to the overall cost-effectiveness, the table shows incremental cost-effectiveness (number of AdvancedCBD cases prevented for each additional $1,000,000 allocated). The total cost ranges from $10 million to $68 million, depending on the frequency of second-stage screening and whether frequency is adjusted according to personal risk. Annual secondary screening of all BeS subjects has the greatest yield of Detected cases but has
FIGURE 3. CBD cases saved per $1 million. The figure illustrates A-CBD cases avoided per million dollar program cost (compared with uniform 6-year schedule) with alternate second-stage screening intervals. Open diamonds represent same schedule for all and filled circles represent adjusted by estimated progression risk (see Table 1). CBD, chronic beryllium disease.
TABLE 1. Effect of Alternative Screening Schedules* Screening Schedule (Years) Lower risk Higher risk Total detected Total advanced-CBD Total cost (million) Cost-effectiveness Sensitivity, % Incremental cost (million) Incremental cost/CBD avoided # Case saved/$1 million # Detected/$1 million CBD saved/$1 million
1 1
2 1
2 2
263 29 $68 $0.26 90 $26 $1.51 0.7 −0.3 1.76
255 46 $42 $0.16 85 $8 $0.50 2.0 −1.4 2.45
244 62 $34 $0.14 80 $1 −$1.36 −0.7 −3.6 2.56
3 1 Effects 239 61 $32 $0.14 80 $9 $0.26 3.9 −4.5 2.68
6 1
3 2
3 3
6 3
6 6
199 96 $23 $0.12 67 $1 $0.01 78.9 103.6 2.21
277 156 $23 0.08 64 $2 −$0.14 −7.2 −14.3 −0.37
249 142 $21 $0.08 64 $8 −$0.43 −2.3 −9.5 0.30
175 124 $13 $0.07 59 $3 $0.11 9.0 −9.4 1.87
150 148 $10 $0.07 50 — — — — —
*The table summarizes estimated effect of alternative screening approaches (for people in BeS category) on the basis of approximately 50% greater risk of progression in 25% of subjects. Results are based on 1000 initial program participants and average cost of $2000 per visit. Detected: detected by screening with preclinical CBD; cost-effectiveness: ratio of cost to number Detected; sensitivity: proportion of people with CBD who are Detected. Incremental costs are based on comparing a program to the next lower-cost design, expressed as total cost increment and cost per Advanced-CBD case avoided. CBD saved: number of advanced CBD cases compared with the number of cases using fixed 6-year second-stage screening intervals divided by cost in millions. See the text and supplement for details. CBD, chronic beryllium disease.
864
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 8, August 2014
may be adjusted by the number of tests performed. Second, occupational history and LPT results may be used to adjust the frequency with which persons with BeS undergo repeated extensive testing to determine whether they have CBD. These approaches may increase both the accuracy and the cost-effectiveness of programs.
Repeated Testing Using LPT for Identifying BeS The beryllium LPT is used to identify individuals with immunologic sensitization among workers who have had beryllium exposure. Commonly used interpretation strategies have the goal of limiting false positives to 5%.19,20 For an LPT, a test result is a continuous variable, called the stimulation index (SI). The SI for each test is considered positive if the SI is greater than a criterion value, which is chosen to limit false positives to 5%. Nevertheless, BeE workers in screening programs have the test performed many times, often on an annual basis. The test is imperfect, having both false positives and false negatives. Sensitization is operationally defined by having two or more positive LPTs. It is logical that the likelihood of having at least two positive tests increases as the number of tests increases. Unlike screening for some other disorders, there is no generally accepted clinical “criterion standard” to define sensitization or CBD independent of the test procedures per se. Therefore, simple measures such as sensitivity/specificity are less applicable when screening testing is applied multiple times. As shown in Fig. 2, the overall accuracy of a sequence of tests in a population including both persons with and without true sensitization depends on the number of times the test is repeated. Therefore, as applied to beryllium workers screening, the goal of limiting false positives to 5% is not achieved by the common algorithm. For declaring an individual to have BeS, it seems preferable to consider the number of positives and the number of tests performed rather than simply considering someone sensitized if they have two positive tests. (Good clinicians may consider this implicitly, particularly when the SIs of the positive tests are barely more than the criterion values.) These calculations have been intentionally simplified for illustrative purposes. They do not consider borderline or uninterpretable test results.21 If borderline tests were considered, the false-positive proportion would probably increase because the current clinical practice considers one positive and one borderline to be almost equivalent to two positives. If the prevalence of sensitization is less than 50%, this would, therefore, decrease the overall estimate of accuracy. The calculations assume that the overall test characteristics (specificity and sensitivity) remain constant over the testing cycles and that each test has the same outcome probabilities. This may also lead to overestimating accuracy if there is autocorrelation of results within individuals. Finally, this illustration assumes that the prevalence of sensitization in the tested population remains constant over time. The prevalence of previously undetected BeS is likely to actually decline after the first several years of a screening program because the incidence of new cases is likely to be lower than the number of individuals who are removed from the screenee group. Over time, persons with genetic susceptibility and/or particularly large exposures are likely to have been detected by screening or development of symptoms, so that the residual proportion of undetected true positive declines in the screen population. Analysis of data from the BBB showed that latencies of those who remained in BeE were significantly higher than those who had progressed, implying that the rate of progression declines.17 Furthermore, persons who progress to CBD may be identified outside of the screening program because of radiological or clinical findings, also decreasing the residual proportion of persons with true sensitization among screenees. In addition, there is limited basis for the common medical practice of choosing the cutpoint (criterion value) for a 5% falsepositive proportion on the basis of methods developed for agricul-
Beryllium Biobank 3
tural research.22 Rather, the optimal cutpoint may depend on the relative costs and benefits of false positives and false negatives. A related analysis of BBB data showed that a criterion value of 6.0 performed better than the usual criterion value of 3.0 for predicting progression to CBD.14 Additional empirical studies are needed to select the optimal approach. For example, classification in BeS might require at least three positive tests in a person with a large number of tests. Alternatively, a statistical calculation that includes the absolute value of each SI rather than a simple positive/negative classification may be preferable.
Testing Intervals for Extensive Evaluation of BeS Cases Pulmonary studies of data from the BBB do not create a comprehensive progression risk index for each individual, but they have demonstrated that personal characteristics such as peak exposure weighted hours and actual LPT values affect progression likelihood (ie, help determine who is likely to benefit from extensive testing).13,14 The model in this report illustrates the potential utility of combining empirically determined data and a model of the course of disease progression.17 The preliminary analyses considered only the following two items: exposure history and LPT results. Nevertheless, most participants in early detection programs have considerably more data available such as multiple points in time for LPT data, radiography, and lung function testing. Therefore, analyses considering additional factors would probably be even more effective than this study’s demonstrated ability to predict progression and identify who requires intensive second-stage screening. Although genetic factors significantly affect risk,18 an individual’s genetic characteristics are not known to clinicians for practical and ethical reasons. Previous beryllium disease–related analytic models have considered basing pre-placement screening on genetic factors23 and optimizing the primary screening (LPT) interval.24 The analysis presented in this brief article focuses on second-stage screening of BeE workers. For such workers, screening costs of the first-stage screening (detecting those who have transitioned from BeE to BeS) are primarily driven by the large number of exposed individuals who undergo screening5 with a relatively simple and inexpensive test (LPT). Conversely, for the second-stage screening (detecting those who have transitioned from BeS to CBD), costs per individual screening episode are considerably greater, but the number of potential persons requiring second-stage screening is lower. Similarly, the benefit of detecting early CBD and the harm of failure to detect early CBD in a timely fashion are potentially greater. Although long-term, longitudinal, large-scale, randomized controlled trials are optimal for establishing clinical and public health policy, program design and clinical decision-making must often use approaches such as quantitative modeling. Extremely large study populations are necessary for relatively uncommon outcomes, and very long empirical trials are needed for long latency diseases such as CBD. The many possible permutations of program design (eg, wide range of possible screening intervals and methods for partitioning risk among members) preclude trials of each possibility. Perhaps most importantly, public-health and clinical practice decisions must be made without delaying for such long-term trials, even if they were feasible. Therefore, simulation modeling may be used as an effective tool for guiding decision making in the absence of randomized controlled trials. More complex simulation models in the future may consider sensitivity analyses for second-stage screening sensitivity and specificity, temporal changes in screening characteristics, and sensitivity analyses for the transition probabilities and other parameters incorporated. The transition model and calculations were intentionally narrowly focused because the goal of this article was simply to
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
865
JOEM r Volume 56, Number 8, August 2014
Harber and Su
illustrate the potential long-term implications of adjusting screening intervals. Furthermore, a broader array of outcome utilities may be used in addition to the cost-effectiveness of screening per se (eg, consider other costs and benefits of detection including work loss, income loss, and compensation costs).
CONCLUSIONS Current algorithms underlying beryllium worker screening programs have been effective. Nevertheless, their accuracy and costeffectiveness may be incrementally improved by adjusting algorithms for each person’s available data. For example, the number of positive LPT tests to be classified as sensitized may increase in people who have had many tests. The interval between repeated extensive testing of persons who are sensitized may be adjusted on the basis of factors affecting the likelihood of progression to CBD.
10. 11.
12.
13.
14.
ACKNOWLEDGMENTS The authors thank Gabriela Alongi and the Beryllium BioBank Steering Committee for invaluable assistance. The authors also thank the US DOE Beryllium BioBank for the assistance in this project. The Beryllium BioBank was developed with funding from the US Department of Energy.
15.
16.
17.
REFERENCES 1. Newman LS, Mroz MM, Maier LA, Daniloff EM, Balkissoon R. Efficacy of serial medical surveillance for chronic beryllium disease in a beryllium machining plant. J Occup Environ Med. 2001;43:231–237. 2. Rossman MD. Justification for screening for chronic beryllium disease: closer to reality. Eur Respir J. 2008;32:543–544. 3. Sackett HM, Maier LA, Silveira LJ, et al. Beryllium medical surveillance at a former nuclear weapons facility during cleanup operations. J Occup Environ Med. 2004;46:953–961. 4. Takaro TK, Pepper L. Screening for chronic beryllium disease. Am J Respir Crit Care Med. 2005;172:1230. 5. Henneberger PK, Goe SK, Miller WE, Doney B, Groce DW. Industries in the United States with airborne beryllium exposure and estimates of the number of current workers potentially exposed. J Occup Environ Hyg. 2004;1:648–659. 6. Stange AW, Hilmas DE, Furman FJ, Gatliffe TR. Beryllium sensitization and chronic beryllium disease at a former nuclear weapons facility. Appl Occup Environ Hyg. 2001;16:405–417. 7. Henneberger PK, Cumro D, Deubner DD, Kent MS, McCawley M, Kreiss K. Beryllium sensitization and disease among long-term and short-term workers in a beryllium ceramics plant. Int Arch Occup Environ Health. 2001;74:167– 176. 8. Donovan EP, Kolanz ME, Galbraith DA, Chapman PS, Paustenbach DJ. Performance of the beryllium blood lymphocyte proliferation test based on a
866
9.
18.
19.
20.
21.
long-term occupational surveillance program. Int Arch Occup Environ Health. 2007;81:165–178. Kreiss K, Mroz MM, Newman LS, Martyny J, Zhen B. Machining risk of beryllium disease and sensitization with median exposures below 2 micrograms/m3. Am J Ind Med. 1996;30:16–25. Maier LA. Beryllium health effects in the era of the beryllium lymphocyte proliferation test. Appl Occup Environ Hyg. 2001;16:514–520. Colorado School of Public Health. Clinical data & questionnaire. Available at: http://www.ucdenver.edu/academics/colleges/PublicHealth/research/ ResearchProjects/bbb/Pages/Clinical-Data.aspx. Published 2012. Accessed May 29, 2012. Borak J, Woolf SH, Fields CA. Use of beryllium lymphocyte proliferation testing for screening of asymptomatic individuals: an evidence-based assessment. J Occup Environ Med. 2006;48:937–947. Harber P, Su J, Alongi G. Exposure factors associated with chronic beryllium disease development in Beryllium BioBank participants. J Occup Environ Med. 2014;56:852–856. Harber P, Su J, Alongi G. Beryllium BioBank: 2. Lymphocyte proliferation testing. J Occup Environ Med. 2014;56:857–860. National Lung Screening Trial Research Team Aberle DR, Adams AM, Berg CD, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365:395–409. Pralong JA, Moullec G, Suarthana E, et al. Screening for occupational asthma by using a self-administered questionnaire in a clinical setting. J Occup Environ Med. 2013;55:527–531. Harber P, Bansal S, Balmes J. Progression from beryllium exposure to chronic beryllium disease: an analytic model. Environ Health Perspect. 2009;117:970–974. Van Dyke MV, Martyny JW, Mroz MM, et al. Risk of chronic beryllium disease by HLA-DPB1 E69 genotype and beryllium exposure in nuclear workers. Am J Respir Crit Care Med. 2011;183:1680–1688. Frome EL, Newman LS, Cragle DL, Colyer SP, Wambach PF. Identification of an abnormal beryllium lymphocyte proliferation test. Toxicology. 2003;183:39–56. Frome EL, Smith MH, Littlefield LG, Neubert RL, Colyer SP. Statistical methods for the blood beryllium lymphocyte proliferation test. Environ Health Perspect. 1996;104(suppl 5):957–968. Middleton DC, Fink J, Kowalski PJ, Lewin MD, Sinks T. Optimizing BeLPT criteria for beryllium sensitization. Am J Ind Med. 2008;51:166–172.
22. Fisher R. The arrangement of field experiments. J Min Agric G Br. 1926; 33:503–513. 23. Bartell SM, Ponce RA, Takaro TK, Zerbe RO, Omenn GS, Faustman EM. Risk estimation and value-of-information analysis for three proposed genetic screening programs for chronic beryllium disease prevention. Risk Anal. 2000;20:87–99. 24. Judd NL, Griffith WC, Takaro T, Faustman EM. A model for optimization of biomarker testing frequency to minimize disease and cost: example of beryllium sensitization testing. Risk Anal. 2003;23:1211– 1220.
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
Beryllium Biobank 3: Considerations for Improving Chronic Beryllium Disease Screening Philip Harber, MD, MPH and Jing Su, MS
Objective: To optimize beryllium worker screening. Methods: Berylliumexposed persons are classified as beryllium-exposed, beryllium-sensitized (BeS), or chronic beryllium disease. Implications of defining BeS by two or more positive lymphocyte proliferation tests (LPTs) were investigated with a simple binomial model. The potential effect of adjusting the interval for repeated intensive testing to detect chronic beryllium disease among persons with BeS was assessed with a Markov model. Results: Accuracy of properly identifying BeS is reduced as the number of repeated tests increases. Markov simulation illustrates that adjusting second-stage screening intervals on the basis of personal risk may significantly affect cost-effectiveness. Conclusions: The criteria for classification as BeS should be adjusted depending on the number of LPTs performed. Modifying the interval for repeated intensive testing on the basis of each worker’s data can improve cost-effectiveness.
C
hronic beryllium disease (CBD) is an immunologically mediated disease, which can lead to severe pulmonary dysfunction or death. Secondary prevention is detection of illness in an early stage at which intervention can prevent worsening, and screening is a systematic program to identify cases earlier than would occur through typical diagnostic approaches. The utility of systematically identifying early CBD is favored for several reasons as follows: 1. The disease develops sequentially. Beryllium-exposed (BeE) workers may develop immunologic beryllium sensitization (BeS) without disease. Then, some with BeS develop actual CBD. Initially, early CBD is mild and largely asymptomatic. Nevertheless, many persons with early CBD will progress to advanced CBD. Subjects must pass through BeS before developing disease. 2. Treatment of early CBD with corticosteroids or other immunosuppressive agents can be effective in preventing progression to advanced symptomatic CBD. 3. A noninvasive procedure (blood lymphocyte proliferation test [LPT]) is useful for detecting BeS. Extensive experience has shown the utility of the test, and automation has made its cost reasonable. Studies have shown that screening effectively identifies individuals needing treatment.1–4 4. There is a large population at risk, estimated to include 300,000 persons in the United States.5 5. A large cohort may be specifically identified because many workers in the US nuclear industry, particularly in work associated with weapons production, have been exposed and are potentially at risk.6 6. An existing organizational structure (the US Department of Energy) can organize screening programs. In addition, a US law From the Community, Environment, and Policy Division, Mel and Enid Zuckerman College of Public Health, University of Arizona, Tucson. Sources of funding: None. Conflicts of interest: None declared. Address correspondence to: Philip Harber, MD, MPH, Community, Environment, and Policy Division, Mel and Enid Zuckerman College of Public Health, University of Arizona, 1656 E. Mabel St, Room 112, Tucson, AZ 85724 ([email protected]). C 2014 by American College of Occupational and Environmental Copyright Medicine DOI: 10.1097/JOM.0000000000000200
provides specific benefits encouraging identification of berylliumrelated diseases in current and past nuclear weapons workers (Energy Employees Occupational and Illness Compensation Program). Several factors make CBD of particular interest when considering design of screening and secondary prevention programs. These factors include the following: 1. Systematic efforts to identify early CBD must occur over many years because CBD can develop very soon after initiating exposure7–9 and also develop with very long latency, often many years after the most recent exposure.3 2. Systematic screening programs to identify early CBD may be considered a form of two-stage screening. In the first stage, relatively simple procedures such as the LPT are applied to large populations. Then, some population members, typically those with BeS, undergo more extensive testing to determine whether they have early CBD. This testing often includes invasive tests such as bronchoscopy and biopsy.10 This should be considered a form of “second-stage screening” when the extensive testing is performed for a patient derived from a screening program rather than individually referred because of symptoms or an identified radiographic abnormality. (Clinical judgment is, however, involved in both the first- and second-stage screening processes.) 3. Extensive data concerning screening and diagnosis have been systematically collected by federal agencies (eg, the US Department of Energy and National Institute for Occupational Safety and Health, Washington DC), academic clinics (eg, at the National Jewish Medical Center), and a novel data/tissue repository (Beryllium BioBank [BBB]).11 4. Because CBD may develop many years after an exposure, the primary screening test must be repeated periodically (eg, annually), even if an individual is no longer exposed. The invasive secondary testing may also be repeated, especially because of the potential for false-negative biopsy tests and progression from BeS to CBD after many years. 5. Nevertheless, there are significant opportunities for improvement in the screening process and a need to demonstrate costeffectiveness.12 The BBB11 is a clinical data and biologic specimen repository from five collaborating centers. Two related articles describe analyses concerning the occupational exposure history and LPT testing.13,14 This article describes the potential usefulness of relatively simple modeling techniques for potentially incrementally improving screening program design. Although this article focuses on BeE workers, the principles and findings are likely applicable to many other screening programs that involve repetitive testing over many years and/or a combination of a simple first-stage screening test to identify persons who should undergo more extensive and expensive secondary testing. For example, rapid computed tomographic scanning of smokers identifies those who need more extensive additional procedures. Similarly, several occupational asthma screening programs in Canada and the United States use questionnaires as a first stage screening to identify persons who then are evaluated in more detail.15,16
JOEM r Volume 56, Number 8, August 2014 Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
861
JOEM r Volume 56, Number 8, August 2014
Harber and Su
METHODS Two aspects of screening were considered—implications repetitive LPT testing on accuracy of classification as BeS and guiding selection of the optimal interval for extensive clinical testing of persons with BeS. The approaches are conceptual, but they have been suggested by the empirical analyses of exposure history and LPT data from the BBB.13,14
Repeated LPT Testing The main purpose of LPT testing in programs for worker surveillance is to identify individuals with BeS. Once an individual has been classified as having BeS, he or she is typically entered into the more intensive testing program, termed “second stage” described in the introductory paragraphs. According to the current practice, BeS is defined by having “positive” LPT tests on at least two different occasions. Each LPT is interpreted as positive or negative without regard to the history of all prior tests performed on the patient. The criterion for considering an individual test to positive generally has the goal of limiting false positives to 5%. Several metrics are commonly used to describe test performance in screening programs. The purpose of LPT is to identify truly sensitized persons (who have either BeS or CBD). A simplified test shows the effect of repetitive testing cycles on these metrics on the basis of the following three characteristics: (1) test specificity (proportion of true negatives found to be negative), (2) sensitivity (proportion of true positives reported as positive), and (3) prevalence (proportion of the target population with BeS or CBD). The proportion of persons who would have zero, one, or at least two positive results is calculated on the basis of the number of cycles of tests performed on the individual. Spreadsheet calculations (Microsoft Excel) implementing the binomial formula facilitated these calculations, which were done separately for the true negative (BeE) and true positive (BeS + CBD) persons. PSensitized =
n m
(Sensitivity)m (1 − Sensitivity)n−m
For n testing cycles, the probability of exactly m positive tests among individuals who are truly sensitized is determined by the formula; comparable calculations were performed for those who were truly not sensitized. The correct classifications are those in which the total number of positive tests is 2 or more among the true positives and less than 2 among the true negatives. The overall accuracy is determined by a weighted average of the accurate proportion between these two groups, with weights equal to the prevalence.
Selection of Testing Interval for Persons With BeE Clinicians and screening program directors must decide which persons classified as BeE should undergo the extensive testing necessary to determine whether they have early CBD. This testing must be repeated periodically because of potential false-negative evaluations and because of new cases developing over time. In this article, we consider the extensive testing of the individual detected solely because of the screening program to be part of the program, hence using the term “second-stage screening.” This does not apply to diagnostic evaluations performed for individual patients referred because of symptoms or radiographic abnormalities. Changes in disease status over time may be represented by Markov models. This approach uses a set of classes on the basis of disease status. An individual may move from one class to another over time. A Markov model was developed to illustrate potential effects of alternative second-stage screening strategies as illustrated in Fig. 1. Markov models represent transition between mutually exclusive states over time. The model incorporates the following five states: 862
FIGURE 1. Markov model for screening interval. The figure illustrates Markov model transitions. CBD, Advanced-CBD; BeNS, beryllium-nonsensitized; BeS, beryllium-sensitized; CBD, chronic beryllium disease; Detected, E-CBD cases detected by second-stage screening. The arrows represent potential state transitions. Calculations were performed for various screening intervals.
BeNS, BeS, Early-CBD, Advanced-CBD, and Detected. “EarlyCBD” corresponds to the state during which second-stage screening is most useful for detecting the disease before it is otherwise evident and while it is amenable to preventive therapy (ie, the goal of screening). “Detected” represents individuals whose early CBD was detected by a screening program. This classification differs from the clinical classification because it represents the “true status” of the individual, even if not known to the clinician. (For example, only those in the “Detected” group would be classified as CBD in a clinical program. Because the art of clinical diagnosis is not 100% accurate, some persons with true CBD may be clinically classified as BeS and conversely.) Transition probabilities express the likelihood of changing from one state to another in a unit of time (eg, from BeS to EarlyCBD). Calculations were performed using a commercially available decision-modeling system (TreeAge Pro Suite version 2012, Treeage Software, Inc, Williamstown, MA). Transition probabilities reflect the likelihood of changing states within any time period. State transitions are calculated 10 times per year (deci-years) over a 40-year (400 deci-years) horizon. Transition probabilities for BeS to Early-CBD and Early-CBD to Advanced-CBD were selected to reflect cross-sectional prevalence of the stages in the published population data, as discussed in an earlier publication.17 The following overall transition probabilities were used: BeS → Early (mild) CBD = 0.001 per deci-year and Early (mild) CBD → CBD Advanced = 0.025 per deci-year. For this simple illustration, second-stage screening is assumed to be specific (BeS → Detected = 0) and sensitive (Early [mild] CBD → Detected = 1.0 in years with screening, 0 otherwise). Because this illustration focuses on second-stage screening, transition to BeS is not including calculations, but NeNS is included in the figure to illustrate that a similar approach might be used for first-stage (eg, LPT) screening. The BeS population is considered to include the following two groups: persons at “typical risk” of progression to CBD and persons with “increased risk” of such progression. The influence of factors affecting personal risk (eg, LPT results and exposure characteristics) is expressed by the risk factor adjustment (RFA), which represents the ratio of the “increased risk” subjects’ transition probability to the “typical risk” subjects’ transition probability. The effect of personal risk factors (eg, LPT result and exposure) on likelihood of progression is modeled by multiplying the composite (whole population) transition probabilities #1 and #2 shown earlier by the RFA as appropriate. For “increased risk” subjects, RFA = 1.5; for “lower risk” subjects, RFA = 0.8. These values were chosen to illustrate the effect of relatively small differences in risk as might be achievable by examining factors such as the LPT results and exposure history. The selected RFA illustration is intentionally much lower than the effect of genetic factors; although such genetic factors may have
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 8, August 2014
odds ratios as high as 20,18 information about a BeS person’s genetic status is not available to clinicians deciding whether to perform a bronchoscopy or similar second-stage screening procedure. Screening of BeS persons is applied at several possible screening frequencies; when the stage is an even multiple of the screening frequency, the appropriate Early (mild) CBD → Detected transition probability is 1.0, but otherwise is zero. For simplicity of this illustration, second stage is assumed to be both sensitive and specific. Screening Detected and Advanced-CBD are considered absorbing states, from which no one leaves. Results are calculated over a 40-year horizon on the basis of 10-time slices (deci-years) per year. Outcomes of interest are the number of Advanced-CBD cases, the number of people successfully detected with screening (“Detected”), cost (on the basis of the number of screenings performed), and cost-effectiveness (cost divided by the number “Detected”). Both overall and incremental results are calculated. Outcomes are calculated for several different screening intervals for both “increased risk” and “typical risk” people. Then, the potential benefit of implementing personalized screening on the basis of personal risk factors is estimated using a weighted combination of less frequent screening for low-risk individuals and more frequent screening for high-risk individuals. “Saved” cases are based on a comparison to a 6-year screening interval with no risk differentiation. For descriptive purposes, model results are based on the initial cohort of 1000 persons without replacement of those who move out of the BeS state. Costs were accumulated over the 40-year time span, and total costs are used in calculating effect of calculations. “Cost” for second-stage screening varies considerably on the basis of regional cost differences and contractual agreements (eg, the payments for a spirometry or bronchoscopy procedure vary widely and the relationship between actual cost and charges for many health services is unknown in the United States). In addition, there is variation in
Beryllium Biobank 3
the proportion of second-stage examinations that include expensive procedures such as computerized axial tomography scans and bronchoscopy. This represents both temporal changes in pulmonary practices such as relying more on computerized axial tomography scans than on biannual bronchoscopy and the potential influence of symptoms on clinicians’ recommendations. For calculation purposes, an estimate of an average $2000 cost for radiography, clinical examination, and pulmonary function testing at each secondary screen and bronchoscopy for some is used. This value was selected on the basis of review of Medicare resource-based relative value scale payments by the Current Procedural Terminology code for procedures and on ad hoc discussion with practicing bronchoscopists. This is a lower bound for cost because it includes a hybrid of individuals who have only radiography and pulmonary function testing as well as persons undergoing bronchoscopy. The cost does not include any cost for nonmonetary consequences of second-stage screening (eg, potential complications of testing and discomfort from procedures), nor does it include indirect costs such as lost work time.
RESULTS Repeated Testing Using LPT The potential effects of repeated testing using LPT are shown in Fig. 2. The figure shows the overall accuracy using the traditional criterion that beryllium sensitization is present if the individual has two or more positive tests. Panels A to E each represent calculations for one set of parameters for sensitivity, specificity, and prevalence, and panel F compares the overall accuracy for the five sets of parameters. The true prevalence rate is specified at 5% in parameter sets A to D and 20% for set E. Panels A to E also show results separately for those who are truly sensitized and those who are truly not sensitized. Sets of illustrative parameters were chosen to reflect reasonable ranges expected in practice:
FIGURE 2. Effect of repeated LPT testing. The vertical axis represents the proportion of correct classifications, and the x axis represents the number of times the test is repeated. Parameters (sensitivity, specificity, and prevalence) are described in the Methods section. In panels A to E, the solid black line represents the overall percentage, dotted line represents the percentage of correct classification among persons who are truly not sensitized, and the dashed line represents the percentage of correct classification among those who are truly sensitized. Panel F summarizes results for the correct classification from each of the panels. LPT, lymphocyte proliferation test. C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
863
JOEM r Volume 56, Number 8, August 2014
Harber and Su
Parameters
A
B
C
D
E
Sensitivity Specificity Prevalence
95% 90% 5%
75% 90% 5%
80% 80% 5%
70% 70% 5%
75% 90% 20%
markedly worse cost-effectiveness than the other strategies. Figure 3 graphically displays the prevented cases of Advanced-CBD (when compared with a fixed 6-year screening interval) and program cost. Several of the hybrid screening strategies-–in which the secondary screening frequency is adjusted for personal risk-–have the greatest yield of Advanced-CBD cases saved per dollar invested.
DISCUSSION Repeating the tests increases accuracy initially, but the overall accuracy later declines as more cycles of testing are conducted. As long as the prevalence is less than 50%, extending testing will eventually reduce accuracy. This adverse effect is increasingly important with more testing cycles, lower prevalence, and lower specificity. With an optimal situation of 95% specificity and 80% sensitivity for the individual LPT, total accuracy remains more than 95% for as many as seven cycles. For an optimal situation with 95% specificity for each individual test, accuracy remains more than 95% for the first seven cycles. Nevertheless, even minor reduction of specificity to 90% significantly reduces overall accuracy after the test is done four times (panels A, B, and E). With significantly lower sensitivity and specificity (eg, 70% for each), only two thirds are accurate with for test cycles and only one third are accurate when testing is repeated seven times. Panels C and D may reflect situations compatible with empirical data.13,14
Screening programs should optimize the net benefit to individuals and maximize cost-effectiveness. Existing screening program methods have been effective for classifying cases into two well-accepted categories and for detection of early CBD. Nevertheless, this article suggests the importance of evaluating two possible enhancements to commonly used practices. First, the number of positive LPT tests needed to consider an individual to be sensitized (BeS)
Testing Interval for Diagnosing CBD Among Persons Classed as BeS The potential effect of alternative screening intervals and adjusting screening intervals considering personal factors associated with an increased risk of progression is summarized in Table 1. The estimated number of detected early cases (Detected), advanced cases (Advanced-CBD), and program costs over a 40-year horizon is shown. In addition to the overall cost-effectiveness, the table shows incremental cost-effectiveness (number of AdvancedCBD cases prevented for each additional $1,000,000 allocated). The total cost ranges from $10 million to $68 million, depending on the frequency of second-stage screening and whether frequency is adjusted according to personal risk. Annual secondary screening of all BeS subjects has the greatest yield of Detected cases but has
FIGURE 3. CBD cases saved per $1 million. The figure illustrates A-CBD cases avoided per million dollar program cost (compared with uniform 6-year schedule) with alternate second-stage screening intervals. Open diamonds represent same schedule for all and filled circles represent adjusted by estimated progression risk (see Table 1). CBD, chronic beryllium disease.
TABLE 1. Effect of Alternative Screening Schedules* Screening Schedule (Years) Lower risk Higher risk Total detected Total advanced-CBD Total cost (million) Cost-effectiveness Sensitivity, % Incremental cost (million) Incremental cost/CBD avoided # Case saved/$1 million # Detected/$1 million CBD saved/$1 million
1 1
2 1
2 2
263 29 $68 $0.26 90 $26 $1.51 0.7 −0.3 1.76
255 46 $42 $0.16 85 $8 $0.50 2.0 −1.4 2.45
244 62 $34 $0.14 80 $1 −$1.36 −0.7 −3.6 2.56
3 1 Effects 239 61 $32 $0.14 80 $9 $0.26 3.9 −4.5 2.68
6 1
3 2
3 3
6 3
6 6
199 96 $23 $0.12 67 $1 $0.01 78.9 103.6 2.21
277 156 $23 0.08 64 $2 −$0.14 −7.2 −14.3 −0.37
249 142 $21 $0.08 64 $8 −$0.43 −2.3 −9.5 0.30
175 124 $13 $0.07 59 $3 $0.11 9.0 −9.4 1.87
150 148 $10 $0.07 50 — — — — —
*The table summarizes estimated effect of alternative screening approaches (for people in BeS category) on the basis of approximately 50% greater risk of progression in 25% of subjects. Results are based on 1000 initial program participants and average cost of $2000 per visit. Detected: detected by screening with preclinical CBD; cost-effectiveness: ratio of cost to number Detected; sensitivity: proportion of people with CBD who are Detected. Incremental costs are based on comparing a program to the next lower-cost design, expressed as total cost increment and cost per Advanced-CBD case avoided. CBD saved: number of advanced CBD cases compared with the number of cases using fixed 6-year second-stage screening intervals divided by cost in millions. See the text and supplement for details. CBD, chronic beryllium disease.
864
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
JOEM r Volume 56, Number 8, August 2014
may be adjusted by the number of tests performed. Second, occupational history and LPT results may be used to adjust the frequency with which persons with BeS undergo repeated extensive testing to determine whether they have CBD. These approaches may increase both the accuracy and the cost-effectiveness of programs.
Repeated Testing Using LPT for Identifying BeS The beryllium LPT is used to identify individuals with immunologic sensitization among workers who have had beryllium exposure. Commonly used interpretation strategies have the goal of limiting false positives to 5%.19,20 For an LPT, a test result is a continuous variable, called the stimulation index (SI). The SI for each test is considered positive if the SI is greater than a criterion value, which is chosen to limit false positives to 5%. Nevertheless, BeE workers in screening programs have the test performed many times, often on an annual basis. The test is imperfect, having both false positives and false negatives. Sensitization is operationally defined by having two or more positive LPTs. It is logical that the likelihood of having at least two positive tests increases as the number of tests increases. Unlike screening for some other disorders, there is no generally accepted clinical “criterion standard” to define sensitization or CBD independent of the test procedures per se. Therefore, simple measures such as sensitivity/specificity are less applicable when screening testing is applied multiple times. As shown in Fig. 2, the overall accuracy of a sequence of tests in a population including both persons with and without true sensitization depends on the number of times the test is repeated. Therefore, as applied to beryllium workers screening, the goal of limiting false positives to 5% is not achieved by the common algorithm. For declaring an individual to have BeS, it seems preferable to consider the number of positives and the number of tests performed rather than simply considering someone sensitized if they have two positive tests. (Good clinicians may consider this implicitly, particularly when the SIs of the positive tests are barely more than the criterion values.) These calculations have been intentionally simplified for illustrative purposes. They do not consider borderline or uninterpretable test results.21 If borderline tests were considered, the false-positive proportion would probably increase because the current clinical practice considers one positive and one borderline to be almost equivalent to two positives. If the prevalence of sensitization is less than 50%, this would, therefore, decrease the overall estimate of accuracy. The calculations assume that the overall test characteristics (specificity and sensitivity) remain constant over the testing cycles and that each test has the same outcome probabilities. This may also lead to overestimating accuracy if there is autocorrelation of results within individuals. Finally, this illustration assumes that the prevalence of sensitization in the tested population remains constant over time. The prevalence of previously undetected BeS is likely to actually decline after the first several years of a screening program because the incidence of new cases is likely to be lower than the number of individuals who are removed from the screenee group. Over time, persons with genetic susceptibility and/or particularly large exposures are likely to have been detected by screening or development of symptoms, so that the residual proportion of undetected true positive declines in the screen population. Analysis of data from the BBB showed that latencies of those who remained in BeE were significantly higher than those who had progressed, implying that the rate of progression declines.17 Furthermore, persons who progress to CBD may be identified outside of the screening program because of radiological or clinical findings, also decreasing the residual proportion of persons with true sensitization among screenees. In addition, there is limited basis for the common medical practice of choosing the cutpoint (criterion value) for a 5% falsepositive proportion on the basis of methods developed for agricul-
Beryllium Biobank 3
tural research.22 Rather, the optimal cutpoint may depend on the relative costs and benefits of false positives and false negatives. A related analysis of BBB data showed that a criterion value of 6.0 performed better than the usual criterion value of 3.0 for predicting progression to CBD.14 Additional empirical studies are needed to select the optimal approach. For example, classification in BeS might require at least three positive tests in a person with a large number of tests. Alternatively, a statistical calculation that includes the absolute value of each SI rather than a simple positive/negative classification may be preferable.
Testing Intervals for Extensive Evaluation of BeS Cases Pulmonary studies of data from the BBB do not create a comprehensive progression risk index for each individual, but they have demonstrated that personal characteristics such as peak exposure weighted hours and actual LPT values affect progression likelihood (ie, help determine who is likely to benefit from extensive testing).13,14 The model in this report illustrates the potential utility of combining empirically determined data and a model of the course of disease progression.17 The preliminary analyses considered only the following two items: exposure history and LPT results. Nevertheless, most participants in early detection programs have considerably more data available such as multiple points in time for LPT data, radiography, and lung function testing. Therefore, analyses considering additional factors would probably be even more effective than this study’s demonstrated ability to predict progression and identify who requires intensive second-stage screening. Although genetic factors significantly affect risk,18 an individual’s genetic characteristics are not known to clinicians for practical and ethical reasons. Previous beryllium disease–related analytic models have considered basing pre-placement screening on genetic factors23 and optimizing the primary screening (LPT) interval.24 The analysis presented in this brief article focuses on second-stage screening of BeE workers. For such workers, screening costs of the first-stage screening (detecting those who have transitioned from BeE to BeS) are primarily driven by the large number of exposed individuals who undergo screening5 with a relatively simple and inexpensive test (LPT). Conversely, for the second-stage screening (detecting those who have transitioned from BeS to CBD), costs per individual screening episode are considerably greater, but the number of potential persons requiring second-stage screening is lower. Similarly, the benefit of detecting early CBD and the harm of failure to detect early CBD in a timely fashion are potentially greater. Although long-term, longitudinal, large-scale, randomized controlled trials are optimal for establishing clinical and public health policy, program design and clinical decision-making must often use approaches such as quantitative modeling. Extremely large study populations are necessary for relatively uncommon outcomes, and very long empirical trials are needed for long latency diseases such as CBD. The many possible permutations of program design (eg, wide range of possible screening intervals and methods for partitioning risk among members) preclude trials of each possibility. Perhaps most importantly, public-health and clinical practice decisions must be made without delaying for such long-term trials, even if they were feasible. Therefore, simulation modeling may be used as an effective tool for guiding decision making in the absence of randomized controlled trials. More complex simulation models in the future may consider sensitivity analyses for second-stage screening sensitivity and specificity, temporal changes in screening characteristics, and sensitivity analyses for the transition probabilities and other parameters incorporated. The transition model and calculations were intentionally narrowly focused because the goal of this article was simply to
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
865
JOEM r Volume 56, Number 8, August 2014
Harber and Su
illustrate the potential long-term implications of adjusting screening intervals. Furthermore, a broader array of outcome utilities may be used in addition to the cost-effectiveness of screening per se (eg, consider other costs and benefits of detection including work loss, income loss, and compensation costs).
CONCLUSIONS Current algorithms underlying beryllium worker screening programs have been effective. Nevertheless, their accuracy and costeffectiveness may be incrementally improved by adjusting algorithms for each person’s available data. For example, the number of positive LPT tests to be classified as sensitized may increase in people who have had many tests. The interval between repeated extensive testing of persons who are sensitized may be adjusted on the basis of factors affecting the likelihood of progression to CBD.
10. 11.
12.
13.
14.
ACKNOWLEDGMENTS The authors thank Gabriela Alongi and the Beryllium BioBank Steering Committee for invaluable assistance. The authors also thank the US DOE Beryllium BioBank for the assistance in this project. The Beryllium BioBank was developed with funding from the US Department of Energy.
15.
16.
17.
REFERENCES 1. Newman LS, Mroz MM, Maier LA, Daniloff EM, Balkissoon R. Efficacy of serial medical surveillance for chronic beryllium disease in a beryllium machining plant. J Occup Environ Med. 2001;43:231–237. 2. Rossman MD. Justification for screening for chronic beryllium disease: closer to reality. Eur Respir J. 2008;32:543–544. 3. Sackett HM, Maier LA, Silveira LJ, et al. Beryllium medical surveillance at a former nuclear weapons facility during cleanup operations. J Occup Environ Med. 2004;46:953–961. 4. Takaro TK, Pepper L. Screening for chronic beryllium disease. Am J Respir Crit Care Med. 2005;172:1230. 5. Henneberger PK, Goe SK, Miller WE, Doney B, Groce DW. Industries in the United States with airborne beryllium exposure and estimates of the number of current workers potentially exposed. J Occup Environ Hyg. 2004;1:648–659. 6. Stange AW, Hilmas DE, Furman FJ, Gatliffe TR. Beryllium sensitization and chronic beryllium disease at a former nuclear weapons facility. Appl Occup Environ Hyg. 2001;16:405–417. 7. Henneberger PK, Cumro D, Deubner DD, Kent MS, McCawley M, Kreiss K. Beryllium sensitization and disease among long-term and short-term workers in a beryllium ceramics plant. Int Arch Occup Environ Health. 2001;74:167– 176. 8. Donovan EP, Kolanz ME, Galbraith DA, Chapman PS, Paustenbach DJ. Performance of the beryllium blood lymphocyte proliferation test based on a
866
9.
18.
19.
20.
21.
long-term occupational surveillance program. Int Arch Occup Environ Health. 2007;81:165–178. Kreiss K, Mroz MM, Newman LS, Martyny J, Zhen B. Machining risk of beryllium disease and sensitization with median exposures below 2 micrograms/m3. Am J Ind Med. 1996;30:16–25. Maier LA. Beryllium health effects in the era of the beryllium lymphocyte proliferation test. Appl Occup Environ Hyg. 2001;16:514–520. Colorado School of Public Health. Clinical data & questionnaire. Available at: http://www.ucdenver.edu/academics/colleges/PublicHealth/research/ ResearchProjects/bbb/Pages/Clinical-Data.aspx. Published 2012. Accessed May 29, 2012. Borak J, Woolf SH, Fields CA. Use of beryllium lymphocyte proliferation testing for screening of asymptomatic individuals: an evidence-based assessment. J Occup Environ Med. 2006;48:937–947. Harber P, Su J, Alongi G. Exposure factors associated with chronic beryllium disease development in Beryllium BioBank participants. J Occup Environ Med. 2014;56:852–856. Harber P, Su J, Alongi G. Beryllium BioBank: 2. Lymphocyte proliferation testing. J Occup Environ Med. 2014;56:857–860. National Lung Screening Trial Research Team Aberle DR, Adams AM, Berg CD, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365:395–409. Pralong JA, Moullec G, Suarthana E, et al. Screening for occupational asthma by using a self-administered questionnaire in a clinical setting. J Occup Environ Med. 2013;55:527–531. Harber P, Bansal S, Balmes J. Progression from beryllium exposure to chronic beryllium disease: an analytic model. Environ Health Perspect. 2009;117:970–974. Van Dyke MV, Martyny JW, Mroz MM, et al. Risk of chronic beryllium disease by HLA-DPB1 E69 genotype and beryllium exposure in nuclear workers. Am J Respir Crit Care Med. 2011;183:1680–1688. Frome EL, Newman LS, Cragle DL, Colyer SP, Wambach PF. Identification of an abnormal beryllium lymphocyte proliferation test. Toxicology. 2003;183:39–56. Frome EL, Smith MH, Littlefield LG, Neubert RL, Colyer SP. Statistical methods for the blood beryllium lymphocyte proliferation test. Environ Health Perspect. 1996;104(suppl 5):957–968. Middleton DC, Fink J, Kowalski PJ, Lewin MD, Sinks T. Optimizing BeLPT criteria for beryllium sensitization. Am J Ind Med. 2008;51:166–172.
22. Fisher R. The arrangement of field experiments. J Min Agric G Br. 1926; 33:503–513. 23. Bartell SM, Ponce RA, Takaro TK, Zerbe RO, Omenn GS, Faustman EM. Risk estimation and value-of-information analysis for three proposed genetic screening programs for chronic beryllium disease prevention. Risk Anal. 2000;20:87–99. 24. Judd NL, Griffith WC, Takaro T, Faustman EM. A model for optimization of biomarker testing frequency to minimize disease and cost: example of beryllium sensitization testing. Risk Anal. 2003;23:1211– 1220.
C 2014 American College of Occupational and Environmental Medicine
Copyright © 2014 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
