Sensitivity and specificity of repeated test results from a commercial milk enzyme-linked immunosorbent assay for detection of Mycobacterium avium subspecies paratuberculosis in dairy cattle Carrie J. Lavers, DVM, PhD; Ian R. Dohoo, DVM, PhD; Shawn L. B. McKenna, DVM, PhD; Greg P. Keefe, DVM, MSc, MBA

RUMINANTS

Objective—To investigate the sensitivity and specificity of results of initial and repeated milk ELISAs (at 6- or 12-month intervals) to detect cows that were shedding Mycobacterium avium subsp paratuberculosis (ie, were infectious) and to evaluate factors influencing the probability that the results of a repeated milk ELISA would be positive for an infectious cow if the results of the initial milk ELISA were negative. Design—Prospective cohort study. Animals—3,145 dairy cows from 32 herds. Procedures—Herds from the 3 Maritime provinces in Canada (Prince Edward Island, New Brunswick, and Nova Scotia), participating in a Dairy Herd Improvement program, and that had undergone a prior Mycobacterium avium subsp paratuberculosis awareness project were selected for the study. Sample collection occurred between April 2009 and March 2011 with milk and fecal samples collected from all lactating cows in study herds every 6 months. Herds completing < 3 herd visits with collection of individual cow fecal or milk samples, within this sampling timeframe, were excluded from analyses. Fecal samples were cultured in liquid medium and a cow was defined as infectious if ≥ 1 sample was culture positive (reference test). A milk ELISA (index test) was completed with a commercial kit, following manufacturer’s instructions. Results—For a 6-month test interval, sensitivities of the milk ELISA to detect infectious cows were 22.0% and 32.6% for initial and combined initial and repeated tests (parallel interpretation), respectively. Specificity of the initial ELISA was 99.6% and was 99.2% for combined tests. For a 12-month test interval, sensitivities of the milk ELISA to detect infectious cows were 25.6% and 45.3% for initial and combined initial and repeated tests (parallel interpretation), respectively. Specificity of the initial ELISA was 99.6% and was 98.9% for combined tests. In infectious cows, magnitude of the initial negative ELISA result was a positive predictor for a positive repeated ELISA result. Conclusions and Clinical Relevance—Results of a repeated milk ELISA improved detection of Mycobacterium avium subsp paratuberculosis infectious cows, with minimal loss of specificity. A 12-month test interval provided a greater increase in sensitivity, relative to an initial test, than did a 6-month interval. Infectious cows with an initial negative milk ELISA result close to the cutoff for a positive test were more likely to have positive results on a repeated ELISA. Repeated testing improved detection of infectious cows and reduced risk of misclassification compared with a single ELISA result. (J Am Vet Med Assoc 2015;246:236–244)

P

aratuberculosis (Johne’s disease) is a chronic, infectious enteritis in ruminants caused by MAP.1 There is no perfect antemortem test for paratuberculosis,2 creating a challenge for cow-level diagnosis. Diagnostic test accuracy varies with the 3 infection stages: infected (carrying MAP but not infectious), infectious (shedding MAP at the time of testing), and affected (clinical signs present).3 For infected cattle, sensitivity of the milk ELISA as high as 39%4 is reported, with specificity estimates ranging from 96%4 to 99.7%.5 For infectious cattle, estimates of the sensitivity of the milk ELISA range from 21%6 to 61%7 and specificity from 95%7,8 to 98%.6 Specificity of bacteriologic culture of From the Department of Health Management, Atlantic Veterinary College, University of Prince Edward Island, Charlottetown, PE C1A 4P3, Canada. This manuscript represents a portion of a thesis submitted by Dr. Lavers to the Atlantic Veterinary College Department of Health Management as partial fulfillment of the requirements for a Doctor of Philosophy degree. Supported in part by Maritime Quality Milk, a dairy research program of the Atlantic Veterinary College. The ELISA kits used in this study were donated by IDEXX Laboratories Inc. Address correspondence to Dr. Lavers ([email protected]). 236

Scientific Reports

ABBREVIATIONS CI MAP STARD STRADAS-paraTB

Confidence interval Mycobacterium avium subsp paratuberculosis Standards for Reporting of Diagnostic Accuracy Studies Standards for Reporting of Animal Diagnostic Accuracy Studies for Paratuberculosis)

feces approaches 100%,3 but sensitivity is limited, with estimates ranging from 23%9 to 29%10 for infected and 74% for infectious animals.11 Care should be taken when comparing diagnostic test characteristics because factors such as target condition, case definition, within-herd prevalence, and statistical methods will all influence estimates.3 Given the lower cost and faster processing time, relative to bacteriologic culture of feces,12,13 milk ELISA is the testing option of choice for many veterinarians and dairy producers. Considering the imperfect test characteristics JAVMA, Vol 246, No. 2, January 15, 2015

with detection of infectious cows on repeated testing, such as cow age and magnitude of initial ELISA results, may be useful for developing strategies to further improve the probability of detecting infectious cows with a repeated milk ELISA. The objectives of the study reported here were to investigate the sensitivity and specificity of results of initial and repeated milk ELISAs (at 6- or 12-month intervals) to detect cows that were shedding MAP (ie, were infectious) and to evaluate factors influencing the probability of results of repeated milk ELISA to be positive for an infectious cow if results of the initial milk ELISA were negative. Materials and Methods Study design and sample collection—This study was designed with adherence to the STARD statementa (Figure 1). The STARDa and STRADAS-paraTB2 checklists were completed (Online supplement available at avmajournals. avma.org/toc/javma/246/2). To be eligible to be selected to participate in this study, herds had to be from 1 of the 3 Maritime Canadian provinces and be participants in a Dairy Herd Improvement milk testing program. A herd’s involvement in this milk testing program ensured availability of herd and cow-level information required to investigate the study objectives, which were established a priori. From this source population, herds were selected on the basis of risk assessments completed as part of a previous MAP awareness project at the Atlantic Veterinary College, with the aim to attain a combination of high-prevalence (> 5% herd culture positive), low-prevalence, and MAP-negative herds. Thirty-four herds were selected from the dairy farms

Figure 1—STARD flowcharta illustrating the method of recruitment of study herds, the order of testing, and the number of study animals undergoing the test being evaluated (index test) and the reference test in a prospective cohort study evaluating the sensitivity and specificity of a commercial milk ELISA for detection of Mycobacterium avium subsp paratuberculosis in dairy cattle in Atlantic Canada. JAVMA, Vol 246, No. 2, January 15, 2015

Scientific Reports

237

RUMINANTS

of milk ELISA as well as the chronic progressive nature of paratuberculosis, a potential strategy to improve detection of infectious cows involves repeating a milk ELISA at a later date and evaluating initial and repeated test results together, rather than attempting to make a diagnosis on the basis of a single ELISA result. In cows > 3 years of age, repeated milk ELISA produced an area under the curve of 0.93, compared with an area under the curve of 0.86 for a single result (P = 0.003).14 However, it has also been reported that there is variability in results of repeated serum ELISAs15 and that comparing a current serum ELISA result with previous results provides minimal advantage over evaluating just the current ELISA result.16 Although repeated testing has the potential to increase milk ELISA sensitivity for detection of infectious cows, before recommending this practice to producers, it is necessary to quantify the information gained from a repeated ELISA performed with a commercially available test. It is also necessary to evaluate a testing interval that would potentially be both economically feasible and convenient for producers. A test interval that could be applied annually at a predetermined point in a cow’s lactation cycle, with a milk sample that is already being collected as part of routine herd milk testing, may be acceptable to many producers from both a convenience and cost perspective. Factors influencing the outcome of a single milk ELISA result have been investigated. Milk ELISA sensitivity for detection of MAP infection increases with cow age, and specificity decreases.17 Milk ELISA numeric values also increase with age.17 Magnitude of ELISA response also correlates with progression of the disease.18 Consideration of factors associated

RUMINANTS

of Prince Edward Island, New Brunswick, and Nova Scotia, Canada, to participate in this study, with sample collection occurring between April 2009 and March 2011. Herds completing < 3 herd visits with collection of individual cow fecal or milk samples, within this sampling frame, were excluded from the analysis. Further details on herd selection, herd MAP-status classification, herd demographics, and fecal sample collection have been described.19 Median herd size was 66 milking cows (mean, 82 cows; range, 28 to 220 cows). Median cow age at testing was 4.0 years (mean, 4.4 years; range, 1.8 to 17.3 years). Eleven of 32 (34%) facilities were tie-stall, and 21 of 32 (66%) were free-stall. The research project was reviewed and approved by the Atlantic Veterinary College Animal Care Committee, and all herd owners provided informed written consent to have their herds participate in the study. Sample collection involved 3 rounds of whole-herd fecal and milk sample collection, gathered at 6-month intervals (Table 1). Individual fecal samples were collected at 6-month intervals from all lactating cows by study personnel with a clean full-length plastic glove lubricated with sterile water. If samples could not be processed immediately, they were frozen at –20° or –80°C if processing would occur within either 2 weeks or between 2 and 6 weeks after collection, respectively. Individual milk sample collection has been described.20 Individual bronopol-preserved 45-mL milk samples were collected 3 times at 6-month intervals from all lactating cows as part of routine herd milk testing by the regional Dairy Herd Improvement organization (Valacta, Montreal, QC, Canada). Milk samples were transported and stored at 4°C and were processed within 3 days after collection.

Fecal samples were inoculated into liquid culture medium,b according to the manufacturer’s protocol, with the exception that samples were incubated for 49 days in the culture system,c rather than 42 days. Two grams of feces from each cow sample was put into a weigh boat, and the resulting 10 g of pooled feces was mixed with a wooden tongue depressor, then 2 g of the pooled sample was added to 35 mL of sterile water in a 50-mL centrifuge tube. For individual cow samples, 2 g of manure was placed into 35 mL of sterile water in a 50-mL centrifuge tube. Samples were agitated for 10 minutes and then placed upright for 30 minutes. A 5-mL sample was drawn from the feces and water mixture and placed into a centrifuge tube containing 25 mL of 0.9% 1-hexadecylpyridinium chloride in halfstrength brain-heart infusion broth. The tubes were incubated for 18 to 24 hours at 35° to 37°C, followed by centrifugation at 3,000 X g for 20 minutes. The supernatant was discarded, and the pellet was resuspended in 1 mL of brain-heart infusion broth and antimicrobial mixture (50 µg of amphotericin B, 100 µg of vancomycin, and 100 µg of nalidixic acid). Tubes were incubated for 24 hours at 35° to 37°C. On day 3, 2.5 mL of a premixed reagent (1 mL of egg yolk supplement, 1 mL of growth supplement, and 0.5 mL of antimicrobial supplement) and 1 mL of the decontaminated sample were added to a broth bottle. The broth bottle was agitated and then placed in the culture systemc for incubation up to a maximum of 49 days. The culture system’s computer output was examined daily for indications of positive curves. Presumptive positive samples (acidfast positive or positive culture system growth curves) were processed for confirmation with a PCR amplification kitd targeting the hspX gene. A culture sample was considered positive if the presumptive positive sample was confirmed by PCR assay.

Bacteriologic culture of feces—Methods used for bacteriologic culture of feces have been described previously.19,20 Individual cow fecal samples were pooled with 5 cows in each pool.21 Pooling was accomplished by mixing fecal samples from 5 cows, ordered by age. Individual cow samples from positive pools, which had been frozen at –80°C, were thawed and submitted for culture individually to determine cow MAP infection status. Bacteriologic culture of feces was performed by a laboratory technician in the Maritime Quality Milk laboratory, Charlottetown, PE, Canada, which was accredited by the USDA for this technique.

Milk ELISA—A commercial milk ELISAe was used, completed by 2 nonblinded laboratory technicians from the Maritime Quality Milk laboratory, which was accredited by the USDA for milk ELISA procedures. Following manufacturer’s instructions, all reagents and samples were brought to room temperature (18° to 25°C) for at least 1 hour before use. Then, 100 µL of each milk sample and 10 µL each of positive and negative controls were added to the wells of the dilution plate, 100 µL of diluent solution was then added to the

Table 1—Representation of the origin of data used in the calculation of sensitivity and specificity of initial and combined initial and repeated (6-month and 12-month interval) ELISAs on milk samples collected every 6 months, for 3 rounds of testing (at 0, 4 to 6, and 10 to 14 months), for the detection of MAP in 3,145 cows from 32 Atlantic Canadian dairy herds in the Maritime provinces. Round of testing Cow testing scenario 1 2 3 4 5 6 7

1 Tested Not tested Tested Tested Tested Not tested Not tested

2 Tested Tested Not tested Tested Not tested Tested Not tested

6-month interval 3 Tested Tested Tested Not tested Not tested Not tested Tested

12-month interval

Initial

Combined

Initial

Combined

1 2 1 1 1 2 NA

1 and 2 2 and 3 NA 1 and 2 NA NA NA

1 NA 1 1 1 NA NA

1 and 3 NA 1 and 3 NA NA NA NA

NA = Not applicable. If an initial test was not available, the initial and combined test characteristics were not calculated. If a repeated test was not available, the combined test characteristics were not calculated.

238

Scientific Reports

JAVMA, Vol 246, No. 2, January 15, 2015

A cow milk sample with a sample to positive percentage > 40% was considered ELISA positive on the basis of the criterion provided in the kit. Otherwise, results were considered negative. Data management—For a cow’s data to be included in this data set, the cow had to have ≥ 1 bacteriologic culture of feces result and ≥ 1 milk ELISA result. If a cow’s first ELISA result came from the final round of testing (round 3), it was excluded from the data set because it did not have the opportunity to have the ELISA repeated. For test characteristics of initial and repeated tests collected within a 6-month test interval, the chronologically first milk ELISA for a cow was considered the initial test, and the result from the next round of testing was the repeated test (Table 1). The 6-month test interval data came from a combination of rounds 1 (initial) and 2 (repeated) and rounds 2 (initial) and 3 (repeated) combinations. If a cow had data from both combinations (ie, 3 ELISA results available), the first chronological pair was included in the estimation of sensitivity and specificity. This was to allow for calculation of 95% CIs, which was not possible if all data were included owing to dependence of test results within a cow. For the 6-month test interval, the repeated ELISA had to be ≥ 110 days and ≤ 255 days following the initial ELISA. For analysis of a 12-month test interval, the only possible test combination was rounds 1 (initial) and 3 (repeated). The repeated ELISA had to be ≥ 290 days and ≤ 430 days following the initial ELISA. JAVMA, Vol 246, No. 2, January 15, 2015

For evaluation of predictors impacting an infectious cow, initially milk ELISA negative, to be milk ELISA positive on a repeated ELISA, all complete test combinations were eligible for inclusion in the model, provided the cow was infectious within the study period and the initial milk ELISA result was negative. In a subset of cows that had data for all 3 rounds, data from rounds 1 and 2 (6-month interval) and data from rounds 1 and 3 (12-month interval) were included. Consequently, a cow might have contributed 2 observations (a 6-month interval and a 12-month interval) to the analyses. Statistical analysis—Data used in this analysis represent a portion of the data collected in the total project. As described by Lavers et al,19 sample collection included 4 rounds of herd testing. The first round of testing, which was not performed on all herds, was not included as part of this analysis. Rounds 1, 2, and 3 in this analysis correspond to the final 3 rounds of testing described in the previous publication.19 A cow was considered infectious for MAP within a 1-year period if at least 1 fecal sample collected during the study period (minimum of 1 and maximum of 3 fecal samples collected/cow) was culture positive for MAP (case definition). There is no perfect gold standard test for paratuberculosis,2 and given the low sensitivity of bacteriologic culture of feces in subclinical animals,3 an infectious cow could be misclassified as noninfected. Repeated bacteriologic culture of feces was used to minimize this possibility. Cows not meeting the case definition (not having a positive culture result during the study period) were considered noninfected. Given the chronic, progressive nature of paratuberculosis, it is possible that a cow classified as noninfected was truly infected but did not shed the agent in its feces or did not shed at detectable levels during the study period. Conversely, although bacteriologic culture of feces is recognized as the current reference standard,3 the phenomenon of passive low shedding by uninfected animals has been documented in cows living in heavily contaminated environments.22 In noninfected cows, the specificity of an initial milk ELISA was defined as the probability of the initial ELISA being negative, given that the cow was noninfected (eg, 0.98). The specificity of combined initial and repeated milk ELISAs was calculated as follows: Specificity = P(TI–) X P(TR–|TI–)

where P(TI–) is the probability of an initial test result being negative, and P(TR–|TI–) is the conditional probability of a repeated test result being negative, given that the initial test result was negative (and the repeated test was not missing). Specificities were calculated separately for data collected from 6- and 12-month test intervals. For infectious cows, the sensitivity of an initial milk ELISA was defined as the probability of the initial ELISA being positive, given the cow was infectious. The combined initial and repeated milk ELISAs were evaluated in parallel, where the combined test result was considered positive if results of either the initial or repeated test or both tests were positive. The sensitivity of combined tests was calculated as 1 – specificity. Scientific Reports

239

RUMINANTS

wells containing milk samples to create a 1:1 dilution of the milk sample, and 190 µL of diluent solution was added to the control wells to create positive and negative control dilutions of 1:20. The plates were gently shaken by hand and left to incubate at room temperature for 15 minutes. Following incubation, 100 µL of each sample and control was transferred by pipette to the coated ELISA microplates and were then incubated for a further 45 minutes. At the end of this incubation period, the microplates were washed 3 times, with a 2-minute soak on the third wash. After washing, the microplates were tapped dry onto clean absorbent paper, and 100 µL of diluted conjugate (1:100 dilution) was added to each well. The microplate was incubated at room temperature for 30 minutes and then the wash was repeated. The washed microplate was tapped dry onto clean absorbent paper, and then 100 µL of substrate solution was added to each well. Microplates were incubated at room temperature in the dark for 10 minutes after addition of the substrate solution, and then 100 µL of stop solution was added to each well. The absorbance of each well was read following addition of the stop solutions to the wells. Positive and negative controls were completed in duplicate with each microplate of samples processed. If either of the positive or negative controls did not meet kit parameters, the entire plate was discarded and the milk samples were processed on a new plate. The ELISA output was reported as a sample to positive percentage:

RUMINANTS

Sensitivities were calculated separately for data collected from 6- and 12-month test intervals. To optimize utilization of data, if the first test (round 1) was missing, the subsequent sample (round 2) was used for initial sensitivity, creating a larger data set for 6-month than 12-month interval sensitivity and specificity data (testing scenarios 2 and 6; Table 1). Test characteristics were compared by evaluation of point estimates and respective 95% CIs. The 95% CIs for test characteristics of an initial test were calculated with the large-sample approximation in available statistical software.f The 95% CIs for 2 test combinations were computed with 2.5% and 97.5% percentile values from parametric bootstrapped estimates based on 1,000 resamples in a commercial software package.g Logistic regression analyses were used to determine which predictors impacted the probability that an infectious cow, initially milk ELISA negative, tested positive on the repeated milk ELISA. Predictors evaluated were interval (6 or 12 months), cow age in days at repeated testing and centered with 2 years as baseline, sample to positive percentage of the initial milk ELISA, and mean within-herd MAP prevalence based on bacteriologic culture of feces. Interval was forced into all models as the key predictor of interest, and age was included in all models as a potential confounder of the interval-outcome relationship (based on an a priori causal diagram; not shown). Univariable analyses were initially performed, and subsequently, all remaining factors were evaluated within a multivariable model with backward selection. Linear relationships between continuous predictors and outcome were determined on the basis of fractional polynomial models, to determine whether transformations (ie, quadratic) of the predictor provided a better fit for the model. Clustering of observations within a cow and within a herd was initially investigated with a mixed effects model with random effects for herd and cow. Subsequently, within-herd clustering (which was very small) was ignored and generalized estimating equations models were used to account for clustering within cows, yet still providing population average estimates of sensitivity and specificity. An exchangeable correlation structure was used to account for correlation between multiple observations within a cow, and the correlation between these observations was determined. The logistic analyses were conducted with the aid of a statistical software package,h and all model assumptions were met. Values of P ≤ 0.05 were considered significant for inclusion of predictors in the final model, with the exception of variables that were forced into the model. Results Descriptive statistics—Two of the 34 herds were excluded from the analyses because they did not have 3 herd visits with collection of individual cow fecal or milk samples. For the 32 herds with 3 rounds of testing, there were 4,145 cows that had ≥ 1 bacteriologic culture of feces or ELISA result throughout the 3 rounds of testing (Figure 1). There were 288 cows with no bacteriologic culture of feces data and 269 cows with no ELISA data that were not included in the data set. There were 443 cows with a single ELISA in round 3 (5 infec240

Scientific Reports

tious and 438 noninfected cows). Applying these criteria, the final data set for analysis included 3,145 cows, of which 164 (5.2%) cows were infectious. Within the 164 infectious cows, 138 (84%) had ≥ 2 fecal samples collected, and 120 (73%) had ≥ 2 milk samples collected. Within the 2,981 noninfected cows, 2,384 (80%) cows had ≥ 2 fecal samples collected, and 2,444 (82%) had ≥ 2 milk samples collected (Table 2). There were 26 of 164 (16%) infectious cows with 1 bacteriologic culture of feces result. There were 76 of 164 (46%) infectious cows with 2 bacteriologic culture of feces results; 51 were culture positive on 1 of 2 fecal samples, and 25 were culture positive on both fecal samples. There were 62 of 164 (38%) infectious cows with 3 bacteriologic culture of feces results; 35 were culture positive on 1 of 3 fecal samples, 16 were culture positive on 2 of 3 fecal samples, and 11 were culture positive on all 3 fecal samples. In the complete data set considering both 6- and 12-month test intervals and all possible test pairs from every cow, there were 21 test pairs in which both initial and repeated milk ELISAs were positive. These 21 positive-positive test pairs came from 16 cows, owing to the fact that some cows had 3 tests and were positive on every test. Fifteen of these 16 cows were infectious. The mean sample to positive percentage of the initial ELISA in these 15 cows was 136% (range, 44% to 252%). Eleven of these 15 infectious cows had sample to positive percentage ≥ 70% on the initial ELISA. Test characteristics of initial and repeated milk ELISAs—All ELISA microplates met control specifications. Initial and repeated milk ELISA results from a 6-month test interval were summarized (Table 3). Mean ± SD interval between tests was 174 ± 29 days. Observed sensitivity of initial ELISAs was 22.0% (95% CI, 15.6% to 28.3%), and that of combined initial and repeated ELISAs was 32.6% (95% CI, 26.3% to 41.2%). Observed specificity of initial ELISAs was 99.6% (95% CI, 99.4% to 99.8%), and that of combined initial and repeated ELISAs was 99.2% (95% CI, 98.8% to 99.5%). Initial and repeated milk ELISA results from a 12-month test interval were summarized (Table 4).

Table 2—Frequency of bacteriologic culture of feces and milk ELISAs for 3,145 cows from 32 Atlantic Canadian dairy herds. Fecal culture test frequency Milk ELISA frequency MAP-infectious* (n = 164 cows) 1 2 3 Total Noninfected (n = 2,981 cows) 1 2 3 Total

1

2

3

16 9 1 26

24 32 20 76

3 30 29 62

304 258 25 587

164 622 415 1,201

71 501 621 1,193

*A cow was considered infectious if it had at least 1 positive result of bacteriologic culture of feces during the study period (1 year), and was otherwise classified noninfected.

JAVMA, Vol 246, No. 2, January 15, 2015

Mean ± SD interval between tests was 364 days ± 36 days. Observed sensitivity of initial ELISAs was 25.6% (95% CI, 18.2% to 33.0%), and that of combined initial and repeated ELISAs was 45.3% (95% CI, 35.6% to 54.3%). Observed specificity of initial ELISAs was 99.6% (95% CI, 99.4% to 99.9%), and that of combined ELISAs was 98.9% (95% CI, 98.4% to 99.4%). Predictors associated with repeated positive milk ELISA results in infectious cows—There were 108 infectious cows with an initial negative milk ELISA result and at least 1 repeated ELISA. Sixty-three cows contributed 1 pair of ELISAs (initial and repeated test), and 45 Table 3—Initial and repeated MAP milk ELISA results collected at a 6-month test interval for 164 infectious and 2,981 noninfected cows, with a minimum of 140 days and maximum of 253 days between initial and repeated tests. Milk ELISA result

Bacteriologic culture of feces classification

Repeated

Infectious (n = 164)

Noninfected (n = 2,981)

Negative Negative Negative Positive Positive Positive

Negative Positive Not tested Positive Negative Not tested

74 14 40 11 1 24

2,068 9 892 0 9 3

A cow milk sample with a sample to positive percentage > 40% was considered ELISA positive on the basis of the criterion provided in the kit. Otherwise, results were considered negative.

Discussion

Table 4—Initial and repeated MAP milk ELISA results collected at a 12-month test interval for 133 infectious and 2,183 noninfected cows, with a minimum of 293 days and maximum of 426 days between initial and repeated tests. Milk ELISA result

Bacteriologic culture of feces classification

Initial

Repeated

Infectious (n = 133)

Noninfected (n = 2,185)

Negative Negative Negative Positive Positive Positive

Negative Positive Not tested Positive Negative Not tested

47 17 35 6 1 27

1,321 10 846 0 5 3

See Table 3 for key.

In the present study of 3,145 dairy cows from 32 herds in 3 Maritime provinces in Canada, combining information from initial and repeated milk ELISAs increased the probability of detecting an infectious cow, compared with an initial ELISA only. For 6-month and in particular 12-month test intervals, a repeated ELISA provided considerable increase in sensitivity with a relatively small decrease in specificity. When a 6-month test interval was used, the sensitivity of combined initial and repeated tests was 32.6%, a 48% increase versus sensitivity of an initial test. With a 12-month interval, the probability to detect an infectious cow was 45.3%, a 77% increase over initial test sensitivity. Specificity of combined tests for a 6-month interval was only half a percentage point lower than initial test specificity, and 0.7 percentage points lower given a 12-month interval. Huda et al14 also reported that repeated milk ELISA increased the probability of detecting an infectious cow.

Table 5—Odds ratios for a logistic regression model (fit with generalized estimating equations) for predictors associated with the probability for an MAP culture-positive cow with a negative initial milk ELISA result to have a positive repeated milk ELISA result. Predictor Intercept* Test interval (6 or 12 months) Cow age (in days)† at repeated ELISA Initial ELISA sample to positive percentage

Coefficient

OR (95% CI)

P value

–2.72 1.00 0.0004 0.16

— (–3.74 to –1.69) 2.72 (0.80 to 9.20) 1.0004 (0.9996 to 1.001) 1.18 (1.08 to 1.28)

< 0.001 0.11 0.35 < 0.001

Interval was analyzed as a dichotomous variable with a 6-month interval assigned a value of 0 and a 12-month interval assigned a value of 1. *Correlation between multiple observations within a cow was –0.03. †Cow age centered with 2 years as baseline. — = Not applicable. JAVMA, Vol 246, No. 2, January 15, 2015

Scientific Reports

241

RUMINANTS

Initial

cows contributed 2 pairs (6- and 12-month intervals). Initially, predictors were tested within a logistic mixed model with random effects at herd and cow levels, accounting for the 3-level hierarchical structure of data (herd, cow, and repeated test). However, clustering at the herd level (variance, 2.84 X 10–8) and cow level (variance, 9.89 X 10–6) was minimal, and only cowlevel factors were significant (P ≤ 0.05), so ignoring herd-level clustering would have minimal effect on the results. Given that we wanted to obtain population average estimates of sensitivity and specificity, we used a generalized estimating equations model to account for clustering at the cow level. This model accounted for multiple test results at the cow level, which allowed for accurate estimation of 95% CIs around estimates and produced population average estimates. Results from the generalized estimating equations model indicated that the sample to positive percentage of an initial negative ELISA result was a significant predictor for a repeated positive ELISA result in infectious cows (Table 5). For every 5% increase in initial sample to positive percentage of a negative test, an infectious cow was 2.3 times as likely to test positive on a repeated ELISA. Infectious cows were to be more likely to test positive on a repeated ELISA with a 12-month test interval, compared with a 6-month interval (OR, 2.7; 95% CI, 0.80 to 9.20), although this was not significant (P = 0.11).

RUMINANTS

By use of an in-house ELISA with various cutoffs dependent on cow age, Huda et al14 found that area under the curve was significantly greater for receiver operating characteristic curves representing 3 repeated milk ELISA samples, compared with 1 sample, for cows ≥ 3 years of age. However, the use of an in-house ELISA precluded extrapolation of results to commercial milk ELISAs. Further investigation of the test interval found that whereas probability to detect an infectious cow with a 12-month test interval, compared with a 6-month interval, was not significantly different (OR, 2.7; P = 0.11), we suggest that small sample size likely contributed to lack of significance for the interval variable in the present study. Increased test interval has been suggested as important in a prior study15 with a serum ELISA, in which the authors noted that a longer test interval could result in more cows seroconverting from negative to positive ELISA status.15 Paratuberculosis is a chronic, progressive disease, and it is expected the probability for a test to detect disease will increase as an animal ages.23 Milk ELISA detection of MAP infection increases approximately linearly from 2 to 5 years of age.17 In our data set, age was not a significant predictor for an infectious cow, initially ELISA negative, to be positive on a repeated ELISA. Lack of significance may have been related to the fact that the model also accounted for a cow’s initial ELISA result, which was a significant predictor. The impact of a repeated test on specificity in the present study was minimal; however, the number of repeated tests on the same animals was minimal. Nielsen and Ersboll23 used data collected from monthly milk ELISAs performed over a 3-year period. With monthly testing, by 4 years of age, a cow that was never culture positive had an 18% probability of having a positive milk ELISA (approximated from figure). A lower specificity due to high test frequency may be acceptable when producers and veterinarians anticipate and understand this likelihood. For example, in the Danish Control Program for Bovine Paratuberculosis,18 where cattle are subjected to milk ELISA 4 times/y, communication and education is a key part of program success. However, if a high test frequency is applied without the understanding that overall specificity will be lowered, test results will be frustrating for the producer to interpret. In addition, a 12-month test interval might be more financially acceptable and convenient to herds undertaking an MAP control program. Nielsen and Ersboll23 concluded that although monthly testing in cows < 4 years of age would increase the sensitivity of the ELISA, a cost-benefit analysis to determine optimal test frequency is required.19 An annual test would fit conveniently within current production systems because many preventative health measures are based on the lactation cycle of the cow. Quantitative value (sample to positive percentage) of an initial negative milk ELISA was predictive of an infectious cow being detected on a repeated milk ELISA. This is in agreement with an earlier study24 evaluating single serum ELISA results on repository samples, which found that cows with an optical density just below the cutoff for a positive test were 15 times as likely 242

Scientific Reports

to be infected than noninfected. Regarding ELISA values above the cutoff, the magnitude of the quantitative value may correlate with the infectious status,18 and it has been reported that moderate and heavy MAP shedders have significantly higher quantitative ELISA values than culture-negative cows.25 The relationship between quantitative values and milk production has also been studied, and a negative relationship between milk production and optical density of the ELISA was reported.26 Together, these results suggest it is beneficial to consider the magnitude of the ELISA value, in addition to the dichotomous result. If, for example, a producer is unable to repeat testing of all ELISA-negative cows, the quantitative value may be useful to target repeated testing to cows with a value close to the cutoff. Collins and Sockett24 also suggested that quantitative values may be useful when prioritizing culling decisions. In this study, 16 cows had a positive milk ELISA result on both initial and repeated tests. Small sample size precluded application of statistical models to this subgroup, as did the fact that 15 of the 16 cows with a positive-positive pattern were infectious. The noninfected cow with a positive-positive pattern originated from an infected herd that had a mean within-herd prevalence of almost 10% as determined on the basis of bacteriologic culture of feces. It is possible that this cow was truly infected but bacteriologic culture of feces produced false-negative results. These results suggest that repeated positive ELISAs represent a high likelihood for a cow to be infectious within a 1-year period. Other publications are also supportive of this likelihood, and it has been noted that cows with repeated positive ELISA results are more likely to be shedding MAP in the near future than cows with fluctuating antibody response profiles.27 One of the recommendations in Denmark’s MAP control program is to cull cows with 2 positive ELISA results when testing 4 times/y.28 It has also been reported that culling repeatedly ELISA-positive cows was 1 of 3 management strategies found to significantly decrease within-herd prevalence as determined on the basis of ELISA.29 In addition to repeated positive results representing a high likelihood of MAP infection, it has been noted that cows with repeated positive results as well as cows with their last test result positive and previous test results negative for MAP produced significantly less milk than cows that were repeatedly milk ELISA negative.30 Cows with repeated milk ELISA-positive results are more likely to be infected with MAP, and therefore, when interpreting results of repeated milk ELISAs, such cows should be managed as infectious within a herd risk management program. Fluctuation of serum ELISA results from positive to negative has been described15,16 and is a potential source of confusion for producers and veterinarians. Among 21 cows with an initial positive milk ELISA result, 10 (48%) reverted to a negative ELISA result 6 months later. All 9 noninfected cows reverted to a negative ELISA status, but only 1 of 12 of infectious cows reverted. Reversion proportions for the 12-month interval were almost identical. These results are similar to serum ELISA studies. Hirst et al15 reported that 62 of 157 (40%) seropositive cows were negative on a repeated ELISA. However, the culture status of these animals JAVMA, Vol 246, No. 2, January 15, 2015

3.

a. b.

19.

c. d. e. f. g. h.

STARD statement (www.stard-statement.org). Accessed Feb 1, 2009. ESP para-JEM broth, TREK Diagnostic Systems Inc, Cleveland, Ohio. ESP Culture System II, TREK Diagnostic Systems Inc, Cleveland, Ohio. VetAlert Johne’s Real-Time PCR kit, Tetracore Inc, Rockville, Md. Mycobacterium paratuberculosis Antibody Test Kit (Institut Pourquier technology), IDEXX, Westbrook, Me. Minitab, version 16, Minitab Inc, State College, Pa. Poptools, Excel, Microsoft Corp, Redmond, Wash. STATA/IC, version 11.2, StataCorp LP, College Station, Tex.

References 1. 2.

Chiodini RJ, Van Kruiningen HJ, Merkal RS. Ruminant paratuberculosis (Johne’s disease): the current status and future prospects. Cornell Vet 1984;74:218–262. Gardner IA, Nielsen SS, Whittington RJ, et al. Consensus-based reporting standards for diagnostic test accuracy studies for paratuberculosis in ruminants. Prev Vet Med 2011;101:18–34.

JAVMA, Vol 246, No. 2, January 15, 2015

4.

5. 6.

7.

8. 9.

10. 11.

12. 13. 14.

15.

16.

17. 18.

20.

21. 22. 23. 24.

Nielsen SS, Toft N. Ante mortem diagnosis of paratuberculosis: a review of accuracies of ELISA, interferon-gamma assay and faecal culture techniques. Vet Microbiol 2008;129:217–235. Nielsen SS, Gronbaek C, Agger JF, et al. Maximum-likelihood estimation of sensitivity and specificity of ELISAs and faecal culture for diagnosis of paratuberculosis. Prev Vet Med 2002;53:191–204. Collins MT, Wells SJ, Petrini KR, et al. Evaluation of five antibody detection tests for diagnosis of bovine paratuberculosis. Clin Diagn Lab Immunol 2005;12:685–692. Lombard JE, Byrem TM, Wagner BA, et al. Comparison of milk and serum enzyme-linked immunosorbent assays for diagnosis of Mycobacterium avium subspecies paratuberculosis infection in dairy cattle. J Vet Diagn Invest 2006;18:448–458. Hendrick SH, Duffield TE, Kelton DE, et al. Evaluation of enzyme-linked immunosorbent assays performed on milk and serum samples for detection of paratuberculosis in lactating dairy cows. J Am Vet Med Assoc 2005;226:424–428. Klausen J, Huda A, Ekeroth L, et al. Evaluation of serum and milk ELISAs for paratuberculosis in Danish dairy cattle. Prev Vet Med 2003;58:171–178. McKenna SL, Keefe GP, Barkema HW, et al. Evaluation of three ELISAs for Mycobacterium avium subsp. paratuberculosis using tissue and fecal culture as comparison standards. Vet Microbiol 2005;110:105–111. Whitlock RH, Wells SJ, Sweeney RW, et al. ELISA and fecal culture for paratuberculosis (Johne’s disease): sensitivity and specificity of each method. Vet Microbiol 2000;77:387–398. Sockett DC, Carr DJ, Collins MT. Evaluation of conventional and radiometric fecal culture and a commercial DNA probe for diagnosis of Mycobacterium paratuberculosis infections in cattle. Can J Vet Res 1992;56:148–153. Tiwari A, VanLeeuwen JA, McKenna SL, et al. Johne’s disease in Canada part I: clinical symptoms, pathophysiology, diagnosis, and prevalence in dairy herds. Can Vet J 2006;47:874–882. Collins MT. Diagnosis of paratuberculosis. Vet Clin North Am Food Anim Pract 2011;27:581–591. Huda A, Jungersen G, Lind P. Longitudinal study of interferongamma, serum antibody and milk antibody responses in cattle infected with Mycobacterium avium subsp. paratuberculosis. Vet Microbiol 2004;104:43–53. Hirst HL, Garry FB, Salman MD. Assessment of test results when using a commercial enzyme-linked immunosorbent assay for diagnosis of paratuberculosis in repeated samples collected from adult dairy cattle. J Am Vet Med Assoc 2002;220:1685–1689. Sweeney RW, Whitlock RH, McAdams S, et al. Longitudinal study of ELISA seroreactivity to Mycobacterium avium subsp. paratuberculosis in infected cattle and culture-negative herd mates. J Vet Diagn Invest 2006;18:2–6. Nielsen SS, Toft N. Age-specific characteristics of ELISA and fecal culture for purpose-specific testing for paratuberculosis. J Dairy Sci 2006;89:569–579. Nielsen SS. Danish control programme for bovine paratuberculosis. Cattle Pract 2007;15:161–168. Lavers CJ, McKenna SL, Dohoo IR, et al. Evaluation of environmental fecal culture for Mycobacterium avium subspecies paratuberculosis detection in dairy herds and association with apparent within-herd prevalence. Can Vet J 2013;54:1053–1060. Lavers CJ, Barkema HW, Dohoo IR, et al. Evaluation of milk ELISAs for detection of Mycobacterium avium subspecies paratuberculosis in dairy herds and association with within-herd prevalence. J Dairy Sci 2014;97:299–309. Kalis CH, Hesselink JW, Barkema HW, et al. Culture of strategically pooled bovine fecal samples as a method to screen herds for paratuberculosis. J Vet Diagn Invest 2000;12:547–551. Pradhan AK, Schukken YH. Molecular epidemiology of Mycobacterium avium subsp. paratuberculosis in a longitudinal study of three dairy herds. J Clin Microbiol 2011;49:893–901. Nielsen SS, Ersboll AK. Age at occurrence of Mycobacterium avium subspecies paratuberculosis in naturally infected dairy cows. J Dairy Sci 2006;89:4557–4566. Collins MT, Sockett DC. Accuracy and economics of the USDAlicensed enzyme-linked immunosorbent assay for bovine paratuberculosis. J Am Vet Med Assoc 1993;203:1456–1463. Scientific Reports

243

RUMINANTS

was not known. In another longitudinal serum ELISA study, where results of bacteriologic culture of feces were available, Sweeney et al16 found that although 17 of 18 culture-negative cows with an initial positive serum result had a repeated negative serum result, only 14 of 58 (24%) culture-positive cows reverted from seropositive to seronegative. Reversion from positive to negative on a repeated ELISA appears to occur more commonly in noninfected cows. However, as recommended in the Danish paratuberculosis control program, these cows should be regarded as low risk, and further repeated testing is recommended to better establish their MAP infection status.18 The observational structure of this project lends both strengths and weaknesses. A strength was that cows studied were naturally infected within a selection of MAP-negative, low-prevalence, and high-prevalence herds. The cow population would be representative of many herds in the North American industry, where most MAP-positive herds are expected to have low to moderate prevalence.31 Therefore, selection bias at the cow-level in this population should have had little impact on our results. The primary weaknesses of the observational study were that cows provided variable numbers of samples and the large sample size precluded tissue culture at postmortem, which would have allowed for further determination of true MAP status. As well, despite the large sample size, because MAP is a disease of low prevalence, the subset of infectious cows was small. Repeated milk ELISA improved the probability to detect a cow infectious within a 1-year period, with minimal loss of specificity. A 12-month test interval provided a greater increase in sensitivity, relative to a single initial test, than did a 6-month interval. Infectious cows with a negative ELISA result close to the cutoff for a positive test were more likely to be ELISA positive on a repeated milk ELISA. Cows with repeated positive ELISA results were likely to be infectious and should be a priority for risk management strategies. Given the potential for fluctuation between positive and negative ELISA results, repeated testing within an MAP-positive herd provides an improved understanding of a cow’s MAP status, and reduces risk of misclassification based on a single test result.

25. van Schaik G, Rossiter CR, Stehman SM, et al. Longitudinal study to investigate variation in results of repeated ELISA and culture of fecal samples for Mycobacterium avium subsp paratuberculosis in commercial dairy herds. Am J Vet Res 2003;64:479–484. 26. Kudahl A, Nielsen SS, Sorensen JT. Relationship between antibodies against Mycobacterium avium subsp. paratuberculosis in milk and shape of lactation curves. Prev Vet Med 2004;62:119–134. 27. Nielsen SS. Transitions in diagnostic tests used for detection of Mycobacterium avium subsp. paratuberculosis infections in cattle. Vet Microbiol 2008;132:274–282.

28. Nielsen SS. Use of diagnostics for risk-based control of paratuberculosis in dairy herds. In Pract 2009;31:150–154. 29. Nielsen SS, Toft N. Effect of management practices on paratuberculosis prevalence in Danish dairy herds. J Dairy Sci 2011;94:1849–1857. 30. Nielsen S, Enevoldsen C, Toft N. Milk production losses associated with bovine paratuberculosis diagnosed from repeated testing, in Proceedings. 11th Int Symp Vet Epidemiol Econ 2006;6–11. 31. Lombard JE. Epidemiology and economics of paratuberculosis. Vet Clin North Am Food Anim Pract 2011;27:525–535.

From this month’s AJVR

Temporal and spatial dynamics of porcine reproductive and respiratory syndrome virus infection in the United States RUMINANTS

Steven J. P. Tousignant et al Objective—To measure incidence and estimate temporal and spatial dynamics of porcine reproductive and respiratory syndrome virus (PRRSV) infection in US sow herds. Animals—371 sow herds in the United States from 14 production companies. Procedures—The exponentially weighted moving average was used to monitor incident PRRSV infections for onset of an epidemic. The spatial scan statistic was used to identify areas at significantly high risk of PRRS epidemics. A χ2 test was used to estimate whether there were significant differences in the quarterly and annual PRRS incidence among time periods, and a bivariable logistic regression model was used to estimate whether PRRSV infection during a given year increased the odds of that herd being infected in the following year. Results—During the 4-year period of this study, 29% (91/319; 2009 to 2010), 33% (106/325; 2010 to 2011), 38% (135/355; 2011 to 2012), and 32% (117/371; 2012 to 2013) of the herds reported new infections. Weekly incidence was low during spring and summer and high during fall and winter. The exponentially weighted moving average signaled the onset of a PRRSV epidemic during the middle 2 weeks of October each year. Disease incidence was spatially clustered. Infection in the previous year increased the odds of infection in 2010 to 2011 and 2011 to 2012. Conclusions and Clinical Relevance—Results indicated a striking repeatability in annual PRRSV temporal and spatial patterns across 4 years of data among herds from 14 production companies, which suggested that efforts to control PRRSV at a regional level should continue to be supported. (Am J Vet Res 2015;76:70–76)

244

Scientific Reports

January 2015

See the midmonth issues of JAVMA for the expanded table of contents for the AJVR or log on to avmajournals.avma.org for access to all the abstracts.

JAVMA, Vol 246, No. 2, January 15, 2015