Commentary Commentaire Application of confidence intervals to data interpretation Catherine R. Wagg, Grace P.S. Kwong, Daniel S.J. Pang
W
e wish to comment on the conclusions reached by Woodman et al (1) in their study evaluating the effect of dexamethasone [0.05 mg/kg body weight (BW) q24h for 7 d] in adult horses. The authors did not find statistically significant differences in thromboelastography (TEG) parameters between dexamethasone- and saline-treated animals. Based on these observations they reached 2 conclusions. Firstly, that the dose of dexamethasone administered did not affect TEG parameters and secondly, that a larger sample size was required to identify a treatment effect. We suggest an approach that aids interpretation of the data: including the 95% confidence interval (CI) in the presentation of results. When describing the results of hypothesis testing, the common reliance on P-values encourages authors and readers to interpret results as a dichotomy, “positive”/“significant” or “negative”/“non-significant,” based on a predetermined threshold of significance (typically P , 0.05). In contrast, a CI shows the lack of precision in the estimated difference between treatments, with a 95% range conventionally reported. The value of reporting a 95% CI is in assessing the range of possible true differences between treatment groups, giving an indication of whether a scientifically or clinically important difference has been overlooked due to insufficient sample size or unexpected data variability, or both (2,3). We have taken the maximum amplitude (MA) data of Woodman et al (1) to illustrate this approach (data from CW). The results of the reported Bonferroni multiple comparison test are summarized in Table 1 alongside the 95% CI. It is clear that there was no significant treatment effect at each time point (P . 0.99). However, the presentation of 95% CI allows further interpretation. The range of upper limits of 95% CI values (3.37 to 9.82) indicates that dexamethasone treatment could increase MA by up to 9.82 mm. Although this is unlikely, as the true treatment effect will be closer to the mean differences, presentation of the 95% CI allows the authors and readers to decide if these potential increases in MA
Table 1. Maximum amplitude (mm) data following a Bonferroni multiple comparison test between treatment groups (n = 8 horses per treatment group). Data are mean 6 standard deviation Day Dexamethasone Saline 0 61.86 6 4.3 1 61.21 6 3.4 2 62.12 6 3.4 4 63.28 6 4.9 5 62.99 6 3.7 6 61.53 6 2.2
Mean 95% CI of difference difference
62.63 6 5.4 20.77 63.03 6 6.6 21.83 61.76 6 7.8 0.36 61.44 6 7.1 1.84 62.61 6 5.9 0.38 61.08 6 4.4 0.44
26.22 to 4.68 28.11 to 4.46 27.44 to 8.15 26.13 to 9.82 24.71 to 5.48 22.48 to 3.37
P-value . 0.99 . 0.99 . 0.99 . 0.99 . 0.99 . 0.99
could be clinically important. If this is the case, further studies with a larger sample size, a meta-analysis, or techniques to reduce data variability would be necessary to determine a more precise (narrower 95% CI) estimate of the difference between treatment groups. In contrast, if only larger differences in MA than those observed with the 95% CI are relevant it can be concluded that dexamethasone (in the protocol studied) does not lead to a statistically significant or clinically relevant increase in MA. Though we have used the upper limits as an example, a similar logic applies to interpreting the relevance of the lower limit values. While the current level of evidence for interpretation of TEG in horses is limited, precluding firm conclusions to be drawn from these data (4), the presentation of 95% CI maximizes the contribution of these data to the literature.
References 1. Woodman J, Wagg CR, Boysen SR, Leguillette R, Mizen K, Roy MF. Evaluation of coagulation via thromboelastography in healthy horses administered dexamethasone. Can Vet J 2015;56:1271–1274. 2. Gardner MJ, Altman DG. Confidence intervals rather than P-values: Estimation rather than hypothesis testing. Br Med J 1986;292:746–750. 3. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Br Med J 2010;340:c869. 4. Hanel RM, Chan DL, Conner B, et al. Systematic evaluation of evidence on veterinary viscoelastic testing Part 4: Definitions and data reporting. J Vet Emerg Crit Care 2014;24:47–56.
Veterinary Clinical and Diagnostic Sciences (Wagg, Pang), Faculty of Veterinary Medicine (UCVM) (Kwong), University of Calgary, Calgary, Alberta. Use of this article is limited to a single copy for personal study. Anyone interested in obtaining reprints should contact the CVMA office ([email protected]) for additional copies or permission to use this material elsewhere. CVJ / VOL 57 / MAY 2016
547
W
e wish to comment on the conclusions reached by Woodman et al (1) in their study evaluating the effect of dexamethasone [0.05 mg/kg body weight (BW) q24h for 7 d] in adult horses. The authors did not find statistically significant differences in thromboelastography (TEG) parameters between dexamethasone- and saline-treated animals. Based on these observations they reached 2 conclusions. Firstly, that the dose of dexamethasone administered did not affect TEG parameters and secondly, that a larger sample size was required to identify a treatment effect. We suggest an approach that aids interpretation of the data: including the 95% confidence interval (CI) in the presentation of results. When describing the results of hypothesis testing, the common reliance on P-values encourages authors and readers to interpret results as a dichotomy, “positive”/“significant” or “negative”/“non-significant,” based on a predetermined threshold of significance (typically P , 0.05). In contrast, a CI shows the lack of precision in the estimated difference between treatments, with a 95% range conventionally reported. The value of reporting a 95% CI is in assessing the range of possible true differences between treatment groups, giving an indication of whether a scientifically or clinically important difference has been overlooked due to insufficient sample size or unexpected data variability, or both (2,3). We have taken the maximum amplitude (MA) data of Woodman et al (1) to illustrate this approach (data from CW). The results of the reported Bonferroni multiple comparison test are summarized in Table 1 alongside the 95% CI. It is clear that there was no significant treatment effect at each time point (P . 0.99). However, the presentation of 95% CI allows further interpretation. The range of upper limits of 95% CI values (3.37 to 9.82) indicates that dexamethasone treatment could increase MA by up to 9.82 mm. Although this is unlikely, as the true treatment effect will be closer to the mean differences, presentation of the 95% CI allows the authors and readers to decide if these potential increases in MA
Table 1. Maximum amplitude (mm) data following a Bonferroni multiple comparison test between treatment groups (n = 8 horses per treatment group). Data are mean 6 standard deviation Day Dexamethasone Saline 0 61.86 6 4.3 1 61.21 6 3.4 2 62.12 6 3.4 4 63.28 6 4.9 5 62.99 6 3.7 6 61.53 6 2.2
Mean 95% CI of difference difference
62.63 6 5.4 20.77 63.03 6 6.6 21.83 61.76 6 7.8 0.36 61.44 6 7.1 1.84 62.61 6 5.9 0.38 61.08 6 4.4 0.44
26.22 to 4.68 28.11 to 4.46 27.44 to 8.15 26.13 to 9.82 24.71 to 5.48 22.48 to 3.37
P-value . 0.99 . 0.99 . 0.99 . 0.99 . 0.99 . 0.99
could be clinically important. If this is the case, further studies with a larger sample size, a meta-analysis, or techniques to reduce data variability would be necessary to determine a more precise (narrower 95% CI) estimate of the difference between treatment groups. In contrast, if only larger differences in MA than those observed with the 95% CI are relevant it can be concluded that dexamethasone (in the protocol studied) does not lead to a statistically significant or clinically relevant increase in MA. Though we have used the upper limits as an example, a similar logic applies to interpreting the relevance of the lower limit values. While the current level of evidence for interpretation of TEG in horses is limited, precluding firm conclusions to be drawn from these data (4), the presentation of 95% CI maximizes the contribution of these data to the literature.
References 1. Woodman J, Wagg CR, Boysen SR, Leguillette R, Mizen K, Roy MF. Evaluation of coagulation via thromboelastography in healthy horses administered dexamethasone. Can Vet J 2015;56:1271–1274. 2. Gardner MJ, Altman DG. Confidence intervals rather than P-values: Estimation rather than hypothesis testing. Br Med J 1986;292:746–750. 3. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Br Med J 2010;340:c869. 4. Hanel RM, Chan DL, Conner B, et al. Systematic evaluation of evidence on veterinary viscoelastic testing Part 4: Definitions and data reporting. J Vet Emerg Crit Care 2014;24:47–56.
Veterinary Clinical and Diagnostic Sciences (Wagg, Pang), Faculty of Veterinary Medicine (UCVM) (Kwong), University of Calgary, Calgary, Alberta. Use of this article is limited to a single copy for personal study. Anyone interested in obtaining reprints should contact the CVMA office ([email protected]) for additional copies or permission to use this material elsewhere. CVJ / VOL 57 / MAY 2016
547
