READERS' FORUM
Letters to the editor* Statistical issues and ambiguities in a recent report on condylar position
3.
I
read with interest the article of Kandasamy et al (Kandasamy S, Boeddinghaus R, Kruger E. Condylar position assessed by magnetic resonance imaging after various bite position registrations. Am J Orthod Dentofacial Orthop 2013;144:512-7). I believe some points need clarification or correction. 1.
It was not mentioned on which basis the authors estimated the power. Was it based on a pilot study or a previous study? Since there was no cited previous study, it seems a pilot study. So, if the authors had seen in their pilot study that the differences were as small as 0.1 mm, why did they set their power to detect a 1.0-mm difference instead of a 0.1-mm difference? The mean difference inputted into the power formula (ie, the 1.0-mm difference) was too high. I think so because, according to the table, the average of mean (absolute) differences was 0.141 mm (ranging from 0.01 to 0.36 mm). Therefore, instead of 1 mm, the authors probably needed to calculate the sample size for detection of differences as small as about 0.14 mm. It is surprising that none of the comparisons became statistically significant when the calculated test power was 80%. I believe the reason is that the power calculation was incorrect, and the actual power was far smaller than reported. Calculating the power for detection of a much greater difference might have considerably inflated the calculated power and falsely reduced the required sample size. 2. To assess the variability and reproducibility of the techniques, the authors tested each technique thrice on 2 patients. Then they used repeated-measures ANOVA and interpreted its nonsignificant result as an indicator of the reliability of these methods (p. 514). It is incorrect. In such a small sample (2 participants 3 3 repeated experiments for each method), the lack of significance is very likely only a sign of the low test power (type II error). Besides, using repeated-measures ANOVA in such a small sample is likely incorrect (since normality and sphe-
*The viewpoints expressed are solely those of the author(s) and do not reflect those of the editor(s), publisher(s), or Association.
4.
5.
6.
7.
ricity are least likely to hold). The authors should probably have run a Friedman test instead. Two different ANOVA tests and a Tukey test were used. However, in the results, there is absolutely no mention of any statistical tests and their P values. It is only stated that there were no significant differences! So what were the P values? Both P 5 0.056 and P 5 0.950 are nonsignificant. So which one applied to this study? And if the ANOVA tests were nonsignificant, why was the Tukey test used? The authors stated that the results of all techniques were “highly variable” (p. 515). This is inconsistent with normality (which is an assumption of ANOVA tests). Thus, ANOVA usage should be justified, especially since there was no mention of normality assessments. Condylar positions were summarized only for CO. Why did the authors not calculate (and compare) the same for CR and Roth-CR? Lines 3 to 8 of the first paragraph of the “Discussion” are quite unclear. How could the authors “safely infer” something about differences between CR and Roth-CR from information pertaining to CO? In numerous parts of the “Discussion,” “Conclusions,” and “Abstract,” there are assertions that are not backed up by any statistical substantiations. The authors strongly discuss, interpret, and conclude (eg, that the Roth method is unjustified and so on) based on unsubstantiated or unreliable findings. The authors needed to state their limitations and warn the reader that their results were inconclusive. Vahid Rakhshan Tehran, Iran
Am J Orthod Dentofacial Orthop 2014;146:3 0889-5406/$36.00 Copyright Ó 2014 by the American Association of Orthodontists. http://dx.doi.org/10.1016/j.ajodo.2014.05.007
Authors' response
T
hank you for the opportunity to reply to Dr Rakhshan's letter regarding the apparent statistical “issues and ambiguities” with our research. 1.
The estimation of power for this study was not simply based on a pilot study, but also based on the 3
4
2.
3.
4.
5.
Readers' forum
only other study carried out using MRI to assess condylar position/displacement, and where displacements of 1 mm were detected.1 More importantly, is Dr Rakhshan actually advising the readership that mean (absolute) differences of 0.141 mm (ranging from 0.01 to 0.36 mm) in the glenoid fossae are something of a clinical health concern? Should we have actually adjusted the detection of differences to 0.1 mm? Would this have been clinically relevant? Repeated-measures ANOVA can be used to assess the variability of techniques. We also agree that with smaller samples a nonparametric test like the Friedman could have also been used, but we used parametric methods. In contradiction to the claims made by Dr Rakhshan, in our “Results” section, there is reference to the P values. All P values were lower than 0.05 and, as such, were not statistically significant. The Tukey was the post-hoc test to use if any significant differences were detected; however, there were none. The underlying distribution was deemed normal, and as such, ANOVA was used. Variable results do not necessarily indicate nonnormal underlying distributions. Condylar positions were discussed in relation to CO. Because the Roth power and CR bite registrations were supposedly capable of positioning condyles in the glenoid fossae in relation to CO, we discussed the differences in relation to CO.
July 2014 Vol 146 Issue 1
6.
7.
We do not agree that lines 3 to 8 are unclear; they are in fact quite the opposite. If there was any significant positioning difference that was detectable, we would have found it, but we did not. This was more than safe to infer. It appears that Dr Rakhshan has sadly and simply fixated his whole critique on his own interpretation of the statistics, claiming that our findings were unreliable or unsubstantiated. We are acutely aware that limitations do exist with any research. Our study simply and openly shows that the differences detected between the registrations were so small and highly variable that using certain bite registrations to accurately and predictably position condyles into specific locations in the glenoid fossae is not evidence based. Our findings are reliable and substantiated. Sanjivan Kandasamy Rudolf Boeddinghaus Estie Kruger Perth, Western Australia, Australia
Am J Orthod Dentofacial Orthop 2014;146:3-4 0889-5406/$36.00 Copyright Ó 2014 by the American Association of Orthodontists. http://dx.doi.org/10.1016/j.ajodo.2014.05.006
REFERENCE 1. Alexander SR, Moore RN, DuBois LM. Mandibular condyle position: comparison of articulator mountings and magnetic resonance imaging. Am J Orthod Dentofacial Orthop 1993;104:230-9.
American Journal of Orthodontics and Dentofacial Orthopedics
Letters to the editor* Statistical issues and ambiguities in a recent report on condylar position
3.
I
read with interest the article of Kandasamy et al (Kandasamy S, Boeddinghaus R, Kruger E. Condylar position assessed by magnetic resonance imaging after various bite position registrations. Am J Orthod Dentofacial Orthop 2013;144:512-7). I believe some points need clarification or correction. 1.
It was not mentioned on which basis the authors estimated the power. Was it based on a pilot study or a previous study? Since there was no cited previous study, it seems a pilot study. So, if the authors had seen in their pilot study that the differences were as small as 0.1 mm, why did they set their power to detect a 1.0-mm difference instead of a 0.1-mm difference? The mean difference inputted into the power formula (ie, the 1.0-mm difference) was too high. I think so because, according to the table, the average of mean (absolute) differences was 0.141 mm (ranging from 0.01 to 0.36 mm). Therefore, instead of 1 mm, the authors probably needed to calculate the sample size for detection of differences as small as about 0.14 mm. It is surprising that none of the comparisons became statistically significant when the calculated test power was 80%. I believe the reason is that the power calculation was incorrect, and the actual power was far smaller than reported. Calculating the power for detection of a much greater difference might have considerably inflated the calculated power and falsely reduced the required sample size. 2. To assess the variability and reproducibility of the techniques, the authors tested each technique thrice on 2 patients. Then they used repeated-measures ANOVA and interpreted its nonsignificant result as an indicator of the reliability of these methods (p. 514). It is incorrect. In such a small sample (2 participants 3 3 repeated experiments for each method), the lack of significance is very likely only a sign of the low test power (type II error). Besides, using repeated-measures ANOVA in such a small sample is likely incorrect (since normality and sphe-
*The viewpoints expressed are solely those of the author(s) and do not reflect those of the editor(s), publisher(s), or Association.
4.
5.
6.
7.
ricity are least likely to hold). The authors should probably have run a Friedman test instead. Two different ANOVA tests and a Tukey test were used. However, in the results, there is absolutely no mention of any statistical tests and their P values. It is only stated that there were no significant differences! So what were the P values? Both P 5 0.056 and P 5 0.950 are nonsignificant. So which one applied to this study? And if the ANOVA tests were nonsignificant, why was the Tukey test used? The authors stated that the results of all techniques were “highly variable” (p. 515). This is inconsistent with normality (which is an assumption of ANOVA tests). Thus, ANOVA usage should be justified, especially since there was no mention of normality assessments. Condylar positions were summarized only for CO. Why did the authors not calculate (and compare) the same for CR and Roth-CR? Lines 3 to 8 of the first paragraph of the “Discussion” are quite unclear. How could the authors “safely infer” something about differences between CR and Roth-CR from information pertaining to CO? In numerous parts of the “Discussion,” “Conclusions,” and “Abstract,” there are assertions that are not backed up by any statistical substantiations. The authors strongly discuss, interpret, and conclude (eg, that the Roth method is unjustified and so on) based on unsubstantiated or unreliable findings. The authors needed to state their limitations and warn the reader that their results were inconclusive. Vahid Rakhshan Tehran, Iran
Am J Orthod Dentofacial Orthop 2014;146:3 0889-5406/$36.00 Copyright Ó 2014 by the American Association of Orthodontists. http://dx.doi.org/10.1016/j.ajodo.2014.05.007
Authors' response
T
hank you for the opportunity to reply to Dr Rakhshan's letter regarding the apparent statistical “issues and ambiguities” with our research. 1.
The estimation of power for this study was not simply based on a pilot study, but also based on the 3
4
2.
3.
4.
5.
Readers' forum
only other study carried out using MRI to assess condylar position/displacement, and where displacements of 1 mm were detected.1 More importantly, is Dr Rakhshan actually advising the readership that mean (absolute) differences of 0.141 mm (ranging from 0.01 to 0.36 mm) in the glenoid fossae are something of a clinical health concern? Should we have actually adjusted the detection of differences to 0.1 mm? Would this have been clinically relevant? Repeated-measures ANOVA can be used to assess the variability of techniques. We also agree that with smaller samples a nonparametric test like the Friedman could have also been used, but we used parametric methods. In contradiction to the claims made by Dr Rakhshan, in our “Results” section, there is reference to the P values. All P values were lower than 0.05 and, as such, were not statistically significant. The Tukey was the post-hoc test to use if any significant differences were detected; however, there were none. The underlying distribution was deemed normal, and as such, ANOVA was used. Variable results do not necessarily indicate nonnormal underlying distributions. Condylar positions were discussed in relation to CO. Because the Roth power and CR bite registrations were supposedly capable of positioning condyles in the glenoid fossae in relation to CO, we discussed the differences in relation to CO.
July 2014 Vol 146 Issue 1
6.
7.
We do not agree that lines 3 to 8 are unclear; they are in fact quite the opposite. If there was any significant positioning difference that was detectable, we would have found it, but we did not. This was more than safe to infer. It appears that Dr Rakhshan has sadly and simply fixated his whole critique on his own interpretation of the statistics, claiming that our findings were unreliable or unsubstantiated. We are acutely aware that limitations do exist with any research. Our study simply and openly shows that the differences detected between the registrations were so small and highly variable that using certain bite registrations to accurately and predictably position condyles into specific locations in the glenoid fossae is not evidence based. Our findings are reliable and substantiated. Sanjivan Kandasamy Rudolf Boeddinghaus Estie Kruger Perth, Western Australia, Australia
Am J Orthod Dentofacial Orthop 2014;146:3-4 0889-5406/$36.00 Copyright Ó 2014 by the American Association of Orthodontists. http://dx.doi.org/10.1016/j.ajodo.2014.05.006
REFERENCE 1. Alexander SR, Moore RN, DuBois LM. Mandibular condyle position: comparison of articulator mountings and magnetic resonance imaging. Am J Orthod Dentofacial Orthop 1993;104:230-9.
American Journal of Orthodontics and Dentofacial Orthopedics
