G Model
JINJ-6147; No. of Pages 5 Injury, Int. J. Care Injured xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Injury journal homepage: www.elsevier.com/locate/injury
Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial Robert J. Derksen a,*, Lisa M. Knijnenberg b, Gerwin Fransen c, Roelf S. Breederveld a, Martijn W. Heymans d, Inger B. Schipper e a
Department of Surgery, Red Cross Hospital, Beverwijk, The Netherlands Department of Emergency Medicine, Academic Medical Centre, Amsterdam, The Netherlands Department of Radiology, Zaans Medical Centre, Zaandam, The Netherlands d Department of Epidemiology and Biostatistics, VU University Medical Centre, Amsterdam, The Netherlands e Department of Traumasurgery, Leiden University Medical Centre, Leiden, The Netherlands b c
A R T I C L E I N F O
A B S T R A C T
Article history: Accepted 30 March 2015
Purpose: The Ottawa ankle rules (OAR) brought about a reduction of radiographs on the Emergency Department (ED). However, still 50% of patients with ankle injuries undergo unnecessary radiography. Compared to the OAR, the Bernese ankle rule (BAR) has an acclaimed 84% reduction in radiography without loss of sensitivity. The primary aim of this study was to compare the diagnostic accuracy and reproducibility of both rules. Furthermore, the ability of triage nurses to accurately interpret the BAR was assessed. Methods: Participants were assessed by both the ED resident and the triage nurse, applying the OAR and the BAR. After standardised data collection, ankle and foot radiographs were performed in all patients. Sensitivity and specificity of both tests applied by both observers were obtained and compared by McNemar’s test. Reproducibility was calculated with Cohen’s kappa. Results: A total of 203 patients with ankle trauma were included. For the OAR obtained by the ED residents, the sensitivity and specificity were 0.97 and 0.29, respectively. For the BAR, the sensitivity and specificity of the ED residents were 0.69 and 0.45, respectively. For the triage nurses, the OAR sensitivity and specificity were 0.86 and 0.25, respectively. The BAR sensitivity and specificity for the nurses were 0.86 and 0.40, respectively. The reproducibility of the OAR was 0.45, and for the BAR, it was 0.48. Conclusion: Both rules showed comparable reproducibility. Although the BAR showed a superior specificity compared to the OAR, its sensitivity was too low to promote clinical use. The triage nurses demonstrated too low sensitivity on both rules to allow safe application. Therefore, the OAR remain the decision rules of choice for ankle injuries despite its modest ‘ruling out’ capacity. ß 2015 Elsevier Ltd. All rights reserved.
Keywords: Ankle distortion Bernese ankle rule Ottawa ankle rules Diagnostic accuracy Reproducibility
Introduction Patients with acute ankle and midfoot trauma constitute approximately 5% of all Emergency Department (ED) presentations [1]. In about 15% of patients visiting the ED with ankle or midfoot trauma, a fracture is diagnosed [2]. To decide if radiography is needed, in 1992 Stiell et al. designed the Ottawa ankle rules (OAR) [3] (Fig. 1). The OAR have an excellent potential to detect fractures with a sensitivity of nearly 100%, and they are routinely applied at EDs.
* Corresponding author at: Department of Surgery, Red Cross Hospital, Vondellaan 13, 1942 LE Beverwijk, The Netherlands Tel.: +31 251 655344. E-mail address: [email protected] (R.J. Derksen).
However, the capacity to rule out fractures is much less accurate with a reported specificity of only 32% [3]. Although the OAR reduced the number of ordered radiographs without missing significant fractures, the number of patients undergoing radiography still remains high (64%). As only 15% of ankle and midfoot injuries concern fractures, approximately 50% of all injured patients undergo unnecessary radiography on the basis of the OAR findings. The use of the OAR therefore leads to unnecessary radiation exposure, increased waiting times, and medical costs. For this reason in 2003, Eggli et al. developed the Bernese ankle rule (BAR) [4]. The BAR consists of three items: indirect fibular stress, direct medial malleolar stress, and compression stress of the midfoot and hindfoot (Fig. 2). In the original study, all fractures were detected correctly by the application of the BAR, resulting in a sensitivity of 100%. They
http://dx.doi.org/10.1016/j.injury.2015.03.038 0020–1383/ß 2015 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Derksen RJ, et al. Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial. Injury (2015), http://dx.doi.org/10.1016/j.injury.2015.03.038
G Model
JINJ-6147; No. of Pages 5 2
R.J. Derksen et al. / Injury, Int. J. Care Injured xxx (2015) xxx–xxx
Fig. 1. The Ottawa ankle rules. Ankle radiography is required if there is pain in the malleolar zone, and if one of the following is present: bone tenderness at the posterior edge of the lateral (A) or the medial (B) malleolus or the inability to bear weight (four steps) immediately and in the ED. Foot radiography is required if there is pain in the midfoot zone, and if one of the following is present: bone tenderness at the base of the fifth metatarsal (C) or at the navicular (D) or the inability to bear weight (four steps) immediately and in the ED.
found a remarkably high specificity of 91%. Based on these results, a possible reduction of 84% of ankle and midfoot radiographs could be achieved. These results are promising; however, to date, only two studies have compared the OAR with the BAR [5,6]. In these studies, a sensitivity of the BAR ranging from 56% to 94% and a specificity of 79–95% were found compared to 75–100% and 69–77%, respectively, for the OAR. This variation in diagnostic characteristics is too large to ascertain a dependable sensitivity and specificity for the BAR. Furthermore, the interobserver agreement (reproducibility), another important clinimetric property of a diagnostic test, has not been assessed in the previously mentioned studies. Furthermore, in many hospitals, triage nurses assess patients with ankle distortions. The nurses order radiographs after the application of a decision rule. It is therefore important to assess whether triage nurses are capable of interpreting the ankle rules with sufficient accuracy. This study was primarily aimed at assessing and comparing the diagnostic accuracy of the BAR versus the OAR. Secondly, the diagnostic accuracy of triage nurses in interpreting both rules is compared to ED residents. Finally, the study aimed at assessing and comparing the reproducibility of both tests. Our null hypothesis is that the sensitivity of BAR is non-inferior to the OAR, and that the specificity of the BAR is significantly higher than that of the OAR. The triage nurses are expected to show noninferior accuracy results compared to the ED residents for both rules. Furthermore, the reproducibility of the BAR is expected to be higher than that of the OAR due to fewer items in the rule.
Materials and methods Study design and setting A double randomised (both for first observer and for first rule applied), single-blinded (radiologist), controlled, interobserver trial was performed between November 2013 and August 2014. This monocentre study was conducted in the ED of an urban teaching hospital (patient census 30,000 visits/year). Randomisation occurred by the drawing of concealed envelopes. The study protocol was approved by both the regional and the local medical ethics committees as required in the Netherlands (registration number NL43168.094.13). The research project was carried out in accordance with the Declaration of Helsinki (2013) of the World Medical Association. Written informed consent was obtained from all study participants. Selection of participants Patients were eligible for the study when they had sustained a foot or an ankle sprain within 48 h prior to presentation in the ED. Included patients needed to be 18 years. Patients were excluded if they were unwilling to provide written informed consent, if the ankle sprain was part of a multitrauma, if there was a history of ankle or midfoot fracture on the ipsilateral side, and in case of substance abuse interfering with pain perception and in case of mental or physical disabilities, which could lead to an unreliable
Fig. 2. The Bernese ankle rule. The rules are positive and indicate the need for radiography if one of these steps caused pain. (1) Indirect fibular stress: the malleolar fork is compressed approximately 10 cm proximally to the fibular tip. (2) Direct medial malleolar stress: the thumb is pressed flat on the medial malleolus. (3) Compression stress of the midfoot and the hindfoot: one hand fixes the calcaneus in a neutral position and the other hand applies a sagittal load on the forefoot, so that the midfoot and hindfoot are compressed.
Please cite this article in press as: Derksen RJ, et al. Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial. Injury (2015), http://dx.doi.org/10.1016/j.injury.2015.03.038
G Model
JINJ-6147; No. of Pages 5 R.J. Derksen et al. / Injury, Int. J. Care Injured xxx (2015) xxx–xxx
3
Ankle and/or midfoot injury
Assessment ED resident
• • •
First observerrandomised First rulerandomised CRF filled in
Assessment triage nurse
OAR + BAR
OAR + BAR
X-ray of ankle and foot
Definitive diagnosis and treatment by EDR Fig. 3. A schematic representation of the study protocol. After inclusion, the first observer (Emergency Department resident (EDR) versus triage nurse (TN)) and the first rule applied (Ottawa ankle rules (OARs) versus Bernese ankle rules (BARs)) were randomised. Both observers filled in their findings on standardised case-report forms (CRFs), without seeing the CRF of the other observer. After clinical examination, all patients had radiography of the ankle and the foot, after which the EDR treated the injury according to his/her own findings, and to the hospital protocol.
assessment (e.g., dementia, clubfoot, etc.). Patients were enrolled 24 h a day, 7 days a week. Interventions Both the triage nurse and the ED resident interpreted both decision rules in each patient (Fig. 3). The observers were blinded to each other’s findings, and they recorded these findings on separate standardised case-report forms. The forms were put in an envelope that was sealed, after which the participant underwent radiography of both the ankle and the foot. The participants were treated onwards by the ED resident as usual (according to the hospital protocol). Outcome measures Primary outcome measures were the sensitivity and specificity of both decision rules as applied by the ED residents. Furthermore, these accuracy parameters were compared between observer groups (the ED residents and the triage nurses) to determine whether both groups could accurately interpret the rules. The tertiary outcome measure consisted of the interobserver agreement (reproducibility) between both types of observers. Data analysis Both for the ED residents and for the triage nurses, the sensitivity and specificity were calculated for both decision rules together with their 95% confidence intervals. Consequently, these accuracy parameters were compared by McNemar’s test. P < 0.05 was considered statistically significant (Statistical Package for Social Science (SPSS) 21, Chicago, IL, USA). The interobserver agreement (reproducibility) between the ED residents and the triage nurses was calculated by Cohen’s kappa (SPSS 21, Chicago, IL, USA). Sample size Our primary aim was to establish whether the BAR was more specific than the OAR with comparably high sensitivity. The specificity of the BAR was considered to differ clinically meaningful compared to the OAR when the difference would be >12.5% in ruling out clinically significant fractures (e.g., those requiring
treatment with a cast or an operation). The sample-size calculation was performed using StatsToDo with which a two-sided test for paired comparisons was performed (StatsToDo trading Pty Ltd., Brisbane, Australia). Considering the specificity of the OAR being 31.5% (power, 80%; a, 0.05), the sample size required should be at least 187 patients. Observer groups The ED residents groups consisted of 10 residents. The mean age for the ED resident group was 27 years (range 25–30). The mean clinical experience in emergency medicine was 1 year (range 0–2). The ED residents group consisted of two males and eight females. The triage nurses group consisted of 22 ED nurses. The mean age for the triage nurses was 42 (range 27–59) years, and their clinical experience on the ED was 5 years (range 2–15). The triage nurses groups consisted of 5 males and 17 females. Both observer groups were instructed on how to apply both rules by a hands-on course that took approximately 1 h, and this course was given by an experienced trauma surgeon. Results During the 10-month study (November 2013–August 2014), 604 patients were assessed for eligibility to enter the study. In total, 401 patients did not meet the inclusion and exclusion criteria, or they refused to participate in the study. Consequently, Table 1 Characteristics and fracture type distribution of the patients. Mean age (years) (range) Sex, n (%) Male Female Distribution of fractures, n (%) Weber A Weber B Weber C Trimalleolar Weber B Malleolus tertius Fifth metatarsal (including Jones) Fifth and fourth metatarsals Lisfranc injury Navicular
37 (18–73)
95 (46.8) 108 (53.2)
4 12 1 1 1 7 1 1 1
(13.8) (41.4) (3.4) (3.4) (3.4) (24.1) (3.4) (3.4) (3.4)
Please cite this article in press as: Derksen RJ, et al. Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial. Injury (2015), http://dx.doi.org/10.1016/j.injury.2015.03.038
G Model
JINJ-6147; No. of Pages 5 R.J. Derksen et al. / Injury, Int. J. Care Injured xxx (2015) xxx–xxx
4
Table 2 The diagnostic accuracy (95% confidence interval) of the Ottawa ankle rules (OAR) and the Bernese ankle rule (BAR) in the Emergency Department resident group (ED resident) and the triage nurses group is displayed. In the column on the right, P-values are shown that were found by comparing the accuracy variables from the same row using McNemar’s test. The same applies for the P-values displayed in the lowest row; these P-values were found by comparing the accuracy variables in the same column using McNemar’s test. Rule
Observer ED residents
BAR sensitivity BAR specificity OAR sensitivity OAR specificity P-value
ED residents
69 (49–84)% 45(38–53)% 97 (82–99)%
Primary outcome (accuracy of the OAR versus the BAR) The sensitivity of the OAR as interpreted by the ED residents was 97% compared to 69% for the BAR. This difference was statistically significant (P = 0.008). The specificity of the OAR as judged by the ED residents (29%) was significantly lower than the specificity of the BAR (45%) in the same observer group (P < 0.001) (Table 2). Secondary outcome (accuracy of the ED residents versus the triage nurses) The sensitivity of the OAR interpreted by the ED residents was 97% compared to a sensitivity of 86% for the OAR judged by the triage nurses. This difference did not reach statistical significance (P = 0.38). The sensitivity for the BAR interpreted by the ED residents was 69%, and it was not significantly different from the BAR sensitivity (86%) judged by the triage nurses (P = 0.13). The specificity of the OAR applied by the ED residents was 29%, and it was not significantly different from the OAR specificity of 25% for the triage nurses (P = 0.41). Finally, the specificity for the BAR interpreted by the ED residents (45%) was not significantly different from the BAR specificity in the triage nurses group (40%, P = 0.17). Tertiary outcome (reproducibility) The interobserver agreement of the OAR as a whole evaluated by Cohen’s kappa was 0.45, and it is considered moderate according to the Koch and Landis categories of Cohen’s kappa values [7]. The interobserver agreement (Cohen’s kappa) of the BAR as a whole was 0.48, and it also considered moderate agreement. The interobserver agreement (Cohen’s kappa) of individual items from the OAR ranged from 0.37 to 0.63. Individual items from the BAR ranged from 0.35 to 0.49 (Table 3). Table 3 Interobserver agreement (Cohen’s kappa) for individual items of the Ottawa ankle rules (OAR) and the Bernese ankle rule (BAR). Furthermore, the overall agreement (Cohen’s kappa) of both rules is shown. BAR
OAR 0.45 0.53 0.38 0.63 0.36 0.37
40 (32–47)% 86 (68–96)%
29 (22–36)%
JINJ-6147; No. of Pages 5 Injury, Int. J. Care Injured xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Injury journal homepage: www.elsevier.com/locate/injury
Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial Robert J. Derksen a,*, Lisa M. Knijnenberg b, Gerwin Fransen c, Roelf S. Breederveld a, Martijn W. Heymans d, Inger B. Schipper e a
Department of Surgery, Red Cross Hospital, Beverwijk, The Netherlands Department of Emergency Medicine, Academic Medical Centre, Amsterdam, The Netherlands Department of Radiology, Zaans Medical Centre, Zaandam, The Netherlands d Department of Epidemiology and Biostatistics, VU University Medical Centre, Amsterdam, The Netherlands e Department of Traumasurgery, Leiden University Medical Centre, Leiden, The Netherlands b c
A R T I C L E I N F O
A B S T R A C T
Article history: Accepted 30 March 2015
Purpose: The Ottawa ankle rules (OAR) brought about a reduction of radiographs on the Emergency Department (ED). However, still 50% of patients with ankle injuries undergo unnecessary radiography. Compared to the OAR, the Bernese ankle rule (BAR) has an acclaimed 84% reduction in radiography without loss of sensitivity. The primary aim of this study was to compare the diagnostic accuracy and reproducibility of both rules. Furthermore, the ability of triage nurses to accurately interpret the BAR was assessed. Methods: Participants were assessed by both the ED resident and the triage nurse, applying the OAR and the BAR. After standardised data collection, ankle and foot radiographs were performed in all patients. Sensitivity and specificity of both tests applied by both observers were obtained and compared by McNemar’s test. Reproducibility was calculated with Cohen’s kappa. Results: A total of 203 patients with ankle trauma were included. For the OAR obtained by the ED residents, the sensitivity and specificity were 0.97 and 0.29, respectively. For the BAR, the sensitivity and specificity of the ED residents were 0.69 and 0.45, respectively. For the triage nurses, the OAR sensitivity and specificity were 0.86 and 0.25, respectively. The BAR sensitivity and specificity for the nurses were 0.86 and 0.40, respectively. The reproducibility of the OAR was 0.45, and for the BAR, it was 0.48. Conclusion: Both rules showed comparable reproducibility. Although the BAR showed a superior specificity compared to the OAR, its sensitivity was too low to promote clinical use. The triage nurses demonstrated too low sensitivity on both rules to allow safe application. Therefore, the OAR remain the decision rules of choice for ankle injuries despite its modest ‘ruling out’ capacity. ß 2015 Elsevier Ltd. All rights reserved.
Keywords: Ankle distortion Bernese ankle rule Ottawa ankle rules Diagnostic accuracy Reproducibility
Introduction Patients with acute ankle and midfoot trauma constitute approximately 5% of all Emergency Department (ED) presentations [1]. In about 15% of patients visiting the ED with ankle or midfoot trauma, a fracture is diagnosed [2]. To decide if radiography is needed, in 1992 Stiell et al. designed the Ottawa ankle rules (OAR) [3] (Fig. 1). The OAR have an excellent potential to detect fractures with a sensitivity of nearly 100%, and they are routinely applied at EDs.
* Corresponding author at: Department of Surgery, Red Cross Hospital, Vondellaan 13, 1942 LE Beverwijk, The Netherlands Tel.: +31 251 655344. E-mail address: [email protected] (R.J. Derksen).
However, the capacity to rule out fractures is much less accurate with a reported specificity of only 32% [3]. Although the OAR reduced the number of ordered radiographs without missing significant fractures, the number of patients undergoing radiography still remains high (64%). As only 15% of ankle and midfoot injuries concern fractures, approximately 50% of all injured patients undergo unnecessary radiography on the basis of the OAR findings. The use of the OAR therefore leads to unnecessary radiation exposure, increased waiting times, and medical costs. For this reason in 2003, Eggli et al. developed the Bernese ankle rule (BAR) [4]. The BAR consists of three items: indirect fibular stress, direct medial malleolar stress, and compression stress of the midfoot and hindfoot (Fig. 2). In the original study, all fractures were detected correctly by the application of the BAR, resulting in a sensitivity of 100%. They
http://dx.doi.org/10.1016/j.injury.2015.03.038 0020–1383/ß 2015 Elsevier Ltd. All rights reserved.
Please cite this article in press as: Derksen RJ, et al. Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial. Injury (2015), http://dx.doi.org/10.1016/j.injury.2015.03.038
G Model
JINJ-6147; No. of Pages 5 2
R.J. Derksen et al. / Injury, Int. J. Care Injured xxx (2015) xxx–xxx
Fig. 1. The Ottawa ankle rules. Ankle radiography is required if there is pain in the malleolar zone, and if one of the following is present: bone tenderness at the posterior edge of the lateral (A) or the medial (B) malleolus or the inability to bear weight (four steps) immediately and in the ED. Foot radiography is required if there is pain in the midfoot zone, and if one of the following is present: bone tenderness at the base of the fifth metatarsal (C) or at the navicular (D) or the inability to bear weight (four steps) immediately and in the ED.
found a remarkably high specificity of 91%. Based on these results, a possible reduction of 84% of ankle and midfoot radiographs could be achieved. These results are promising; however, to date, only two studies have compared the OAR with the BAR [5,6]. In these studies, a sensitivity of the BAR ranging from 56% to 94% and a specificity of 79–95% were found compared to 75–100% and 69–77%, respectively, for the OAR. This variation in diagnostic characteristics is too large to ascertain a dependable sensitivity and specificity for the BAR. Furthermore, the interobserver agreement (reproducibility), another important clinimetric property of a diagnostic test, has not been assessed in the previously mentioned studies. Furthermore, in many hospitals, triage nurses assess patients with ankle distortions. The nurses order radiographs after the application of a decision rule. It is therefore important to assess whether triage nurses are capable of interpreting the ankle rules with sufficient accuracy. This study was primarily aimed at assessing and comparing the diagnostic accuracy of the BAR versus the OAR. Secondly, the diagnostic accuracy of triage nurses in interpreting both rules is compared to ED residents. Finally, the study aimed at assessing and comparing the reproducibility of both tests. Our null hypothesis is that the sensitivity of BAR is non-inferior to the OAR, and that the specificity of the BAR is significantly higher than that of the OAR. The triage nurses are expected to show noninferior accuracy results compared to the ED residents for both rules. Furthermore, the reproducibility of the BAR is expected to be higher than that of the OAR due to fewer items in the rule.
Materials and methods Study design and setting A double randomised (both for first observer and for first rule applied), single-blinded (radiologist), controlled, interobserver trial was performed between November 2013 and August 2014. This monocentre study was conducted in the ED of an urban teaching hospital (patient census 30,000 visits/year). Randomisation occurred by the drawing of concealed envelopes. The study protocol was approved by both the regional and the local medical ethics committees as required in the Netherlands (registration number NL43168.094.13). The research project was carried out in accordance with the Declaration of Helsinki (2013) of the World Medical Association. Written informed consent was obtained from all study participants. Selection of participants Patients were eligible for the study when they had sustained a foot or an ankle sprain within 48 h prior to presentation in the ED. Included patients needed to be 18 years. Patients were excluded if they were unwilling to provide written informed consent, if the ankle sprain was part of a multitrauma, if there was a history of ankle or midfoot fracture on the ipsilateral side, and in case of substance abuse interfering with pain perception and in case of mental or physical disabilities, which could lead to an unreliable
Fig. 2. The Bernese ankle rule. The rules are positive and indicate the need for radiography if one of these steps caused pain. (1) Indirect fibular stress: the malleolar fork is compressed approximately 10 cm proximally to the fibular tip. (2) Direct medial malleolar stress: the thumb is pressed flat on the medial malleolus. (3) Compression stress of the midfoot and the hindfoot: one hand fixes the calcaneus in a neutral position and the other hand applies a sagittal load on the forefoot, so that the midfoot and hindfoot are compressed.
Please cite this article in press as: Derksen RJ, et al. Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial. Injury (2015), http://dx.doi.org/10.1016/j.injury.2015.03.038
G Model
JINJ-6147; No. of Pages 5 R.J. Derksen et al. / Injury, Int. J. Care Injured xxx (2015) xxx–xxx
3
Ankle and/or midfoot injury
Assessment ED resident
• • •
First observerrandomised First rulerandomised CRF filled in
Assessment triage nurse
OAR + BAR
OAR + BAR
X-ray of ankle and foot
Definitive diagnosis and treatment by EDR Fig. 3. A schematic representation of the study protocol. After inclusion, the first observer (Emergency Department resident (EDR) versus triage nurse (TN)) and the first rule applied (Ottawa ankle rules (OARs) versus Bernese ankle rules (BARs)) were randomised. Both observers filled in their findings on standardised case-report forms (CRFs), without seeing the CRF of the other observer. After clinical examination, all patients had radiography of the ankle and the foot, after which the EDR treated the injury according to his/her own findings, and to the hospital protocol.
assessment (e.g., dementia, clubfoot, etc.). Patients were enrolled 24 h a day, 7 days a week. Interventions Both the triage nurse and the ED resident interpreted both decision rules in each patient (Fig. 3). The observers were blinded to each other’s findings, and they recorded these findings on separate standardised case-report forms. The forms were put in an envelope that was sealed, after which the participant underwent radiography of both the ankle and the foot. The participants were treated onwards by the ED resident as usual (according to the hospital protocol). Outcome measures Primary outcome measures were the sensitivity and specificity of both decision rules as applied by the ED residents. Furthermore, these accuracy parameters were compared between observer groups (the ED residents and the triage nurses) to determine whether both groups could accurately interpret the rules. The tertiary outcome measure consisted of the interobserver agreement (reproducibility) between both types of observers. Data analysis Both for the ED residents and for the triage nurses, the sensitivity and specificity were calculated for both decision rules together with their 95% confidence intervals. Consequently, these accuracy parameters were compared by McNemar’s test. P < 0.05 was considered statistically significant (Statistical Package for Social Science (SPSS) 21, Chicago, IL, USA). The interobserver agreement (reproducibility) between the ED residents and the triage nurses was calculated by Cohen’s kappa (SPSS 21, Chicago, IL, USA). Sample size Our primary aim was to establish whether the BAR was more specific than the OAR with comparably high sensitivity. The specificity of the BAR was considered to differ clinically meaningful compared to the OAR when the difference would be >12.5% in ruling out clinically significant fractures (e.g., those requiring
treatment with a cast or an operation). The sample-size calculation was performed using StatsToDo with which a two-sided test for paired comparisons was performed (StatsToDo trading Pty Ltd., Brisbane, Australia). Considering the specificity of the OAR being 31.5% (power, 80%; a, 0.05), the sample size required should be at least 187 patients. Observer groups The ED residents groups consisted of 10 residents. The mean age for the ED resident group was 27 years (range 25–30). The mean clinical experience in emergency medicine was 1 year (range 0–2). The ED residents group consisted of two males and eight females. The triage nurses group consisted of 22 ED nurses. The mean age for the triage nurses was 42 (range 27–59) years, and their clinical experience on the ED was 5 years (range 2–15). The triage nurses groups consisted of 5 males and 17 females. Both observer groups were instructed on how to apply both rules by a hands-on course that took approximately 1 h, and this course was given by an experienced trauma surgeon. Results During the 10-month study (November 2013–August 2014), 604 patients were assessed for eligibility to enter the study. In total, 401 patients did not meet the inclusion and exclusion criteria, or they refused to participate in the study. Consequently, Table 1 Characteristics and fracture type distribution of the patients. Mean age (years) (range) Sex, n (%) Male Female Distribution of fractures, n (%) Weber A Weber B Weber C Trimalleolar Weber B Malleolus tertius Fifth metatarsal (including Jones) Fifth and fourth metatarsals Lisfranc injury Navicular
37 (18–73)
95 (46.8) 108 (53.2)
4 12 1 1 1 7 1 1 1
(13.8) (41.4) (3.4) (3.4) (3.4) (24.1) (3.4) (3.4) (3.4)
Please cite this article in press as: Derksen RJ, et al. Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial. Injury (2015), http://dx.doi.org/10.1016/j.injury.2015.03.038
G Model
JINJ-6147; No. of Pages 5 R.J. Derksen et al. / Injury, Int. J. Care Injured xxx (2015) xxx–xxx
4
Table 2 The diagnostic accuracy (95% confidence interval) of the Ottawa ankle rules (OAR) and the Bernese ankle rule (BAR) in the Emergency Department resident group (ED resident) and the triage nurses group is displayed. In the column on the right, P-values are shown that were found by comparing the accuracy variables from the same row using McNemar’s test. The same applies for the P-values displayed in the lowest row; these P-values were found by comparing the accuracy variables in the same column using McNemar’s test. Rule
Observer ED residents
BAR sensitivity BAR specificity OAR sensitivity OAR specificity P-value
ED residents
69 (49–84)% 45(38–53)% 97 (82–99)%
Primary outcome (accuracy of the OAR versus the BAR) The sensitivity of the OAR as interpreted by the ED residents was 97% compared to 69% for the BAR. This difference was statistically significant (P = 0.008). The specificity of the OAR as judged by the ED residents (29%) was significantly lower than the specificity of the BAR (45%) in the same observer group (P < 0.001) (Table 2). Secondary outcome (accuracy of the ED residents versus the triage nurses) The sensitivity of the OAR interpreted by the ED residents was 97% compared to a sensitivity of 86% for the OAR judged by the triage nurses. This difference did not reach statistical significance (P = 0.38). The sensitivity for the BAR interpreted by the ED residents was 69%, and it was not significantly different from the BAR sensitivity (86%) judged by the triage nurses (P = 0.13). The specificity of the OAR applied by the ED residents was 29%, and it was not significantly different from the OAR specificity of 25% for the triage nurses (P = 0.41). Finally, the specificity for the BAR interpreted by the ED residents (45%) was not significantly different from the BAR specificity in the triage nurses group (40%, P = 0.17). Tertiary outcome (reproducibility) The interobserver agreement of the OAR as a whole evaluated by Cohen’s kappa was 0.45, and it is considered moderate according to the Koch and Landis categories of Cohen’s kappa values [7]. The interobserver agreement (Cohen’s kappa) of the BAR as a whole was 0.48, and it also considered moderate agreement. The interobserver agreement (Cohen’s kappa) of individual items from the OAR ranged from 0.37 to 0.63. Individual items from the BAR ranged from 0.35 to 0.49 (Table 3). Table 3 Interobserver agreement (Cohen’s kappa) for individual items of the Ottawa ankle rules (OAR) and the Bernese ankle rule (BAR). Furthermore, the overall agreement (Cohen’s kappa) of both rules is shown. BAR
OAR 0.45 0.53 0.38 0.63 0.36 0.37
40 (32–47)% 86 (68–96)%
29 (22–36)%
