ORIGINAL ARTICLE
Publication Rate of Abstracts Presented at European Paediatric Orthopaedic Society Annual Meetings, 2006 to 2008 Marie-Theres Kleine-Konig,* Tobias L. Schulte, MD,w Georg Gosheger, MD,w Robert Ro¨dl, MD,* and Frank M. Schiedel, MD*
Background: The quality of the abstracts presented at a conference reflects the scientific work and level of activity of the scientific association concerned. The aim of the present study was to determine the rate of publications of podium presentations and posters at the conferences of the European Paediatric Orthopaedic Society (EPOS) from 2006 to 2008 and to identify factors that favor publication in peer-reviewed journals. The results are compared with those of other international societies. Methods: All 646 abstracts (including podiums, posters, and e-posters) presented at the EPOS conferences were investigated using the PubMed database to identify any corresponding published articles in the journals listed in the database. A period of 5 years before and after the relevant conference was used for the PubMed search. Factors influencing publication and the quality of the study, such as the type of presentation and the level of evidence, were also investigated. Results: A publication rate of 36.7% was observed, corresponding to 237 publications of 646 abstracts. The period to publication showed a mean of 13.88 ± 1.34 months. It was found that abstracts of podium presentations were published significantly more often than poster abstracts (P < 0.001). Experimental studies, with a publication rate of 50.9%, showed better results than clinical studies (36.0%). Overall, the articles were published in 61 different journals, with the largest number (n = 50) appearing in the Journal of Pediatric Orthopaedics. In addition, the present study shows that abstracts with a higher level of evidence were associated with a higher publication rate. Conclusions: At 36.7%, the rate of publication of EPOS abstracts is within the range reached by other specialist orthopaedics societies, such as the German Society of Orthopaedics and Trauma Surgery (36%) and the British Orthopaedic Association (36%). However, it is lower than the publication rate of the Pediatric Orthopaedic Society of North America (POSNA), at 50%. The high percentage of unpublished conference From the *Department of Pediatric Orthopaedics, Deformity Correction and Foot Surgery; and wDepartment of Orthopaedics and Tumor Orthopaedics, Mu¨nster University Hospital, Mu¨nster, Germany. The authors declare no conflicts of interest. Reprints: Frank M. Schiedel, MD, Department of Pediatric Orthopaedics, Deformity Correction, and Foot Surgery, Mu¨nster University Hospital, Albert-Schweitzer-Campus 1, D-48149 Mu¨nster, Germany. E-mail: [email protected]. Copyright r 2014 by Lippincott Williams & Wilkins
J Pediatr Orthop
Volume 34, Number 6, September 2014
abstracts (63%), which did not go through a peer-reviewed process, casts doubts upon the practice of utilizing the citation of abstracts based purely on conference abstracts. Level of Evidence: Statistical study. Key Words: impact, proceedings, EPOS, publication rate, annual meeting, abstract, oral, presentation, poster presentation, level of evidence (J Pediatr Orthop 2014;34:e33–e38)
I
n the field of pediatric orthopaedics, as in every other medical specialty, it is extremely important to present one’s scientific research to the public. Opportunities to do this are provided by conferences and by scientific journals in which original research studies are published. Presentations at conferences are kept quite short and are concerned only with the most important data. This is intended to ensure a rapid chain of publication, from data collection to presentation. Publication in a scientific journal is regarded as the “gold standard” for the dissemination of scientific research,1 as this type of article provides a full account of the research study and is subject to peer review, so that the quality of the research can be regarded as ensured. Lectures and posters presented at conferences, nevertheless, have a considerable influence on scientific research and everyday work, and they are often cited in articles so that conference lectures and posters represent an important component of medical training.1–4 Ensuring the quality of papers presented at conferences should therefore be an important concern for medical societies. Publication in a high-quality peer-reviewed journal is a generally recognized sign of the quality of a study.5 The aim of the present study was to investigate the quality of presentations at the annual European Paediatric Orthopaedic Society (EPOS) conference by determining the rate of publication of lectures and posters presented from 2006 to 2008. In addition, the conference presentations and published articles were investigated to identify any criteria that might have favored publication. This method was intended to allow comparison of the results of the study with publication rates (PR) and results from other conferences on similar topics.1,2,4–13 www.pedorthopaedics.com |
e33
J Pediatr Orthop
Kleine-Ko¨nig et al
METHODS A review of the literature showed that the usual review period following a conference lasts up to 5 years, as over 90% of all published abstracts appeared within 4 years of the conference.1,5,7,11,14 At the start of data collection in the fall of 2012, there was a follow-up period of at least 5 years for presentations at conferences dating from 2006 to 2008. The abstracts from the conferences were obtained from the EPOS abstract volumes, which are also available online for 2007 and 2008.15–17 A sample size of 646 conference abstracts resulted [2006: n = 152; 2007: n = 317 (joint meeting of EPOS and IFPOS); 2008: n = 177], and a search was carried out in the PubMed database for correspondences between these abstracts and publications in peer-reviewed journals using the authors’ names and the key words from the abstracts. Publications were also taken into account up to 5 years before the deadline for submission of abstracts, which is always in mid October of the year preceding each conference. An article was regarded as published if it had at least one identical author and its content/methods corresponded to those in the conference abstract. If there were differences (eg, different sample size), the study was regarded as published if the core of the research (the question investigated) were identical. The publication date used was the month of publication in the journal in which the article was printed. The period between the conference and the publication was measured in full months. For each publication, the name of the journal and its impact factor for the year in which the article concerned was published were recorded. The impact factor is calculated by the JCR by Thompson Reuters.18 As impact factors for 2012 were not yet available, the 2011 values were recorded instead. The study type was recorded for each abstract, with distinctions being made between clinical, experimental, and epidemiological studies, as well as meta-analyses, reviews, case reports, and “unclassifiable.” The clinical studies were further subdivided into therapeutic, prognostic, diagnostic, and economic research. If levels of evidence (LoE) were stated by the authors themselves, they were determined using the Oxford/CEBM criteria, as in all of the other cases as well.19,20 In addition, the abstracts were classified according to topics describing the main emphasis of the study. The following 12 subject categories were used: Hip Lower limb Foot Upper limb Spine Trauma Infections Tumor Genetics Neurological topics Radiological topics Other topics not otherwise classifiable
e34 | www.pedorthopaedics.com
Volume 34, Number 6, September 2014
If several topics were applicable, we regarded the main emphasis of the study as decisive. Exceptions to this were “neurological” topics and “tumors.” Whenever the study was concerned with one of these 2 topics, it was classed under “neurological topic” or “tumor,” independently of whether the hip or arm was affected, for example. This approach was used because various regions are often affected particularly in these 2 areas, although the underlying disease is the same. In addition, diseases with genetic causes, but not congenital fractures or malpositioning, were assigned to the topic “genetics.” The present study also investigated whether one of the following parameters had any influence on publication in a peer-reviewed journal: Randomized versus nonrandomized studies Prospective or retrospective study design Single-center versus multicenter study Significant main result versus nonsignificant main result In case the abstracts and articles lacked sufficient detail on research design, the study was coded as “unclassifiable.” In case of “comparative” studies without an evident prospective or retrospective design, we assumed a retrospective study and classified it as LoE III. Similarly, with regard to randomization, we classified studies lacking further details as nonrandomized. A result was regarded as significant if significance was stated at P < 0.05, or it was explicitly described as “significant.” P-values reported as >0.05 or the use of the words “nonsignificant” or “not significant” were evaluated as indicating nonsignificance; if no details were given in the text, it was classified as “significance not stated.” Statistical analysis was performed using the IBMSPSS-Statistics, version 20.0. Descriptive analyses were carried out, and the w2 test was used to compare the variables. The Kaplan-Meier method was used to estimate the PR over time (Fig. 1). Differences were regarded as significant at P < 0.05.
RESULTS The PR for EPOS conference abstracts from 2006 to 2008 averaged 36.7% (237 of the 646 abstracts were published). The most important criterion favoring publication was found to be the type of presentation. The PR for lectures was significantly higher (P < 0.001) than that of posters, with almost half of the lectures being published but only 1 in 3 of the posters. Publication took place in 61 different journals, the 7 most frequent of which are listed in Table 1. An impact factor was identified for 187 of the published articles, showing a value of 1.73 ± 1.00. None of the conference abstracts was found to have received multiple publications. Forty conference abstracts (16.88%) had already been published before the relevant conference, (median, 5.00 mo before). A further 6 publications that appeared >5 years before the deadline were excluded. The period from the conference to publication for all the r
2014 Lippincott Williams & Wilkins
J Pediatr Orthop
Volume 34, Number 6, September 2014
Publication Rate of EPOS Annual Meetings
FIGURE 1. The Kaplan-Meier plot of publication rates over time for abstracts presented at European Paediatric Orthopaedic Society annual meetings from 2006 to 2008.
237 articles averaged 13.88 ± 1.34 months. The period to publication was significantly influenced by the authorship group and the sample size. The subsequent discussion does not take into account publications that appeared before the corresponding conference (n = 40). An alteration in the authorship group was regarded as occurring if a new author was added before publication of the article. If there were no changes in the authorship group or some authors were omitted, the authorship group was regarded as being “constant.” In 59.4%, the authorship group changed and this led to a prolongation of 6 months in the period to publication (P < 0.001). The sample size was classified as “constant” if its size had not changed by more than ± 2 subjects. Changes of that type were observed in 34 publications, leading on average to an increase of 2 months in the time to publication (P < 0.197). Table 2 demonstrates that every second experimental studies are being published; however, only one in 3 clinical studies are being published. The Fisher-exact
test showed significant differences (P < 0.038) in PR between the various types of clinical study: diagnostic and prognostic studies had a higher PR than therapeutic. Economic studies were also published more frequently than therapeutic studies; however, this is not representative, as there were only 2 economic studies. Higher-quality studies (high LoE) were published more often (P < 0.001) than those of lower quality (Table 2). Abstracts with LoE I were published 2.3 times more often than those with LoE IV. Twenty-three of the 646 abstracts were randomized and with a PR of 82.6%, these were published more frequently (P < 0.001) than nonrandomized studies. Almost one in 2 prospective studies were published (PR, 47.8%), but only 1 in 3 retrospective studies (33.6%; P < 0.002) were published. A significant main result was included in 186 of the conference abstracts, whereas 62 abstracts did not state a significant main result and the remaining 398 gave no details regarding significance. Despite the large number of
TABLE 1. List of Journals Most Frequently Selected for Published European Paediatric Orthopaedic Society Abstracts, 2006 to 2008 Publications Journal Title Journal of Pediatric Orthopaedics Journal of Children’s Orthopaedics Journal of Pediatric Orthopaedics. Part B Spine Clinical Orthopaedics and Related Research The Journal of Bone and Joint Surgery. American Volume The Journal of Bone and Joint Surgery. British Volume
n (%) 50 23 19 19 17 16 12
(21.1) (9.7) (8.0) (8.0) (7.2) (6.8) (5.1)
Journal’s Average Impact Factor at Publication
JCR 2011 (For Comparison)
1.21 NA 0.59 2.55 2.16 2.99 1.98
1.156 0.467 2.078 2.533 3.272 2.832
JCR indicates Journal Citation Reports.21
r
2014 Lippincott Williams & Wilkins
www.pedorthopaedics.com |
e35
J Pediatr Orthop
Kleine-Ko¨nig et al
Volume 34, Number 6, September 2014
TABLE 2. Parameters Investigated in Abstracts Presented at the European Paediatric Orthopaedic Society Annual Meetings From 2006 to 2008
Presentation Poster Podium Type of study Clinical study Experimental study Epidemiological study Meta-analysis Review Case report Other Clinical studies Therapeutic study Diagnostic study Prognostic study Economic study Level of evidence in clinical studies Level I Level II Level III Level IV Level V Randomized study Prospective study Retrospective study Study with significant main result Study with nonsignificant main result Significance not specified Multicenter study Topic Hip Lower limb Foot Upper limb Spine Trauma Tumor Infection Neurological Genetic Radiologic Classification not possible
Abstracts
Publications
n (%)
n (%)
Publication Rate (%)
378 (58.5) 268 (41.5)
115 (48.5) 122 (51.5)
30.4 45.5
522 53 33 3 5 23 7
(80.8) (8.2) (5.1) (0.5) (0.8) (3.6) (1.1)
188 27 13 2 0 5 2
(79.3) (11.4) (5.5) (0.8) (0) (2.1) (0.8)
36.0 50.9 39.4 66.7 0 21.7 28.6
400 74 34 2
(61.9) (11.5) (5.3) (0.3)
128 32 17 1
(54.0) (13.5) (7.2) (0.4)
32.0 43.2 50.0 50.0
22 47 98 344 24 23 136 510 186 62 398 60
(3.4) (7.3) (15.2) (53.3) (3.7) (3.6) (21.1) (78.9) (28.8) (9.6) (61.6) (9.3)
15 25 46 102 6 19 65 171 107 24 106 33
(6.3) (10.5) (19.4) (43.0) (2.5) (8.0) (27.5) (72.5) (45.1) (10.1) (44.7) (13.9)
68.2 53.2 46.9 29.7 25.0 82.6 47.8 33.6 57.5 38.7 26.6 55.0
136 100 64 31 52 55 39 23 84 44 4 14
(21.1) (15.5) (9.9) (4.8) (8.0) (8.5) (6.0) (3.6) (13.0) (6.8) (0.6) (2.2)
49 28 18 12 29 21 12 6 40 13 2 7
(20.7) (11.8) (7.6) (5.1) (12.2) (8.9) (5.1) (2.5) (16.9) (5.5) (0.8) (3.0)
36.0 28.0 28.1 38.7 55.8 38.2 30.8 26.1 47.6 29.5 50.0 50.0
abstracts without details (61.6%), it was found that conference abstracts that had a significant main result were published more frequently (P < 0.01) than those without a significant main result (Table 2). Fifty-five percent of the multicenter studies were published but only 34.8% of the single-center studies were published.
DISCUSSION The EPOS PR was 36.7%, precisely in the middle range of results for other orthopaedics conferences, for which PR of 23.4% to 50% have been reported.1,2,4–13 The EPOS PR is below that of POSNA, its North American equivalent, for which an average PR of 50% has been reported for its 2002 to 2006 conferences.6 Possible explanations include the number and quality of abstracts
e36 | www.pedorthopaedics.com
received as well as methodological differences. As POSNA far outnumbers EPOS in terms of enlisted members (1100 to 300), it is reasonable to assume that POSNA receives more abstracts than EPOS. At the same time, POSNA selects, on average, only 80 abstracts for presentation at its annual conference compared with EPOS with 160 abstracts. Although POSNA does not list its publication criteria on the official website, these statistics suggest a stricter review process at POSNA than at EPOS. Meanwhile, the gulf in PR between POSNA and EPOS may also result from methodological differences. Although Amirhamzeh and colleagues used PubMed and Google Scholar as the basis of their review, this study focused on PubMed. The period to publication was on average 13.88 months and therefore shorter than in all other comparable studies, which have reported periods of 15.04 to 29.00 r
2014 Lippincott Williams & Wilkins
J Pediatr Orthop
Volume 34, Number 6, September 2014
months (Table 3).1,2,4–13 Table 3 shows that the majority of European conferences have a shorter time to publication than their American counterparts. Although some of these studies included—in their calculation of the time to publication—articles that appeared before the conference, the EPOS nonetheless recorded a shorter time to publication.5 In contrast, Amirhamzeh and colleagues, in their study of POSNA publications, excluded abstracts published before the conference presentation. This may explain possible differences in the time to publication between EPOS and POSNA.6 The higher publication rate of lectures vis-a`-vis posters may have resulted from the quality of the preconference Reading Committee. This Committee selects abstracts for oral presentations in accordance with the following criteria: Is the topic exciting? Are the findings capable of changing everyday practice? Does the study design have scientific merit?22 Even though the Reading Committee requires that only unpublished studies be submitted for review, a total of 46 abstracts accepted for presentation have been published beforehand.22 It is unclear whether the Reading Committee considered these abstracts as sufficiently “exciting” to overrule the nonpublication requirement or if it simply was not aware of their publication at that time. In any case, the additional review by the Reading Committee ensured a selection of high-quality papers with a greater likelihood of being published in the future. The Journal of Pediatric Orthopaedics published by far the largest number of articles (21.1%). This may be
Publication Rate of EPOS Annual Meetings
explained by the fact that the journal is affiliated with the POSNA and the EPOS, but not an official publication.23 The POSNA has no official publication and the official publication of the EPOS is the Journal of Children’s Orthopaedics. The mean impact factor for all of the published abstracts, at 1.73, lies in the average range for impact factors for all orthopaedic journals in 2011 (0.017 to 3.904) that were evaluated by JCR.18 POSNA states that its articles have an impact factor of 1.87, which is very close to that of EPOS. The authorship group changed in 59.4% of the published studies, although the first author only changed in 16.2%. Such change in authorship may stem from additional trainees involved in the writing, submission, and revision of the article. In comparison with the study by Amirhamzeh et al6 (noted a new main author in 53.2%), the authorship changes observed in the present study are relatively minor and are, therefore, a further possible explanation for the shorter time to publication. The analysis of the study types underlying the published abstracts showed that clinical and experimental studies were the most frequent. The gap in PR between prognostic and diagnostic studies ensues, most likely, from the LoE. 71.88% of the published diagnostic studies, 29.41% of the published prognostic studies, but only 0.06% of the therapeutic studies have an LoE of ½. Put differently, a higher LoE led to a significantly higher PR, as is confirmed in other studies.1,5,9 The study by Obremskey et al21 provides further support for this trend, showing that orthopaedics journals with high impact values preferentially publish articles with LoE I or II.
TABLE 3. Publication Rates for Presentations at Annual Meetings of Orthopaedic and Other Surgical Societies
References Abicht et
Amirhamzeh et al6 Donegan et al7 Guryel et al8
Macdonald et al4 Scherer et al9 Schulte et al5 Schulte et al1 al10
Voleti et al11
r
Mean Time to Publication (mo)
American College of Foot and Ankle Surgeons Pediatric Orthopaedic Society of North America American Academy of Orthopedic Surgeons British Association for Surgery of the Knee (BASK), British Orthopaedic Foot Surgical Society (BOFSS), British Elbow and Shoulder Society (BESS) British Association of Paediatric Surgeons Cochrane review
—
17.6
4
18.8-29
5
—
—
15.6
—
—
2
18.4
German Society of Orthopaedics and Trauma Surgery Spine Society of Europe
5
15
5
17.7
4.25
19
5
17.7
Conference
al2
Varghese et
Follow-up Period (y)
Pediatric Orthopaedic Society of North America American Academy of Orthopaedic Surgeons
2014 Lippincott Williams & Wilkins
Publication Conference Rate During Abstracts Period Follow-up (%) Included (n) 19992008 20022006 2001
Podium/ Poster Presentation
24.0
825
Poster
50.7
762
Podium+poster
49.0
756
Podium+poster
19971998
33.0
415
not applicable
19992008 79 reports 2003
35.0
862
44.5
29729
Podium+poster
36.0
1100
Podium+poster
37.8
839
Podium+poster
50.0
1191
Podium+poster
49.0
756
Podium+poster
20002003 20032005 2001
Podium
www.pedorthopaedics.com |
e37
J Pediatr Orthop
Kleine-Ko¨nig et al
The PR for randomized studies and for prospective studies clearly shows that randomization and a prospective study design strongly favor publication. It is evident that abstracts on the topic of the hip are the most frequent, as described previously in the literature.6–8 However, the PR for these abstracts is comparatively low in relation to the others (Table 2). The highest PR was observed for abstracts on the topic of the spine, most likely because of their higher quality, as suggested by their LoE. 20.00% of the “spine-articles” reach an LoE 2 compared with only 12.50% of the “hip-articles,” whereas only 25.00% of the “spine-articles” are LoE4 compared with 57.50% of the “hip-articles.” Sixty-three percent of the conference abstracts remained unpublished. Inadequate details provided by the authors regarding their study design make it difficult to assess the abstracts and articles for the reading committee and the present authors and represent a weakness in the design of this study. In summary, however, it may be noted that in comparison with the PR of other specialist orthopaedic societies and their annual conferences (Table 3), EPOS is, well, in the midrange and hardly differs even in direct comparison with POSNA with regard to the impact factors achieved with the publications and selected journals. The abstracts that are selected and presented at the EPOS meetings stand out from all of the other conferences studied through their short period to publication, thereby ensuring rapid dissemination of current medical findings through the choice of publication as an effective means of spreading specialist information internationally. The study type and grade of evidence are thus quality indicators, and conference participants should be informed about these to enable them to distinguish between less important and more important results for their own clinical practice. REFERENCES 1. Schulte TL, Huck K, Osada N, et al. Publication rate of abstracts presented at the Annual Congress of the Spine Society of Europe (years 2000–2003). Eur Spine J. 2012;21:2105–2112. 2. Abicht BP, Donnenwerth MP, Borkosky SL, et al. Publication rates of poster presentations at the American College of Foot and Ankle Surgeons annual scientific conference between 1999 and 2008. J Foot Ankle Surg. 2012;51:45–49. 3. Bhandari M, Devereaux PJ, Guyatt GH, et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84-A:615–621. 4. Macdonald AL, Parsons C, Davenport M. Outcome of abstracts presented at the British Association of Paediatric Surgeons congresses (1999–2008). J Pediatr Surg. 2012;47:386–390.
e38 | www.pedorthopaedics.com
Volume 34, Number 6, September 2014
5. Schulte TL, Trost M, Osada N, et al. Publication rate of abstracts presented at the Annual Congress of the German Society of Orthopaedics and Trauma Surgery. Arch Orthop Trauma Surg. 2012;132:271–280. 6. Amirhamzeh D, Morr MA, Baldwin K, et al. Publication rates of abstracts presented at Pediatric Orthopaedic Society of North America meetings between 2002 and 2006. J Pediatr Orthop. 2012;32:e6–e10. 7. Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468:1428–1435. 8. Guryel E, Durrant AW, Alakeson R, et al. From presentation to publication: the natural history of orthopaedic abstracts in the United Kingdom. Postgrad Med J. 2006;82:70–72. 9. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;2:MR000005. 10. Varghese RA, Chang J, Miyanji F, et al. Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America: is there a difference between accepted versus rejected abstracts? J Pediatr Orthop. 2011;31: 334–340. 11. Voleti PB, Donegan DJ, Kim TW, et al. Level of evidence: does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? J Bone Joint Surg Am. 2013;95:e2. 12. Wang JC, Yoo S, Delamarter RB. The publication rates of presentations at major Spine Specialty Society meetings (NASS, SRS, ISSLS). Spine (Phila Pa 1976). 1999;24:425–427. 13. Whitehouse MR, Atwal NS, Blom AW. Publication rates for hip surgery-related abstracts presented at national and international meetings. Orthopedics. 2009;32:407. 14. Daluiski A, Kuhns CA, Jackson KR, et al. Publication rate of abstracts presented at the annual meeting of the Orthopaedic Research Society. J Orthop Res. 1998;16:645–649. 15. Engel C. European Paediatric Orthopaedic Society (EPOS). [Twenty-fifth Annual Meeting of the European Paediatric Orthopaedic Society (EPOS)]. Orthopade. 2006;35:800–804. German. 16. EPOS. EPOS 27th Annual Meeting: 9–12 April 2008, Warsaw, Poland. J Child Orthop. 2008;2(Suppl 1):1–76. 17. EPOS. EPOS/IFPOS Combined Meeting EPOS—26th Annual Meeting: Sorrento, 11–14 April 2007. J Child Orthop. 2007;1(suppl 1): 13–132. 18. Thompson Reuters. JCR easy knowledge. Available at: http:// webofknowledge.com/JCR/. Accessed March 14, 2013. 19. Slobogean G, Bhandari M. Introducing levels of evidence to the Journal of Orthopaedic Trauma: implementation and future directions. J Orthop Trauma. 2012;26:127–128. 20. Wright JG, Swiontkowski MF, Heckman JD. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85-A:1–3. 21. Obremskey WT, Pappas N, Attallah-Wasif E, et al. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005; 87:2632–2638. 22. EPOS. Reading Committee. Available at: http://www.epos.org/ commitees/reading-committe. Accessed June 26, 2013. 23. Journal of Pediatric Orthopaedics. About the journal. Available at: http://journals.lww.com/pedorthopaedics/pages/aboutthejournal.aspx. Accessed March 11, 2013.
r
2014 Lippincott Williams & Wilkins
Publication Rate of Abstracts Presented at European Paediatric Orthopaedic Society Annual Meetings, 2006 to 2008 Marie-Theres Kleine-Konig,* Tobias L. Schulte, MD,w Georg Gosheger, MD,w Robert Ro¨dl, MD,* and Frank M. Schiedel, MD*
Background: The quality of the abstracts presented at a conference reflects the scientific work and level of activity of the scientific association concerned. The aim of the present study was to determine the rate of publications of podium presentations and posters at the conferences of the European Paediatric Orthopaedic Society (EPOS) from 2006 to 2008 and to identify factors that favor publication in peer-reviewed journals. The results are compared with those of other international societies. Methods: All 646 abstracts (including podiums, posters, and e-posters) presented at the EPOS conferences were investigated using the PubMed database to identify any corresponding published articles in the journals listed in the database. A period of 5 years before and after the relevant conference was used for the PubMed search. Factors influencing publication and the quality of the study, such as the type of presentation and the level of evidence, were also investigated. Results: A publication rate of 36.7% was observed, corresponding to 237 publications of 646 abstracts. The period to publication showed a mean of 13.88 ± 1.34 months. It was found that abstracts of podium presentations were published significantly more often than poster abstracts (P < 0.001). Experimental studies, with a publication rate of 50.9%, showed better results than clinical studies (36.0%). Overall, the articles were published in 61 different journals, with the largest number (n = 50) appearing in the Journal of Pediatric Orthopaedics. In addition, the present study shows that abstracts with a higher level of evidence were associated with a higher publication rate. Conclusions: At 36.7%, the rate of publication of EPOS abstracts is within the range reached by other specialist orthopaedics societies, such as the German Society of Orthopaedics and Trauma Surgery (36%) and the British Orthopaedic Association (36%). However, it is lower than the publication rate of the Pediatric Orthopaedic Society of North America (POSNA), at 50%. The high percentage of unpublished conference From the *Department of Pediatric Orthopaedics, Deformity Correction and Foot Surgery; and wDepartment of Orthopaedics and Tumor Orthopaedics, Mu¨nster University Hospital, Mu¨nster, Germany. The authors declare no conflicts of interest. Reprints: Frank M. Schiedel, MD, Department of Pediatric Orthopaedics, Deformity Correction, and Foot Surgery, Mu¨nster University Hospital, Albert-Schweitzer-Campus 1, D-48149 Mu¨nster, Germany. E-mail: [email protected]. Copyright r 2014 by Lippincott Williams & Wilkins
J Pediatr Orthop
Volume 34, Number 6, September 2014
abstracts (63%), which did not go through a peer-reviewed process, casts doubts upon the practice of utilizing the citation of abstracts based purely on conference abstracts. Level of Evidence: Statistical study. Key Words: impact, proceedings, EPOS, publication rate, annual meeting, abstract, oral, presentation, poster presentation, level of evidence (J Pediatr Orthop 2014;34:e33–e38)
I
n the field of pediatric orthopaedics, as in every other medical specialty, it is extremely important to present one’s scientific research to the public. Opportunities to do this are provided by conferences and by scientific journals in which original research studies are published. Presentations at conferences are kept quite short and are concerned only with the most important data. This is intended to ensure a rapid chain of publication, from data collection to presentation. Publication in a scientific journal is regarded as the “gold standard” for the dissemination of scientific research,1 as this type of article provides a full account of the research study and is subject to peer review, so that the quality of the research can be regarded as ensured. Lectures and posters presented at conferences, nevertheless, have a considerable influence on scientific research and everyday work, and they are often cited in articles so that conference lectures and posters represent an important component of medical training.1–4 Ensuring the quality of papers presented at conferences should therefore be an important concern for medical societies. Publication in a high-quality peer-reviewed journal is a generally recognized sign of the quality of a study.5 The aim of the present study was to investigate the quality of presentations at the annual European Paediatric Orthopaedic Society (EPOS) conference by determining the rate of publication of lectures and posters presented from 2006 to 2008. In addition, the conference presentations and published articles were investigated to identify any criteria that might have favored publication. This method was intended to allow comparison of the results of the study with publication rates (PR) and results from other conferences on similar topics.1,2,4–13 www.pedorthopaedics.com |
e33
J Pediatr Orthop
Kleine-Ko¨nig et al
METHODS A review of the literature showed that the usual review period following a conference lasts up to 5 years, as over 90% of all published abstracts appeared within 4 years of the conference.1,5,7,11,14 At the start of data collection in the fall of 2012, there was a follow-up period of at least 5 years for presentations at conferences dating from 2006 to 2008. The abstracts from the conferences were obtained from the EPOS abstract volumes, which are also available online for 2007 and 2008.15–17 A sample size of 646 conference abstracts resulted [2006: n = 152; 2007: n = 317 (joint meeting of EPOS and IFPOS); 2008: n = 177], and a search was carried out in the PubMed database for correspondences between these abstracts and publications in peer-reviewed journals using the authors’ names and the key words from the abstracts. Publications were also taken into account up to 5 years before the deadline for submission of abstracts, which is always in mid October of the year preceding each conference. An article was regarded as published if it had at least one identical author and its content/methods corresponded to those in the conference abstract. If there were differences (eg, different sample size), the study was regarded as published if the core of the research (the question investigated) were identical. The publication date used was the month of publication in the journal in which the article was printed. The period between the conference and the publication was measured in full months. For each publication, the name of the journal and its impact factor for the year in which the article concerned was published were recorded. The impact factor is calculated by the JCR by Thompson Reuters.18 As impact factors for 2012 were not yet available, the 2011 values were recorded instead. The study type was recorded for each abstract, with distinctions being made between clinical, experimental, and epidemiological studies, as well as meta-analyses, reviews, case reports, and “unclassifiable.” The clinical studies were further subdivided into therapeutic, prognostic, diagnostic, and economic research. If levels of evidence (LoE) were stated by the authors themselves, they were determined using the Oxford/CEBM criteria, as in all of the other cases as well.19,20 In addition, the abstracts were classified according to topics describing the main emphasis of the study. The following 12 subject categories were used: Hip Lower limb Foot Upper limb Spine Trauma Infections Tumor Genetics Neurological topics Radiological topics Other topics not otherwise classifiable
e34 | www.pedorthopaedics.com
Volume 34, Number 6, September 2014
If several topics were applicable, we regarded the main emphasis of the study as decisive. Exceptions to this were “neurological” topics and “tumors.” Whenever the study was concerned with one of these 2 topics, it was classed under “neurological topic” or “tumor,” independently of whether the hip or arm was affected, for example. This approach was used because various regions are often affected particularly in these 2 areas, although the underlying disease is the same. In addition, diseases with genetic causes, but not congenital fractures or malpositioning, were assigned to the topic “genetics.” The present study also investigated whether one of the following parameters had any influence on publication in a peer-reviewed journal: Randomized versus nonrandomized studies Prospective or retrospective study design Single-center versus multicenter study Significant main result versus nonsignificant main result In case the abstracts and articles lacked sufficient detail on research design, the study was coded as “unclassifiable.” In case of “comparative” studies without an evident prospective or retrospective design, we assumed a retrospective study and classified it as LoE III. Similarly, with regard to randomization, we classified studies lacking further details as nonrandomized. A result was regarded as significant if significance was stated at P < 0.05, or it was explicitly described as “significant.” P-values reported as >0.05 or the use of the words “nonsignificant” or “not significant” were evaluated as indicating nonsignificance; if no details were given in the text, it was classified as “significance not stated.” Statistical analysis was performed using the IBMSPSS-Statistics, version 20.0. Descriptive analyses were carried out, and the w2 test was used to compare the variables. The Kaplan-Meier method was used to estimate the PR over time (Fig. 1). Differences were regarded as significant at P < 0.05.
RESULTS The PR for EPOS conference abstracts from 2006 to 2008 averaged 36.7% (237 of the 646 abstracts were published). The most important criterion favoring publication was found to be the type of presentation. The PR for lectures was significantly higher (P < 0.001) than that of posters, with almost half of the lectures being published but only 1 in 3 of the posters. Publication took place in 61 different journals, the 7 most frequent of which are listed in Table 1. An impact factor was identified for 187 of the published articles, showing a value of 1.73 ± 1.00. None of the conference abstracts was found to have received multiple publications. Forty conference abstracts (16.88%) had already been published before the relevant conference, (median, 5.00 mo before). A further 6 publications that appeared >5 years before the deadline were excluded. The period from the conference to publication for all the r
2014 Lippincott Williams & Wilkins
J Pediatr Orthop
Volume 34, Number 6, September 2014
Publication Rate of EPOS Annual Meetings
FIGURE 1. The Kaplan-Meier plot of publication rates over time for abstracts presented at European Paediatric Orthopaedic Society annual meetings from 2006 to 2008.
237 articles averaged 13.88 ± 1.34 months. The period to publication was significantly influenced by the authorship group and the sample size. The subsequent discussion does not take into account publications that appeared before the corresponding conference (n = 40). An alteration in the authorship group was regarded as occurring if a new author was added before publication of the article. If there were no changes in the authorship group or some authors were omitted, the authorship group was regarded as being “constant.” In 59.4%, the authorship group changed and this led to a prolongation of 6 months in the period to publication (P < 0.001). The sample size was classified as “constant” if its size had not changed by more than ± 2 subjects. Changes of that type were observed in 34 publications, leading on average to an increase of 2 months in the time to publication (P < 0.197). Table 2 demonstrates that every second experimental studies are being published; however, only one in 3 clinical studies are being published. The Fisher-exact
test showed significant differences (P < 0.038) in PR between the various types of clinical study: diagnostic and prognostic studies had a higher PR than therapeutic. Economic studies were also published more frequently than therapeutic studies; however, this is not representative, as there were only 2 economic studies. Higher-quality studies (high LoE) were published more often (P < 0.001) than those of lower quality (Table 2). Abstracts with LoE I were published 2.3 times more often than those with LoE IV. Twenty-three of the 646 abstracts were randomized and with a PR of 82.6%, these were published more frequently (P < 0.001) than nonrandomized studies. Almost one in 2 prospective studies were published (PR, 47.8%), but only 1 in 3 retrospective studies (33.6%; P < 0.002) were published. A significant main result was included in 186 of the conference abstracts, whereas 62 abstracts did not state a significant main result and the remaining 398 gave no details regarding significance. Despite the large number of
TABLE 1. List of Journals Most Frequently Selected for Published European Paediatric Orthopaedic Society Abstracts, 2006 to 2008 Publications Journal Title Journal of Pediatric Orthopaedics Journal of Children’s Orthopaedics Journal of Pediatric Orthopaedics. Part B Spine Clinical Orthopaedics and Related Research The Journal of Bone and Joint Surgery. American Volume The Journal of Bone and Joint Surgery. British Volume
n (%) 50 23 19 19 17 16 12
(21.1) (9.7) (8.0) (8.0) (7.2) (6.8) (5.1)
Journal’s Average Impact Factor at Publication
JCR 2011 (For Comparison)
1.21 NA 0.59 2.55 2.16 2.99 1.98
1.156 0.467 2.078 2.533 3.272 2.832
JCR indicates Journal Citation Reports.21
r
2014 Lippincott Williams & Wilkins
www.pedorthopaedics.com |
e35
J Pediatr Orthop
Kleine-Ko¨nig et al
Volume 34, Number 6, September 2014
TABLE 2. Parameters Investigated in Abstracts Presented at the European Paediatric Orthopaedic Society Annual Meetings From 2006 to 2008
Presentation Poster Podium Type of study Clinical study Experimental study Epidemiological study Meta-analysis Review Case report Other Clinical studies Therapeutic study Diagnostic study Prognostic study Economic study Level of evidence in clinical studies Level I Level II Level III Level IV Level V Randomized study Prospective study Retrospective study Study with significant main result Study with nonsignificant main result Significance not specified Multicenter study Topic Hip Lower limb Foot Upper limb Spine Trauma Tumor Infection Neurological Genetic Radiologic Classification not possible
Abstracts
Publications
n (%)
n (%)
Publication Rate (%)
378 (58.5) 268 (41.5)
115 (48.5) 122 (51.5)
30.4 45.5
522 53 33 3 5 23 7
(80.8) (8.2) (5.1) (0.5) (0.8) (3.6) (1.1)
188 27 13 2 0 5 2
(79.3) (11.4) (5.5) (0.8) (0) (2.1) (0.8)
36.0 50.9 39.4 66.7 0 21.7 28.6
400 74 34 2
(61.9) (11.5) (5.3) (0.3)
128 32 17 1
(54.0) (13.5) (7.2) (0.4)
32.0 43.2 50.0 50.0
22 47 98 344 24 23 136 510 186 62 398 60
(3.4) (7.3) (15.2) (53.3) (3.7) (3.6) (21.1) (78.9) (28.8) (9.6) (61.6) (9.3)
15 25 46 102 6 19 65 171 107 24 106 33
(6.3) (10.5) (19.4) (43.0) (2.5) (8.0) (27.5) (72.5) (45.1) (10.1) (44.7) (13.9)
68.2 53.2 46.9 29.7 25.0 82.6 47.8 33.6 57.5 38.7 26.6 55.0
136 100 64 31 52 55 39 23 84 44 4 14
(21.1) (15.5) (9.9) (4.8) (8.0) (8.5) (6.0) (3.6) (13.0) (6.8) (0.6) (2.2)
49 28 18 12 29 21 12 6 40 13 2 7
(20.7) (11.8) (7.6) (5.1) (12.2) (8.9) (5.1) (2.5) (16.9) (5.5) (0.8) (3.0)
36.0 28.0 28.1 38.7 55.8 38.2 30.8 26.1 47.6 29.5 50.0 50.0
abstracts without details (61.6%), it was found that conference abstracts that had a significant main result were published more frequently (P < 0.01) than those without a significant main result (Table 2). Fifty-five percent of the multicenter studies were published but only 34.8% of the single-center studies were published.
DISCUSSION The EPOS PR was 36.7%, precisely in the middle range of results for other orthopaedics conferences, for which PR of 23.4% to 50% have been reported.1,2,4–13 The EPOS PR is below that of POSNA, its North American equivalent, for which an average PR of 50% has been reported for its 2002 to 2006 conferences.6 Possible explanations include the number and quality of abstracts
e36 | www.pedorthopaedics.com
received as well as methodological differences. As POSNA far outnumbers EPOS in terms of enlisted members (1100 to 300), it is reasonable to assume that POSNA receives more abstracts than EPOS. At the same time, POSNA selects, on average, only 80 abstracts for presentation at its annual conference compared with EPOS with 160 abstracts. Although POSNA does not list its publication criteria on the official website, these statistics suggest a stricter review process at POSNA than at EPOS. Meanwhile, the gulf in PR between POSNA and EPOS may also result from methodological differences. Although Amirhamzeh and colleagues used PubMed and Google Scholar as the basis of their review, this study focused on PubMed. The period to publication was on average 13.88 months and therefore shorter than in all other comparable studies, which have reported periods of 15.04 to 29.00 r
2014 Lippincott Williams & Wilkins
J Pediatr Orthop
Volume 34, Number 6, September 2014
months (Table 3).1,2,4–13 Table 3 shows that the majority of European conferences have a shorter time to publication than their American counterparts. Although some of these studies included—in their calculation of the time to publication—articles that appeared before the conference, the EPOS nonetheless recorded a shorter time to publication.5 In contrast, Amirhamzeh and colleagues, in their study of POSNA publications, excluded abstracts published before the conference presentation. This may explain possible differences in the time to publication between EPOS and POSNA.6 The higher publication rate of lectures vis-a`-vis posters may have resulted from the quality of the preconference Reading Committee. This Committee selects abstracts for oral presentations in accordance with the following criteria: Is the topic exciting? Are the findings capable of changing everyday practice? Does the study design have scientific merit?22 Even though the Reading Committee requires that only unpublished studies be submitted for review, a total of 46 abstracts accepted for presentation have been published beforehand.22 It is unclear whether the Reading Committee considered these abstracts as sufficiently “exciting” to overrule the nonpublication requirement or if it simply was not aware of their publication at that time. In any case, the additional review by the Reading Committee ensured a selection of high-quality papers with a greater likelihood of being published in the future. The Journal of Pediatric Orthopaedics published by far the largest number of articles (21.1%). This may be
Publication Rate of EPOS Annual Meetings
explained by the fact that the journal is affiliated with the POSNA and the EPOS, but not an official publication.23 The POSNA has no official publication and the official publication of the EPOS is the Journal of Children’s Orthopaedics. The mean impact factor for all of the published abstracts, at 1.73, lies in the average range for impact factors for all orthopaedic journals in 2011 (0.017 to 3.904) that were evaluated by JCR.18 POSNA states that its articles have an impact factor of 1.87, which is very close to that of EPOS. The authorship group changed in 59.4% of the published studies, although the first author only changed in 16.2%. Such change in authorship may stem from additional trainees involved in the writing, submission, and revision of the article. In comparison with the study by Amirhamzeh et al6 (noted a new main author in 53.2%), the authorship changes observed in the present study are relatively minor and are, therefore, a further possible explanation for the shorter time to publication. The analysis of the study types underlying the published abstracts showed that clinical and experimental studies were the most frequent. The gap in PR between prognostic and diagnostic studies ensues, most likely, from the LoE. 71.88% of the published diagnostic studies, 29.41% of the published prognostic studies, but only 0.06% of the therapeutic studies have an LoE of ½. Put differently, a higher LoE led to a significantly higher PR, as is confirmed in other studies.1,5,9 The study by Obremskey et al21 provides further support for this trend, showing that orthopaedics journals with high impact values preferentially publish articles with LoE I or II.
TABLE 3. Publication Rates for Presentations at Annual Meetings of Orthopaedic and Other Surgical Societies
References Abicht et
Amirhamzeh et al6 Donegan et al7 Guryel et al8
Macdonald et al4 Scherer et al9 Schulte et al5 Schulte et al1 al10
Voleti et al11
r
Mean Time to Publication (mo)
American College of Foot and Ankle Surgeons Pediatric Orthopaedic Society of North America American Academy of Orthopedic Surgeons British Association for Surgery of the Knee (BASK), British Orthopaedic Foot Surgical Society (BOFSS), British Elbow and Shoulder Society (BESS) British Association of Paediatric Surgeons Cochrane review
—
17.6
4
18.8-29
5
—
—
15.6
—
—
2
18.4
German Society of Orthopaedics and Trauma Surgery Spine Society of Europe
5
15
5
17.7
4.25
19
5
17.7
Conference
al2
Varghese et
Follow-up Period (y)
Pediatric Orthopaedic Society of North America American Academy of Orthopaedic Surgeons
2014 Lippincott Williams & Wilkins
Publication Conference Rate During Abstracts Period Follow-up (%) Included (n) 19992008 20022006 2001
Podium/ Poster Presentation
24.0
825
Poster
50.7
762
Podium+poster
49.0
756
Podium+poster
19971998
33.0
415
not applicable
19992008 79 reports 2003
35.0
862
44.5
29729
Podium+poster
36.0
1100
Podium+poster
37.8
839
Podium+poster
50.0
1191
Podium+poster
49.0
756
Podium+poster
20002003 20032005 2001
Podium
www.pedorthopaedics.com |
e37
J Pediatr Orthop
Kleine-Ko¨nig et al
The PR for randomized studies and for prospective studies clearly shows that randomization and a prospective study design strongly favor publication. It is evident that abstracts on the topic of the hip are the most frequent, as described previously in the literature.6–8 However, the PR for these abstracts is comparatively low in relation to the others (Table 2). The highest PR was observed for abstracts on the topic of the spine, most likely because of their higher quality, as suggested by their LoE. 20.00% of the “spine-articles” reach an LoE 2 compared with only 12.50% of the “hip-articles,” whereas only 25.00% of the “spine-articles” are LoE4 compared with 57.50% of the “hip-articles.” Sixty-three percent of the conference abstracts remained unpublished. Inadequate details provided by the authors regarding their study design make it difficult to assess the abstracts and articles for the reading committee and the present authors and represent a weakness in the design of this study. In summary, however, it may be noted that in comparison with the PR of other specialist orthopaedic societies and their annual conferences (Table 3), EPOS is, well, in the midrange and hardly differs even in direct comparison with POSNA with regard to the impact factors achieved with the publications and selected journals. The abstracts that are selected and presented at the EPOS meetings stand out from all of the other conferences studied through their short period to publication, thereby ensuring rapid dissemination of current medical findings through the choice of publication as an effective means of spreading specialist information internationally. The study type and grade of evidence are thus quality indicators, and conference participants should be informed about these to enable them to distinguish between less important and more important results for their own clinical practice. REFERENCES 1. Schulte TL, Huck K, Osada N, et al. Publication rate of abstracts presented at the Annual Congress of the Spine Society of Europe (years 2000–2003). Eur Spine J. 2012;21:2105–2112. 2. Abicht BP, Donnenwerth MP, Borkosky SL, et al. Publication rates of poster presentations at the American College of Foot and Ankle Surgeons annual scientific conference between 1999 and 2008. J Foot Ankle Surg. 2012;51:45–49. 3. Bhandari M, Devereaux PJ, Guyatt GH, et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84-A:615–621. 4. Macdonald AL, Parsons C, Davenport M. Outcome of abstracts presented at the British Association of Paediatric Surgeons congresses (1999–2008). J Pediatr Surg. 2012;47:386–390.
e38 | www.pedorthopaedics.com
Volume 34, Number 6, September 2014
5. Schulte TL, Trost M, Osada N, et al. Publication rate of abstracts presented at the Annual Congress of the German Society of Orthopaedics and Trauma Surgery. Arch Orthop Trauma Surg. 2012;132:271–280. 6. Amirhamzeh D, Morr MA, Baldwin K, et al. Publication rates of abstracts presented at Pediatric Orthopaedic Society of North America meetings between 2002 and 2006. J Pediatr Orthop. 2012;32:e6–e10. 7. Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468:1428–1435. 8. Guryel E, Durrant AW, Alakeson R, et al. From presentation to publication: the natural history of orthopaedic abstracts in the United Kingdom. Postgrad Med J. 2006;82:70–72. 9. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;2:MR000005. 10. Varghese RA, Chang J, Miyanji F, et al. Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America: is there a difference between accepted versus rejected abstracts? J Pediatr Orthop. 2011;31: 334–340. 11. Voleti PB, Donegan DJ, Kim TW, et al. Level of evidence: does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? J Bone Joint Surg Am. 2013;95:e2. 12. Wang JC, Yoo S, Delamarter RB. The publication rates of presentations at major Spine Specialty Society meetings (NASS, SRS, ISSLS). Spine (Phila Pa 1976). 1999;24:425–427. 13. Whitehouse MR, Atwal NS, Blom AW. Publication rates for hip surgery-related abstracts presented at national and international meetings. Orthopedics. 2009;32:407. 14. Daluiski A, Kuhns CA, Jackson KR, et al. Publication rate of abstracts presented at the annual meeting of the Orthopaedic Research Society. J Orthop Res. 1998;16:645–649. 15. Engel C. European Paediatric Orthopaedic Society (EPOS). [Twenty-fifth Annual Meeting of the European Paediatric Orthopaedic Society (EPOS)]. Orthopade. 2006;35:800–804. German. 16. EPOS. EPOS 27th Annual Meeting: 9–12 April 2008, Warsaw, Poland. J Child Orthop. 2008;2(Suppl 1):1–76. 17. EPOS. EPOS/IFPOS Combined Meeting EPOS—26th Annual Meeting: Sorrento, 11–14 April 2007. J Child Orthop. 2007;1(suppl 1): 13–132. 18. Thompson Reuters. JCR easy knowledge. Available at: http:// webofknowledge.com/JCR/. Accessed March 14, 2013. 19. Slobogean G, Bhandari M. Introducing levels of evidence to the Journal of Orthopaedic Trauma: implementation and future directions. J Orthop Trauma. 2012;26:127–128. 20. Wright JG, Swiontkowski MF, Heckman JD. Introducing levels of evidence to the journal. J Bone Joint Surg Am. 2003;85-A:1–3. 21. Obremskey WT, Pappas N, Attallah-Wasif E, et al. Level of evidence in orthopaedic journals. J Bone Joint Surg Am. 2005; 87:2632–2638. 22. EPOS. Reading Committee. Available at: http://www.epos.org/ commitees/reading-committe. Accessed June 26, 2013. 23. Journal of Pediatric Orthopaedics. About the journal. Available at: http://journals.lww.com/pedorthopaedics/pages/aboutthejournal.aspx. Accessed March 11, 2013.
r
2014 Lippincott Williams & Wilkins
