I. Periodomat Res. 13: 487-497, 1978
Citation records of papers published in the Journal of Periodontology and the Journal of Periodontal Research D. M. BRUNETTE, M . J. SIMON AND M . A. REIMERS
Medical Research Council Group in PeriodontaS Physiology, Faculty of Dentistry, University of Toronto, Toronto, Ontario, Canada The citation records of the papers published in the Journal of Periodontology vols. 31-45 and Journal of Periodontal Research vols. 1-9 have been examined and lists of most cited papers ranked by tolal number of citations and citations per year compiled. Several diverse approaches to periodontal research are represented in these lists including microbiology, histology, immunology, and surgery. The pattern of periodontal literature usage with time resembles that found for biomedical literature and other bibliographic indices such as Price's Index and the number of references per paper place periodontal research in or close to the range of values found for "normal" sciences. The distribution of citations among individual papers resembles that found in other sciences in that the majority of papers have very little impact. In the first five years following publication, 18 % of papers published in J. Periodontol. were not cited while less than 10 % were cited ten times or more. The record for papers published in /. Periodontal Res. is slightly better, 9.5 % were not cited and 12.6 % were cited ten times or more in the five years following publication. In this study, there were no instances of papers that failed to be cited in the first five years achieving ten or more citations in any subsequent five year period. One class of research publication, case reports, was associated with a very low citation frequency. It appears that the influence of case reports on subsequent research is small. (Accepted for publication April 26, 1978)
Introduction It is common knowledge that the amount of scientific research literature is increasing sttidily. The annual growth rate has been eaf mated to be 7 % of the extant literature wl-ch is now believed to be close to 7.5
million items (Garfield 1976a). However, ^^^ effectiveness of much of this research has been questioned both in the popular press and in scientific forums. One approach to assessing the effectiveness of scientific research is citation analysis (CA). The fundamental question that can be
B R U N E T T E ,
S I M O N
answered by CA, which uses as its primary source the Science Citation Index (SCI)., is where and by whom a particular piece of scientific work has been cited in the literature. In CA studies the first step is to count the number of times an article has been cited, In general, it is assumed that papers that are never cited are relatively unimportant whereas those that are highly cited are significant or at least influential. Although this and other assumptions of citation analysis have been attacked, largely on the basis of the uncertainty about why authors cite papers, many of the objections have been answered (e.g. see Garfield & Cawkell 1975), and several studies have concluded that frequency of citation is a good indicator of research quality (e.g. Martino I97I, Virgo 1974). One problem with CA is that there is considerable variability in the citation patterns and practices between various disciplines. The most criticai factor in evaluaEing work by the number of citations is the average number of references cited per paper. For example, since mathematicians cite an average of 8 papers whereas biochemists cite around 20, it is to be expected that biochemistry papers have greater impact (as measured by citations per paper) thai) math papers (Garfield 1977), Thus, base-line data on the citation patterns of papers published in any research area are required if citation analysis is to be applied. As far as is known no citation analysis studies have been published in the area of periodontal research. We have examined the citation records of all papers excluding conference and supplementary material published in the Journal of Periodontology for the years 1966-1974 and in the Journal of Periodontal Research for the years 19661974. This material has enabled us to compile a list of the most cited papers found in these journals over the term covered
A N D R E I M E R S
and to make some inferences about the nature and, in certain cases, effectiveness of periodontal research. Materials and Methods The materials for this study were the Journal of Periodontology, Vols. 31-45 and the Journal of Periodontal Research, Vols. 1-9. In these citation studies, the number of citations to each article was counted using the Science Citation Index (Institute of Scientific Information, Philadelphia). This index is produced by listing all the items cited in papers published in nearly 2500 journals representing virtually every discipline in science. The items are arranged alphabetically by the name of the first author followed by other bibliographic information including the year, name of journal, volume and page. Under each item is a list of all the articles that cited the item. Thus to get a citation count one has to merely look up the author, identify the particular article and count the citations. Because this search was done manually it was possibly in some instances Io include citations to articles that were improperly cited, such as when the journal, volume and page number were correct but the author's name was misspelled. In addition as the /. Periodontal Res. Vols. 1-6 and /. Periodontol. Vols, 31-32 were not processed by the SCI; the reference lists in these journals were searched for references to articles in /. Periodontol. or Periodontal Res. In all cases records were kept of the number of citations in each year. In part of the study, articles in J. Periodontol. were groupi d into 5 categories defined below: 1) Animal research (AR) was defined is research in which the primary materi-'S were experimental animals or their t sues. 2) Human experimentation (HE) inclut; ii clinical correlative studies and stud s
C I T A T I O N
C H A R A C T E R I S T I C S
in which an experimental procedure was applied to humans or human tissues as well as histologie studies of normal and diseased human tissues. 3) periodontal techniques (PT). This group of papers were ones in which it was judged that the primary purpose of the paper was to introduce and/or explain procedures to practitioners. 4) Case Reports (CR) were reports of unusual pathologic conditions. Results
Number of References Per Paper Table 1 lists the number of references per article for J. Periodonlol. and J. Periodontal Res. as a function of year of pubiication. The range for the number of references is also given and it is clear that there is considerable variability in the number of citations found in the reference section of different articles.
A N D M O S T
C I T E D
P A P E R S
489
Price's index (per cent of references dated in previous five years) Year of Publication
J.P.
J.P.R.
references. In Table 2 we show the results of calculating the PI for J. Periodontol. and / . Periodontal Res. during different years of publication.
The Usage of Periodontal Research as a Function of Time In the previous section we have looked backward at the antecedents of a given piece of work to see the extent to which it depended on work done recently, i.e. on work done in the previous 5 years. In this section we look forward to see how subsequent authors have utilized the research Price's Index (PI) done at a particular time. Fig. 1 shows the Price (1970) has proposed an index that can be used to generate a hierarchy that number of citations made to the 55 articles corresponds very well with what scientists of J. Periodontol. Vol. 31, 1960 in the years intuit as hard science, soft science and non- that followed. There are quite clearly two science. The index is calculated by examining Ybe references to past literature and counting the number of references dated within the last 5 years and expressing the value as a per cent of the total number of
-, H !
Table 1 Number of references per article as a function r «•^ Ve ar of Pjbl ication 1933 1940 1»0 1960 1966 1970 1974
P
J. P. Range Avera 2.5 13.5 13.4 13.9 16.9 19.9 18.6
0-27 0-76 0-143 0-60 0-74 0-43 0-94
1 J.P. R. Rjinge Av«jrage
23.2 17 23.6
7-115 4-48 2-49
Fig. 1 Histogram illustrating the totai number of citations Xo the articles published in Journal of Periodontology. vol. 31, 1960, in tfia years 1961 to 1975.
B R U N E T T E ,
S I M O N
IJ 11
A N D
R E I M E R S
Flg. 4. Frequency distribution showing the percentage of papers published in Journal of Periodoniology (1966-1971) that are cited a given number of times in the first five years of publication.
Fig. 2. Histogram illustrating the average number ol citations to articles published in Journals of Periodontology and Periodontal Research as a function of year after publication.
humps on this curve. The first peak of utilization came three years after publication, whereas the second hump occurred 8 years after publication. The presence of the second peak may be due to changes in the journal structure of dental research in the years 1963-1972. Several journals that were likely to quote 3. Periodontol. were altered in those years including Arch. Oral Biol. (bimonthly to monthly 1966), Caries Research (started 1967) and /. Periodontol. which went from quarterly to a bimonthly in 1963, and fused with Periodontics and went monthly in 1969. One can obtain a more representative picture of the usage of periodontal research literature by calculating the average number of citations to all papers as a function af
time after publication. Fig. 2 gives this information for /. Periodontol. and / . Periodontal Res. published for the years 19661971. Note that the data for year 10 is hased solely on the 1966 issue, the data for year 9 based on the 1966 and 1967 issues, and so on. Hence, the values of the number of citations per paper at the earlier time periods after publication are based on more papers than the ones at later times after publications. It can be seen that the maximum frequency of citation of papers in both journals occurred 2 to 3 years after publication. Distribution of Citations Among Individual Papers We have thus far treated all the journal articles as equal and computed averages. Such a treatment disregards some of the most interesting information that can be obtained from this study; namely, that there is con-
Pli lirl Fig. (age logy time
n
n
n
3. Frequency distribution showing ihe percenof papers published in Journal of Penodonfo31 that are cited a given number of times In the period 1961 to 1975.
Fig. 5. Frequency distribution showing the percent of papers published in Journal of Periodontsl search {1966-1971) that are cited a given numbe. times in the first (ive years of publication.
C I T A T I O N
C H A R A C T E R I S T I C S
siderable variation in the number of times individual papers are cited. Figure 3 shows the percentage of papers that are cited a given number of times for 3. Periodontol Vol. 31. It is noteworthy that 15 years after publication, 18 % of the articles have never been cited. Figures 4 and 5 show the distribution of percentage of articles having a given number of citations in the first 5 years after publication for articles which appeared in /. Periodontol. (1966-1971) (Fig. 4) and J, Periodontal Res. (1966-1971) (Fig. 5). It is evident from the graphs that there is a large variation in the number of times articles in the same journal are cited. The distributions are highly asymmetrical with the majority of papers having low citation counts. Distribution of Citations Among Different Types of Research In our attempt to discover which types of research were associated with a high subsequent citation rate, we categorized the articles of J. Periodontol. into 4 arbitrary and somewhat crude groups; case reports (CR), animal research (AR), human experimentation (HE) and periodontat techniques (PT). The number of citations for
Distribution of citations among different types of research activity Mean Number of Citations Five Years After Pubiication in the Journal of Periodonto/ogy (Size of Sampie) Publication
CR.
A.R.
H,E.
P.T.
A N D M O S T
C I T E D
P A P E R S
491
the five years after publication were compiled and the results are shown in Table 3. There was one type of research activity that was clearly identifiable by citation frequency and that group was comprised of case reports which were found to have a very low number of citations. Most Cited Papers Table 4 lists the most cited papers in / . Periodontol. and /. Periodontal Res. published in tbe years covered by our study. The articles are ranked by total number of citations. Such a ranking is biased towards older papers as they have been collecting citations over a longer time period. Table 5 ranks the articles by number of citations/year. Because the number of citations, as a general rule, tends to decrease with time, tbis ranking would be expected to favour more recent articles. Late Bloomers One criticism which has been leveled at CA is that it would fail to detect papers that are ahead of their time. The classic example of a scientific advance that was ahead of its lime is the work of the pioneer geneticist, Gregor Mandel. We have arbitrarily defined "late bloomers" as papers that are cited 3 times or iess in the first 5 years after publication and 10 or more times in any subsequent 5 year period. In this study, only 10 papers met this criterion. Of this group, most were at the upper limit (i.e. 3) of citations in the first 5 years, and none of the group had failed to be cited in the first 5 years. It thus appears that late blooming is a comparatively rare event in periodontal research.
1962 1964 1966
(10) 3,6 (10)
(25) 4.8 (36)
Discussion Price (1970) has argued that the number of bibliographic references provides a measure of the social linkage of a field of
B R U N E T T E ,
S I M O N
A N D R E I M E R S
Papers from the Journal of Periodontology (1960-74) and Journal of Periodontal Research (196S-74} receiving the highest total number of citations from year of pubiication through 1976
1.
Loe, H., Theilade, E. & Jensen, S. B. 1965. Experimental gingivitis in man. J. Periodontol. 36; 177.
276
2.
TheiJade, E., Wright, W. H., Jensen, S. B. & Loe, H. 1966. Experimental gingivitis in man. IL A longitudinal clinical and bacteriological investigation. J. Periodontal Res. 1: L
129
3.
Loe, H. & Schiott, C R. 1970. The effect of mouthrinses and topical application of chlorhexidine on the development of dental plaque and gingivitis in man. / . Periodontal Res. 5: 79.
100
4.
Wilderman, M. N., Wentz, F. M. & Orban, B. J. 1960. Histogenesis of repair after mucogingival surgery. / . Periodontol. 31: 283.
84
5.
Loe, H. 1967. The gingival index, the plaque index, and the retention index systems. J. Periodontol. 38: 610.
79
6.
Schroeder, H. E. & Theilade, K. 1966. Electron microscopy of normal human gingiva] epithelium. J. Periodontal Res. 1: 95.
68
7.
Freedman, H. L., Listgarten, M. A. & Taichman, N. S. 1968. Electron microscopic features of chronically inflamed human gingiva. J. Periodonial Res. 3: 313.
62
8.
Gibhons. R. J. & Van Houte, J. 1973. On the formation of dental plaques, i. PeriodontoL 44: 347.
60
9.
Brandtzaeg, P. 1966. Local factors of resistance in the gingival area. / . Periodontal Res. 1: 19.
59
10.
Tempel, T. R., Snyderman, R., Jordan, H. V. & Mergenhagen, S. E. 1970. Factors from saliva and oral bacteria chemotactic for polymorphonuclear leukocytes: their possible role in gingival inflammation. J. Periodontol. 41: 71.
58
11.
Amim, S. S. 1963. The use of disclosing agents for measuring tooth cleanliness. / . Periodontol. 34: 227.
51
12.
Rolla, G., Loe, H. & Schiott, C. R. 1970. The affinity of chlorhexidine for hydroxyapatite and salivary mucins. 7. Periodontal Res. 5: 90.
50
13.
Gjermo, P., Baastaad, K. L. & Rolla, G. 1970. The plaque-inhibiting capacity of 11 antibacterial compounds. J. Periodonta! Res. 5: 102.
46
14.
StaffiJeno, H., Wentz, F. M. & Orban, B. J. 1962. Hislologic study of healing of split thickness flap surgery in dogs. / . PeriodontoL 33: 56.
45
15.
Schalihorn, R. B. 1968. The use of autogenous hip marrow biopsy implants for bony crater defects. J. Periodontol. 39: 145.
45
16.
Mann, M. V. 1963. The correlation of gingivitis, pocket depth and exudate from the gingival crevice. / . Periodontol. 34: 379.
44
17.
Nabers, C. L. & O'Leary, T. J. 1965. Autogenous bone transplants in the treatment of osseous defects. J. Periodontol. 36: 5.
44
C I T A T I O N
C H A R A C T E R I S T I C S
A N D
M O S T
C I T E D
P A P E R S
18.
Ochsenbein, C. I960. Newer concepts of mucogingival surgery. J. Periodontol. 31; 175.
19.
Friedman, N. 1962. Mucogingival surgery: the apically repositioned flap. J, Periodontol. 33: 328. Oliver, R. C , Holm-Pedersen, P. & Loe, H. 1969. The correlation between clinical scoring, exudate measurements and microscopic evaluation of inflammation in the gingiva. J. Periodontol. 40: 201. Muhlemann, H. R. 1960. Ten years of tooth mobility measurements. /. Periodontol, 31: 110.
493
43 42
42 42
science. He has grouped articles into three categories: (1) Ex cathedra: These are papers that contain less than 10 references per paper. Unreferenced articles are thought to occur when an experienced scientist makes a pronouncement out of his innate knowledge of what should be or what is.
jects and dates and has found that the median PI is 32 with a lower quartile of 21 and upper quartile of 42. Hard sciences are found in the upper quartile and are called research front areas. Physics and biochemistry would be two examples of this group and It is felt that such fields are undergoing rapid progress. The PI of periodontal research literature varies. In some
(2) Normal articles: These contain 10-20 references. (3) Review artieles: These contain more than 20 references per paper. In recent years the average number of references per article in / . Periodontol. are found in the "normal" rarjge which marks a significant change from the "ex cathedra" nature of its origins. One could argue that such a change indicates the development of periodontology into a area of scholarship. The J. Periodontal Res. values fall slightly above the normal range. On average it appears that authors publishing in /. Periodontal Res. have a tendency to cite more puhlications than those publishing in / . Peyiodontoi Examination of the range of references per article indicates that all types of paper, i.e., (ex cathedra, normal and review), can be found in both journals, The calculation of the proportion of r' ferences dated in the last five years (PI) g ves another insight into periodontal res- arch. The bench marks for such a study a e provided by the data of Price (1970) v^o examined 162 journals of various sub-
years it is found in this area but in other years, it is found in the range occupied by the "normal" group of sciences; that is, those that move at a somewhat slower pace. The frequencies of citations to papers published in / . Periodontal Res. and /. Periodontol. are shown as a function of time in Figures 1 and 2. This data is in agreement with the work of Margolis (1967) who examined biomedical research, in Ihat we found, that on average, the citations to a paper peaked three years after its publication. One other fact is evident from inspection of Figure 2; namely, that /. Periodontal Res. is cited more extensively than J. Periodontol. A consideration in the comparison of journals by citation analysis is the nature of the journals which cite a particular journal. This information, for recent years, may be obtained from the Journal of Citation Reports (which is part of the SCI). In 1976 (Garfield 1976b), both journals drew the vast majoritty of their citations from the dental literature. As might he expected, 7. Periodontol. and /.
4
B R U N E T T E ,
S I M O N
A N D
R E I M E R S
Table 5
Papers from the Journal of Periodontology (1960-74) and Journal of Periodontal Research (1966-74) receiving the highest number of citations per year from year of publication through 1976 Mean Cit./Yr.
Rank 1.
Loe, H., Theiiade, E. & Jensen, S. B. 1965. Experimental gingivitis in man. / . Periodontol. 36: 177.
23
2.
Gibbons, R. J. & Van Houte, J. 1973. On the formation of dentai plaques. / . Periodontol. 44: 347.
15
3.
Loe, H. & Schiott, C. R. 1970. The effect of mouthrinses and topical application of chlorhexidine on the development of dental plaque and gingivitis in man. J. Periodontal Res. 5: 79.
14.3
4.
Theiiade, E., Wright, W. H., Jensen, S. B. & Loe, H. 3966. Experimental gingivitis in man. II. A longitudinal clinical and hacteriological investigation. J. Periodontal Res. 1: 1.
11.7
5.
Tempel, T. R., Snyderman, 1970. Factors from saliva morphonuclear leukocytes: mation. J. Periodontol. 41;
R., Jordan, H. V. & Mergenhagen, S. E. and oral bacteria chemotoctic for polytheir possible role in gingival inflamma71.
S.3
6.
Horton, J. E., Leiken, S. & Oppenheim, J. J. 1972. Human lymphoproliferative reaction to saliva and dental plaque deposits: An in vitro correlation w. periodontal disease. J. Periodontol. 43: 522.
8.2
7.
Loe, H. 1967. The gingival index, the plaque index, and the retention index system. J. Periodontol. 38: 610.
7.9
8.
Jordan, H. V., Keyes, P. H. & Bellack, S. 1972. Periodontat lesions in hamsters and gnotobiotie rats infected with actinomyces of human origin. / . Periodontal Res. 7: 21.
7.6
9.
Rolla, G., Loe, H. & Schiott, C. R. 1970. The affinity of chlorhexidine for hydroxy apatite and salivary mucins. J. Periodantal Res. 5: 90.
1.1
10 .
Freedman, H. L., Listgarten, M. A. & Taichman, N. S. 1968. Electron microscopic features of chronically inflamed human gingiva. / . Periodontal Res. 3: 313.
6.9
U.
Lindhe, J., Hamp, S. E. & Loe, H. 1973. Experimental periodontitis in the beagle dog. J. Periodonial Res. 8: 1.
6.8
12.
Gjermo, P., Baastaad, K. L. & Rolla, G. 1970. The plaque-inhibiting capacity of 11 antibacterial compounds. / . Periodontal Res. 5: 102.
6.6
13.
Hamp, S. £., Lindhe, J. & Loe, H. 1973. Long term effect of chlorhexidine on developing gingivitis in the beagle dog. / . Periodontal Res. 8: 63.
6.5
14.
Suomi, J. D., Greene, J. C , Vermilion, J. R., Doyle, h, Chang, 3. J. & Leatherwood, E. C. 1971. The effect of controlled oral hygiene procedures on the progression of periodontal disease in adults: Results after third and final year. / . Periodontol. 42: 152.
6.5
15.
Schroeder, H. E. & Theiiade, K. 1966. Electron microscopy of normal human gingiva epithelium. / . Periodontal Res. 1: 95.
6.2
C I T A T I O N
C H A R A C T E R I S T I C S
A N D
M O S T
C I T E D
P A P E R S
Socransky, S. S. 1971. The oral microbiota of man from birth to senility. /. Periodontol. 42: 485. Attstrom, R. 1970. Presence of leukocytes in crevices of healthy and chronically inflamed gingivae. J. Periodontal Res, 5: 42. BrandtZraeg, P. 1966. Local factors of resistance in the gingival area. /. Periodontal Res. 1: 19. Oliver, R. C , Holm-Pedersen, P. & Loe, H. 1969. The correlation between clinical scoring, exudate measurements and microscopic evaluation of inflammation of inflammation in the gingiva. J. Periodontol. 40; 201.
495
5.8 5.6 5.4
5.3
Schallhorn, R. G. 1968. The use of autogenous hip marrow biopsy implants for botiy crater defects. /. Periodonto], 39: 145.
5
Attstrom, R. & Egelberg, J. 1971. Presence of leukocytes within the gingival crevices during developing gingivitis in dogs. /. Periodontal Res. 6: 110.
5
Genco, R. J., Mashimo, P. A., Krygier, G. & Ellison, S. A. 1974. Antibody-mediated effects on the periodontium. /. Periodontal Res, 45: 330.
5
Periodontal Res. cited themselves and each other extensively. Articles which appeared in Arch. Oral Biol. and /. Dent. Res. also supplied a significant proportion of the cifations to the periodontal journals. Artides which appeared in the Scandinavian detital journals cited J. Periodontal Res. more extensively than J. Periodontol. The shape of Figures 3 and 4 is roughly what would be expected from earlier work in the distribution of citations among individual papers. Price found that the percentage of papers cited a given number of times was proportional to n-(2-'5
Citation records of papers published in the Journal of Periodontology and the Journal of Periodontal Research D. M. BRUNETTE, M . J. SIMON AND M . A. REIMERS
Medical Research Council Group in PeriodontaS Physiology, Faculty of Dentistry, University of Toronto, Toronto, Ontario, Canada The citation records of the papers published in the Journal of Periodontology vols. 31-45 and Journal of Periodontal Research vols. 1-9 have been examined and lists of most cited papers ranked by tolal number of citations and citations per year compiled. Several diverse approaches to periodontal research are represented in these lists including microbiology, histology, immunology, and surgery. The pattern of periodontal literature usage with time resembles that found for biomedical literature and other bibliographic indices such as Price's Index and the number of references per paper place periodontal research in or close to the range of values found for "normal" sciences. The distribution of citations among individual papers resembles that found in other sciences in that the majority of papers have very little impact. In the first five years following publication, 18 % of papers published in J. Periodontol. were not cited while less than 10 % were cited ten times or more. The record for papers published in /. Periodontal Res. is slightly better, 9.5 % were not cited and 12.6 % were cited ten times or more in the five years following publication. In this study, there were no instances of papers that failed to be cited in the first five years achieving ten or more citations in any subsequent five year period. One class of research publication, case reports, was associated with a very low citation frequency. It appears that the influence of case reports on subsequent research is small. (Accepted for publication April 26, 1978)
Introduction It is common knowledge that the amount of scientific research literature is increasing sttidily. The annual growth rate has been eaf mated to be 7 % of the extant literature wl-ch is now believed to be close to 7.5
million items (Garfield 1976a). However, ^^^ effectiveness of much of this research has been questioned both in the popular press and in scientific forums. One approach to assessing the effectiveness of scientific research is citation analysis (CA). The fundamental question that can be
B R U N E T T E ,
S I M O N
answered by CA, which uses as its primary source the Science Citation Index (SCI)., is where and by whom a particular piece of scientific work has been cited in the literature. In CA studies the first step is to count the number of times an article has been cited, In general, it is assumed that papers that are never cited are relatively unimportant whereas those that are highly cited are significant or at least influential. Although this and other assumptions of citation analysis have been attacked, largely on the basis of the uncertainty about why authors cite papers, many of the objections have been answered (e.g. see Garfield & Cawkell 1975), and several studies have concluded that frequency of citation is a good indicator of research quality (e.g. Martino I97I, Virgo 1974). One problem with CA is that there is considerable variability in the citation patterns and practices between various disciplines. The most criticai factor in evaluaEing work by the number of citations is the average number of references cited per paper. For example, since mathematicians cite an average of 8 papers whereas biochemists cite around 20, it is to be expected that biochemistry papers have greater impact (as measured by citations per paper) thai) math papers (Garfield 1977), Thus, base-line data on the citation patterns of papers published in any research area are required if citation analysis is to be applied. As far as is known no citation analysis studies have been published in the area of periodontal research. We have examined the citation records of all papers excluding conference and supplementary material published in the Journal of Periodontology for the years 1966-1974 and in the Journal of Periodontal Research for the years 19661974. This material has enabled us to compile a list of the most cited papers found in these journals over the term covered
A N D R E I M E R S
and to make some inferences about the nature and, in certain cases, effectiveness of periodontal research. Materials and Methods The materials for this study were the Journal of Periodontology, Vols. 31-45 and the Journal of Periodontal Research, Vols. 1-9. In these citation studies, the number of citations to each article was counted using the Science Citation Index (Institute of Scientific Information, Philadelphia). This index is produced by listing all the items cited in papers published in nearly 2500 journals representing virtually every discipline in science. The items are arranged alphabetically by the name of the first author followed by other bibliographic information including the year, name of journal, volume and page. Under each item is a list of all the articles that cited the item. Thus to get a citation count one has to merely look up the author, identify the particular article and count the citations. Because this search was done manually it was possibly in some instances Io include citations to articles that were improperly cited, such as when the journal, volume and page number were correct but the author's name was misspelled. In addition as the /. Periodontal Res. Vols. 1-6 and /. Periodontol. Vols, 31-32 were not processed by the SCI; the reference lists in these journals were searched for references to articles in /. Periodontol. or Periodontal Res. In all cases records were kept of the number of citations in each year. In part of the study, articles in J. Periodontol. were groupi d into 5 categories defined below: 1) Animal research (AR) was defined is research in which the primary materi-'S were experimental animals or their t sues. 2) Human experimentation (HE) inclut; ii clinical correlative studies and stud s
C I T A T I O N
C H A R A C T E R I S T I C S
in which an experimental procedure was applied to humans or human tissues as well as histologie studies of normal and diseased human tissues. 3) periodontal techniques (PT). This group of papers were ones in which it was judged that the primary purpose of the paper was to introduce and/or explain procedures to practitioners. 4) Case Reports (CR) were reports of unusual pathologic conditions. Results
Number of References Per Paper Table 1 lists the number of references per article for J. Periodonlol. and J. Periodontal Res. as a function of year of pubiication. The range for the number of references is also given and it is clear that there is considerable variability in the number of citations found in the reference section of different articles.
A N D M O S T
C I T E D
P A P E R S
489
Price's index (per cent of references dated in previous five years) Year of Publication
J.P.
J.P.R.
references. In Table 2 we show the results of calculating the PI for J. Periodontol. and / . Periodontal Res. during different years of publication.
The Usage of Periodontal Research as a Function of Time In the previous section we have looked backward at the antecedents of a given piece of work to see the extent to which it depended on work done recently, i.e. on work done in the previous 5 years. In this section we look forward to see how subsequent authors have utilized the research Price's Index (PI) done at a particular time. Fig. 1 shows the Price (1970) has proposed an index that can be used to generate a hierarchy that number of citations made to the 55 articles corresponds very well with what scientists of J. Periodontol. Vol. 31, 1960 in the years intuit as hard science, soft science and non- that followed. There are quite clearly two science. The index is calculated by examining Ybe references to past literature and counting the number of references dated within the last 5 years and expressing the value as a per cent of the total number of
-, H !
Table 1 Number of references per article as a function r «•^ Ve ar of Pjbl ication 1933 1940 1»0 1960 1966 1970 1974
P
J. P. Range Avera 2.5 13.5 13.4 13.9 16.9 19.9 18.6
0-27 0-76 0-143 0-60 0-74 0-43 0-94
1 J.P. R. Rjinge Av«jrage
23.2 17 23.6
7-115 4-48 2-49
Fig. 1 Histogram illustrating the totai number of citations Xo the articles published in Journal of Periodontology. vol. 31, 1960, in tfia years 1961 to 1975.
B R U N E T T E ,
S I M O N
IJ 11
A N D
R E I M E R S
Flg. 4. Frequency distribution showing the percentage of papers published in Journal of Periodoniology (1966-1971) that are cited a given number of times in the first five years of publication.
Fig. 2. Histogram illustrating the average number ol citations to articles published in Journals of Periodontology and Periodontal Research as a function of year after publication.
humps on this curve. The first peak of utilization came three years after publication, whereas the second hump occurred 8 years after publication. The presence of the second peak may be due to changes in the journal structure of dental research in the years 1963-1972. Several journals that were likely to quote 3. Periodontol. were altered in those years including Arch. Oral Biol. (bimonthly to monthly 1966), Caries Research (started 1967) and /. Periodontol. which went from quarterly to a bimonthly in 1963, and fused with Periodontics and went monthly in 1969. One can obtain a more representative picture of the usage of periodontal research literature by calculating the average number of citations to all papers as a function af
time after publication. Fig. 2 gives this information for /. Periodontol. and / . Periodontal Res. published for the years 19661971. Note that the data for year 10 is hased solely on the 1966 issue, the data for year 9 based on the 1966 and 1967 issues, and so on. Hence, the values of the number of citations per paper at the earlier time periods after publication are based on more papers than the ones at later times after publications. It can be seen that the maximum frequency of citation of papers in both journals occurred 2 to 3 years after publication. Distribution of Citations Among Individual Papers We have thus far treated all the journal articles as equal and computed averages. Such a treatment disregards some of the most interesting information that can be obtained from this study; namely, that there is con-
Pli lirl Fig. (age logy time
n
n
n
3. Frequency distribution showing ihe percenof papers published in Journal of Penodonfo31 that are cited a given number of times In the period 1961 to 1975.
Fig. 5. Frequency distribution showing the percent of papers published in Journal of Periodontsl search {1966-1971) that are cited a given numbe. times in the first (ive years of publication.
C I T A T I O N
C H A R A C T E R I S T I C S
siderable variation in the number of times individual papers are cited. Figure 3 shows the percentage of papers that are cited a given number of times for 3. Periodontol Vol. 31. It is noteworthy that 15 years after publication, 18 % of the articles have never been cited. Figures 4 and 5 show the distribution of percentage of articles having a given number of citations in the first 5 years after publication for articles which appeared in /. Periodontol. (1966-1971) (Fig. 4) and J, Periodontal Res. (1966-1971) (Fig. 5). It is evident from the graphs that there is a large variation in the number of times articles in the same journal are cited. The distributions are highly asymmetrical with the majority of papers having low citation counts. Distribution of Citations Among Different Types of Research In our attempt to discover which types of research were associated with a high subsequent citation rate, we categorized the articles of J. Periodontol. into 4 arbitrary and somewhat crude groups; case reports (CR), animal research (AR), human experimentation (HE) and periodontat techniques (PT). The number of citations for
Distribution of citations among different types of research activity Mean Number of Citations Five Years After Pubiication in the Journal of Periodonto/ogy (Size of Sampie) Publication
CR.
A.R.
H,E.
P.T.
A N D M O S T
C I T E D
P A P E R S
491
the five years after publication were compiled and the results are shown in Table 3. There was one type of research activity that was clearly identifiable by citation frequency and that group was comprised of case reports which were found to have a very low number of citations. Most Cited Papers Table 4 lists the most cited papers in / . Periodontol. and /. Periodontal Res. published in tbe years covered by our study. The articles are ranked by total number of citations. Such a ranking is biased towards older papers as they have been collecting citations over a longer time period. Table 5 ranks the articles by number of citations/year. Because the number of citations, as a general rule, tends to decrease with time, tbis ranking would be expected to favour more recent articles. Late Bloomers One criticism which has been leveled at CA is that it would fail to detect papers that are ahead of their time. The classic example of a scientific advance that was ahead of its lime is the work of the pioneer geneticist, Gregor Mandel. We have arbitrarily defined "late bloomers" as papers that are cited 3 times or iess in the first 5 years after publication and 10 or more times in any subsequent 5 year period. In this study, only 10 papers met this criterion. Of this group, most were at the upper limit (i.e. 3) of citations in the first 5 years, and none of the group had failed to be cited in the first 5 years. It thus appears that late blooming is a comparatively rare event in periodontal research.
1962 1964 1966
(10) 3,6 (10)
(25) 4.8 (36)
Discussion Price (1970) has argued that the number of bibliographic references provides a measure of the social linkage of a field of
B R U N E T T E ,
S I M O N
A N D R E I M E R S
Papers from the Journal of Periodontology (1960-74) and Journal of Periodontal Research (196S-74} receiving the highest total number of citations from year of pubiication through 1976
1.
Loe, H., Theilade, E. & Jensen, S. B. 1965. Experimental gingivitis in man. J. Periodontol. 36; 177.
276
2.
TheiJade, E., Wright, W. H., Jensen, S. B. & Loe, H. 1966. Experimental gingivitis in man. IL A longitudinal clinical and bacteriological investigation. J. Periodontal Res. 1: L
129
3.
Loe, H. & Schiott, C R. 1970. The effect of mouthrinses and topical application of chlorhexidine on the development of dental plaque and gingivitis in man. / . Periodontal Res. 5: 79.
100
4.
Wilderman, M. N., Wentz, F. M. & Orban, B. J. 1960. Histogenesis of repair after mucogingival surgery. / . Periodontol. 31: 283.
84
5.
Loe, H. 1967. The gingival index, the plaque index, and the retention index systems. J. Periodontol. 38: 610.
79
6.
Schroeder, H. E. & Theilade, K. 1966. Electron microscopy of normal human gingiva] epithelium. J. Periodontal Res. 1: 95.
68
7.
Freedman, H. L., Listgarten, M. A. & Taichman, N. S. 1968. Electron microscopic features of chronically inflamed human gingiva. J. Periodonial Res. 3: 313.
62
8.
Gibhons. R. J. & Van Houte, J. 1973. On the formation of dental plaques, i. PeriodontoL 44: 347.
60
9.
Brandtzaeg, P. 1966. Local factors of resistance in the gingival area. / . Periodontal Res. 1: 19.
59
10.
Tempel, T. R., Snyderman, R., Jordan, H. V. & Mergenhagen, S. E. 1970. Factors from saliva and oral bacteria chemotactic for polymorphonuclear leukocytes: their possible role in gingival inflammation. J. Periodontol. 41: 71.
58
11.
Amim, S. S. 1963. The use of disclosing agents for measuring tooth cleanliness. / . Periodontol. 34: 227.
51
12.
Rolla, G., Loe, H. & Schiott, C. R. 1970. The affinity of chlorhexidine for hydroxyapatite and salivary mucins. 7. Periodontal Res. 5: 90.
50
13.
Gjermo, P., Baastaad, K. L. & Rolla, G. 1970. The plaque-inhibiting capacity of 11 antibacterial compounds. J. Periodonta! Res. 5: 102.
46
14.
StaffiJeno, H., Wentz, F. M. & Orban, B. J. 1962. Hislologic study of healing of split thickness flap surgery in dogs. / . PeriodontoL 33: 56.
45
15.
Schalihorn, R. B. 1968. The use of autogenous hip marrow biopsy implants for bony crater defects. J. Periodontol. 39: 145.
45
16.
Mann, M. V. 1963. The correlation of gingivitis, pocket depth and exudate from the gingival crevice. / . Periodontol. 34: 379.
44
17.
Nabers, C. L. & O'Leary, T. J. 1965. Autogenous bone transplants in the treatment of osseous defects. J. Periodontol. 36: 5.
44
C I T A T I O N
C H A R A C T E R I S T I C S
A N D
M O S T
C I T E D
P A P E R S
18.
Ochsenbein, C. I960. Newer concepts of mucogingival surgery. J. Periodontol. 31; 175.
19.
Friedman, N. 1962. Mucogingival surgery: the apically repositioned flap. J, Periodontol. 33: 328. Oliver, R. C , Holm-Pedersen, P. & Loe, H. 1969. The correlation between clinical scoring, exudate measurements and microscopic evaluation of inflammation in the gingiva. J. Periodontol. 40: 201. Muhlemann, H. R. 1960. Ten years of tooth mobility measurements. /. Periodontol, 31: 110.
493
43 42
42 42
science. He has grouped articles into three categories: (1) Ex cathedra: These are papers that contain less than 10 references per paper. Unreferenced articles are thought to occur when an experienced scientist makes a pronouncement out of his innate knowledge of what should be or what is.
jects and dates and has found that the median PI is 32 with a lower quartile of 21 and upper quartile of 42. Hard sciences are found in the upper quartile and are called research front areas. Physics and biochemistry would be two examples of this group and It is felt that such fields are undergoing rapid progress. The PI of periodontal research literature varies. In some
(2) Normal articles: These contain 10-20 references. (3) Review artieles: These contain more than 20 references per paper. In recent years the average number of references per article in / . Periodontol. are found in the "normal" rarjge which marks a significant change from the "ex cathedra" nature of its origins. One could argue that such a change indicates the development of periodontology into a area of scholarship. The J. Periodontal Res. values fall slightly above the normal range. On average it appears that authors publishing in /. Periodontal Res. have a tendency to cite more puhlications than those publishing in / . Peyiodontoi Examination of the range of references per article indicates that all types of paper, i.e., (ex cathedra, normal and review), can be found in both journals, The calculation of the proportion of r' ferences dated in the last five years (PI) g ves another insight into periodontal res- arch. The bench marks for such a study a e provided by the data of Price (1970) v^o examined 162 journals of various sub-
years it is found in this area but in other years, it is found in the range occupied by the "normal" group of sciences; that is, those that move at a somewhat slower pace. The frequencies of citations to papers published in / . Periodontal Res. and /. Periodontol. are shown as a function of time in Figures 1 and 2. This data is in agreement with the work of Margolis (1967) who examined biomedical research, in Ihat we found, that on average, the citations to a paper peaked three years after its publication. One other fact is evident from inspection of Figure 2; namely, that /. Periodontal Res. is cited more extensively than J. Periodontol. A consideration in the comparison of journals by citation analysis is the nature of the journals which cite a particular journal. This information, for recent years, may be obtained from the Journal of Citation Reports (which is part of the SCI). In 1976 (Garfield 1976b), both journals drew the vast majoritty of their citations from the dental literature. As might he expected, 7. Periodontol. and /.
4
B R U N E T T E ,
S I M O N
A N D
R E I M E R S
Table 5
Papers from the Journal of Periodontology (1960-74) and Journal of Periodontal Research (1966-74) receiving the highest number of citations per year from year of publication through 1976 Mean Cit./Yr.
Rank 1.
Loe, H., Theiiade, E. & Jensen, S. B. 1965. Experimental gingivitis in man. / . Periodontol. 36: 177.
23
2.
Gibbons, R. J. & Van Houte, J. 1973. On the formation of dentai plaques. / . Periodontol. 44: 347.
15
3.
Loe, H. & Schiott, C. R. 1970. The effect of mouthrinses and topical application of chlorhexidine on the development of dental plaque and gingivitis in man. J. Periodontal Res. 5: 79.
14.3
4.
Theiiade, E., Wright, W. H., Jensen, S. B. & Loe, H. 3966. Experimental gingivitis in man. II. A longitudinal clinical and hacteriological investigation. J. Periodontal Res. 1: 1.
11.7
5.
Tempel, T. R., Snyderman, 1970. Factors from saliva morphonuclear leukocytes: mation. J. Periodontol. 41;
R., Jordan, H. V. & Mergenhagen, S. E. and oral bacteria chemotoctic for polytheir possible role in gingival inflamma71.
S.3
6.
Horton, J. E., Leiken, S. & Oppenheim, J. J. 1972. Human lymphoproliferative reaction to saliva and dental plaque deposits: An in vitro correlation w. periodontal disease. J. Periodontol. 43: 522.
8.2
7.
Loe, H. 1967. The gingival index, the plaque index, and the retention index system. J. Periodontol. 38: 610.
7.9
8.
Jordan, H. V., Keyes, P. H. & Bellack, S. 1972. Periodontat lesions in hamsters and gnotobiotie rats infected with actinomyces of human origin. / . Periodontal Res. 7: 21.
7.6
9.
Rolla, G., Loe, H. & Schiott, C. R. 1970. The affinity of chlorhexidine for hydroxy apatite and salivary mucins. J. Periodantal Res. 5: 90.
1.1
10 .
Freedman, H. L., Listgarten, M. A. & Taichman, N. S. 1968. Electron microscopic features of chronically inflamed human gingiva. / . Periodontal Res. 3: 313.
6.9
U.
Lindhe, J., Hamp, S. E. & Loe, H. 1973. Experimental periodontitis in the beagle dog. J. Periodonial Res. 8: 1.
6.8
12.
Gjermo, P., Baastaad, K. L. & Rolla, G. 1970. The plaque-inhibiting capacity of 11 antibacterial compounds. / . Periodontal Res. 5: 102.
6.6
13.
Hamp, S. £., Lindhe, J. & Loe, H. 1973. Long term effect of chlorhexidine on developing gingivitis in the beagle dog. / . Periodontal Res. 8: 63.
6.5
14.
Suomi, J. D., Greene, J. C , Vermilion, J. R., Doyle, h, Chang, 3. J. & Leatherwood, E. C. 1971. The effect of controlled oral hygiene procedures on the progression of periodontal disease in adults: Results after third and final year. / . Periodontol. 42: 152.
6.5
15.
Schroeder, H. E. & Theiiade, K. 1966. Electron microscopy of normal human gingiva epithelium. / . Periodontal Res. 1: 95.
6.2
C I T A T I O N
C H A R A C T E R I S T I C S
A N D
M O S T
C I T E D
P A P E R S
Socransky, S. S. 1971. The oral microbiota of man from birth to senility. /. Periodontol. 42: 485. Attstrom, R. 1970. Presence of leukocytes in crevices of healthy and chronically inflamed gingivae. J. Periodontal Res, 5: 42. BrandtZraeg, P. 1966. Local factors of resistance in the gingival area. /. Periodontal Res. 1: 19. Oliver, R. C , Holm-Pedersen, P. & Loe, H. 1969. The correlation between clinical scoring, exudate measurements and microscopic evaluation of inflammation of inflammation in the gingiva. J. Periodontol. 40; 201.
495
5.8 5.6 5.4
5.3
Schallhorn, R. G. 1968. The use of autogenous hip marrow biopsy implants for botiy crater defects. /. Periodonto], 39: 145.
5
Attstrom, R. & Egelberg, J. 1971. Presence of leukocytes within the gingival crevices during developing gingivitis in dogs. /. Periodontal Res. 6: 110.
5
Genco, R. J., Mashimo, P. A., Krygier, G. & Ellison, S. A. 1974. Antibody-mediated effects on the periodontium. /. Periodontal Res, 45: 330.
5
Periodontal Res. cited themselves and each other extensively. Articles which appeared in Arch. Oral Biol. and /. Dent. Res. also supplied a significant proportion of the cifations to the periodontal journals. Artides which appeared in the Scandinavian detital journals cited J. Periodontal Res. more extensively than J. Periodontol. The shape of Figures 3 and 4 is roughly what would be expected from earlier work in the distribution of citations among individual papers. Price found that the percentage of papers cited a given number of times was proportional to n-(2-'5
