Research in Social and Administrative Pharmacy 11 (2015) 686–695
Original Research
Quality of pharmacy-specific Medical Subject Headings (MeSH) assignment in pharmacy journals indexed in MEDLINE Fernando Minguet, M.Sc., Pharm.D.a, Teresa M. Salgado, Ph.D.b, Lucienne van den Boogerd, Pharm.D.c, Fernando Fernandez-Llimos, Ph.D., M.B.A.d,* a Valencian Pharmacy Practice Research Group, Valencia, Spain Department of Clinical, Social, and Administrative Sciences, College of Pharmacy, University of Michigan, MI, USA c Department of Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands d Research Institute for Medicines (iMed.ULisboa), Department of Social-Pharmacy, Faculty of Pharmacy, University of Lisbon, Lisbon, Portugal b
Abstract Background: The Medical Subject Headings (MeSH) is the National Library of Medicine (NLM) controlled vocabulary for indexing articles. Inaccuracies in the MeSH thesaurus have been reported for several areas including pharmacy. Objectives: To assess the quality of pharmacy-specific MeSH assignment to articles indexed in pharmacy journals. Methods: The 10 journals containing the highest number of articles published in 2012 indexed under the MeSH ‘Pharmacists’ were identified. All articles published over a 5-year period (2008–2012) in the 10 previously selected journals were retrieved from PubMed. MeSH terms used to index these articles were extracted and pharmacy-specific MeSH terms were identified. The frequency of use of pharmacy-specific MeSH terms was calculated across journals. Results: A total of 6989 articles were retrieved from the 10 pharmacy journals, of which 328 (4.7%) were articles not fully indexed and therefore did not contain any MeSH terms assigned. Among the 6661 articles fully indexed, the mean number of MeSH terms was 10.1 (SD ¼ 4.0), being 1.0 (SD ¼ 1.3) considered as Major MeSH. Both values significantly varied across journals. The mean number of pharmacy-specific MeSH terms per article was 0.9 (SD ¼ 1.2). A total of 3490 (52.4%) of the 6661 articles were indexed in pharmacy journals without a single pharmacy-specific MeSH. Of the total 67193 MeSH terms assigned to articles, on average 10.5% (SD ¼ 13.9) were pharmacy-specific MeSH. A statistically significant different pattern of pharmacy-specific MeSH assignment was identified across journals (Kruskal–Wallis P ! 0.001). Conclusions: The quality of assignment of the existing pharmacy-specific MeSH terms to articles indexed in pharmacy journals can be improved to further enhance evidence gathering in pharmacy. Over half of the
Conflicts of interest disclosure: None to declare. * Corresponding author. Departamento de Socio-Farma´cia, Faculdade de Farmacia, Universidade de Lisboa, Av. Prof. Gama Pinto, 1649-003 Lisboa, Portugal. Tel.: þ351 217 946 400; fax: þ351 217 946 470. E-mail address: f-llimos@ff.ul.pt (F. Fernandez-Llimos). 1551-7411/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.sapharm.2014.11.004
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
687
articles published in the top-10 journals publishing pharmacy literature were indexed without a single pharmacy-specific MeSH. Ó 2015 Elsevier Inc. All rights reserved. Keywords: Evidence-based practice; Medical subject headings; Periodicals as topic; Pharmaceutical services; Pharmacists; Pharmacy
Introduction Searching the literature when attempting to answer a given clinical or research question can sometimes be a difficult process, as the many existing databases nowadays provide access to millions of records. An efficient literature search is an essential skill for evidence-based practice1 and searching in electronic databases offers the most effective use of time and the greatest potential yield.2 To facilitate article retrieval, databases adopt controlled vocabularies as a means of cataloging articles. An appropriate assignment of a controlled vocabulary defines the quality of an information retrieval system.3 Therefore, problems with controlled vocabulary assignment can hinder an article’s diffusion or even cause it to fall into oblivion.4 Common user errors in database searches stem from inappropriate selection of search terms.5 MEDLINE is the National Library of Medicine’s (NLM) journal citation database and contains citations and abstracts for biomedical literature from around the world. PubMed is a free available search engine that provides access to MEDLINE, among other databases, and it contains over 23 million citations.6 PubMed uses the NLM’s controlled vocabulary thesaurus – the Medical Subject Headings (MeSH) – for indexing and cataloging articles.7,8 Although criticized for not appropriately covering some areas, such as pharmacy,9 the MeSH database is continuously updated and revised annually.7 The intent of using the MeSH terminology is to facilitate search retrieval by eliminating (or accounting for) the use of variant terminology for the same concept.10 MeSH indexing facilitates literature retrieval as it provides a powerful method of narrowing search results.11 In fact, literature searches performed using MeSH terms were shown to retrieve a fewer number of irrelevant citations.12 However, knowledge and appropriate use of MeSH are critical elements to conduct an effective MEDLINE search.13 Besides providing a consistent way of retrieving information, MeSH can also be
indicative of popular topics within a scientific area.14–16 Articles indexed under the same set of MeSH terms should refer to the same research field.17 During the indexing process in PubMed, articles are assigned an average of 10 MeSH terms by professional catalogers to reflect their content.18,19 The amount of MeSH terms assigned to one article represents the depth of indexing.20 NLM’s MEDLINE catalogers use the MeSH Browser, an online vocabulary look-up aid with virtually complete MeSH records, to find the term that best describes the concept to be indexed. The catalogers restrict their job to indexing, refraining from any interpretations or evaluations of the content of the articles.10 The philosophy behind indexing is that the content and format of each item are fully and adequately described, and articles are indexed by their most specific term, with Major MeSH terms representing the main topic of the article and other MeSH terms representing additional concepts.10 Although a standardized procedure for indexing articles in MEDLINE is in place, indexing inconsistencies have been found both for MeSH term assignment3,21–24 and for study-type indexing,25,26 essentially in result of the indexing process being ultimately subject to individual interpretation. This was shown in a report that identified a 48% consistency rate among catalogers in the assignment of main MeSH terms.27 Therefore, it is possible that articles describing the same concept do not possess equivalent MeSH indexing profiles.27 Furthermore, the NLM does not retrospectively re-index citations as new MeSH concepts are created,28 which results in discrepancies not being reconciled. Several scientific areas analyzed the use of MeSH. Portaluppi et al drew attention to the fact that the MeSH thesaurus was inaccurate and incomplete for retrieving chronobiologic references.24 Similarly, Richter et al found that several entry terms commonly used by physical therapists did not map to an appropriate MeSH.29 In the pharmacy field, an imbalance was found in
688
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
MeSH coverage of pharmacy concepts relative to MeSH terminology specific to the nursing and dentistry professions.9 Only 26 pharmacy-specific MeSH terms currently existing in the MeSH thesaurus were identified, as opposed to 94 and 145 nursing- and dentistry-specific, respectively.9 Additionally the inclusion of 17 new MeSH terms based on a systematic and objective method was proposed.9 Considering that the quality of indexing and consistent use of terminology are of major importance to increase the chance of successful literature retrieval and citing,22 the aim of the present study was to assess the quality of assignment of the existing pharmacy-specific MeSH terms to articles indexed in pharmacy journals.
Methods As a means of objectively selecting a set of 10 pharmacy journals to include in the analyses, MEDLINE was searched through PubMed in May 2013 to identify journals containing the highest number of articles published in 2012 indexed under the MeSH ‘Pharmacists’ (search strategy: pharmacists[MESH] AND 2012[DP]). The 10 journals with the highest number of articles indexed under the MeSH ‘Pharmacists’ were identified. A second search was performed in June 2013 to retrieve all articles published over a 5-year period (January 2008 to December 2012) in the 10 previously selected journals. PubMed records of these articles were automatically imported using Endnote and then exported into Microsoft ExcelÒ files. The 5-year timeframe was selected to avoid the impact of the delay in article indexation described elsewhere.30 MeSH terms and Major MeSH terms, those representing the main topic of the article, were isolated from their subheadings and MeSH terms relating to specific pharmacy activities (pharmacy-specific terms) described in previous research9 were identified. The pharmacy-specific MeSH ‘Technology, Pharmaceutical’ included in that study was excluded from the current analysis to improve specificity. The prevalence of the use of general MeSH and pharmacy-specific MeSH terms to index articles in each of the journals analyzed was calculated. Descriptive statistical analyses were carried out, with absolute values and relative frequencies reported for categorical variables and central tendency with dispersion measures being reported for continuous or discrete variables. Normality
was assessed using the Kolmogorov–Smirnov test. Bivariate analysis was performed by applying the Kruskal–Wallis test to compare means and Chisquare test to estimate the association between categorical variables. All statistics were performed using SPSS version 16.
Results The search performed in MEDLINE in May 2013 yielded a total of 667 articles indexed with the MeSH ‘Pharmacists’ and published in 194 different journals during the year of 2012. The mean (standard deviation, SD) number of articles per journal was 3.4 (SD ¼ 8.6), with an asymmetrical distribution where the median value was 1. Of the 667 articles, 21.3% (n ¼ 142) were the only articles published in a given journal in 2012 that were indexed under the MeSH ‘Pharmacists’. The 10 pharmacy journals that contained the highest number of articles indexed with the MeSH ‘Pharmacists,’ and which were therefore included in our analysis, are described in Table 1. Of the total 667 articles, 53.5% (n ¼ 357) were published in the 10 journals selected. Over the 5-year period considered to gather the sample of articles, a total of 6989 articles were retrieved in June 2013 from the 10 pharmacy journals previously identified. The number of articles was evenly distributed throughout the study period, ranging from 1289 in 2008 to 1531 in 2012. The frequency of articles retrieved was unevenly distributed across journals with the Am J Health Syst Pharm and Ann Pharmacother Table 1 Pharmacy journals containing the highest number of articles published in 2012 indexed under the MeSH ‘Pharmacists’ as of May 2013 Pharmacy journal
Number of articles indexed with the MeSH ‘pharmacists’
1. Am J Health Syst Pharm 2. J Am Pharm Assoc (2003) 3. Int J Clin Pharm/Pharm World Sci 4. Int J Pharm Pract 5. Am J Pharm Educ 6. Consult Pharm 7. Res Social Adm Pharm 8. J Pharm Pract 9. Ann Pharmacother 10. Pharmacotherapy
75 44 41 40 39 29 29 26 22 12
2.6 (8.0) 12.1 (10.7) 10.5 (13.9) 612 (86.4) 61 (28.6) 3489 0.2 (0.7) 1.4 (1.1) 0.9 (1.2) 0.6 (0.9) 1.8 (1.6) 1.0 (1.3) 11.4 (4.1) 12.7 (5.0) 10.1 (4.0) 126.0 (57.2) 211.4 (148.4) 133.6 (89.8) 24 (3.3) 35 (14.1) 328 708 (96.7) 213 (85.9) 6661
(14.5) (12.8) (7.2) (11.1) (12.3) (14.2) (11.6) (10.3) 10.5 25.6 2.3 9.0 17.2 16.4 6.6 9.1 (55.2) (3.0) (87.0) (49.5) (21.0) (23.6) (66.0) (43.3) 976 27 1142 138 47 130 140 216 (1.0) (1.0) (0.7) (0.9) (1.0) (1.1) (1.0) (1.1) 0.8 2.3 0.2 0.8 1.5 1.5 0.6 1.1 (1.1) (1.6) (0.9) (1.1) (1.1) (1.4) (1.1) (1.5) 0.9 2.0 0.5 0.9 1.1 1.2 0.8 1.4 (3.2) (3.8) (3.8) (3.8) (3.6) (3.9) (3.8) (4.5) 8.6 9.8 10.3 9.7 9.8 10.5 9.9 12.5 (69.7) (76.9) (64.2) (40.7) (52.3) (45.8) (143.7) (88.6) 108.8 133.1 134.2 47.8 70.5 101.6 271.8 251.6 (0.2) (2.6) (4.5) (18.9) (5.5) (8.9) (12.4) (3.3) 4 24 62 65 13 54 30 17 (99.8) (97.4) (95.5) (81.1) (94.5) (91.1) (87.6) (96.7) 1769 895 1312 279 224 550 212 499
689
Am J Health Syst Pharm 1773 Am J Pharm Educ 919 Ann Pharmacother 1374 Consult Pharm 344 Int J Pharm Pract 237 J Am Pharm Assoc 604 J Pharm Pract 242 Int J Clin Pharm/Pharm 516 World Sci Pharmacotherapy 732 Res Social Adm Pharm 248 Total 6989
Articles with no pharmacy MeSH terms assigned n (%) Mean (SD) Number of pharmacy MeSH terms per article Mean (SD) Number of major MeSH terms per article Mean (SD) Number of MeSH terms per article Mean (SD) time for MeSH assignment (days) Articles with no MeSH terms assigned n (%) Articles Articles retrieved fully indexed over 5 n (%) years (n) Journal
Table 2 Summary of overall results per pharmacy journal
publishing the greatest number of articles (1773 and 1374, respectively) and the Int J Pharm Pract, J Pharm Pract and Res Social Adm Pharm publishing the fewest (237, 242, and 248, respectively). The mean delay for MeSH assignment was 134 days (SD ¼ 90), with a median of 113 days. Delay for MeSH assignment significantly varied across journals (Kruskal–Wallis P ! 0.001), ranging from 272 (SD ¼ 144) or 252 (SD ¼ 88) days for the J Pharm Pract and Int J Clin Pharm/Pharm World Sci, respectively, to 48 (SD ¼ 41) days for Consult Pharm. Of the 6989 articles, 328 (4.7%) had not been fully indexed by NLM catalogers and therefore did not have any MeSH terms assigned. A statistically significant difference in the number of articles without any MeSH terms assigned was found (Chi-square P ! 0.001), with differences ranging from 0.2% in the Am J Health Syst Pharm to 18.9% in Consult Pharm (Table 2). Among the 6661 articles fully indexed, the mean number of MeSH per article was 10.1 (SD ¼ 4.0). The number of MeSH terms per article did not follow a normal distribution (Kolmogorov–Smirnov P ! 0.001), ranging from 1 to 30. A significant difference (Kruskal– Wallis P ! 0.001) in the mean number of MeSH used to index articles was found across journals, which varied from 8.6 (SD ¼ 3.2) in the Am J Health Syst Pharm to 12.7 (SD ¼ 5.0) in Res Social Adm Pharm. The mean (SD) number of Major MeSH terms per article was 1.0 (SD ¼ 1.3) and this number also presented significant differences across journals (Kruskal–Wallis P ! 0.001) varying between 0.5 (SD ¼ 0.9) in the Ann Pharmacother and 2.0 (SD ¼ 1.6) in the Am J Pharm Educ (Table 2). The mean number of pharmacy-specific MeSH terms per article was 0.9 (SD ¼ 1.2). A total of 3490 (52.4%) of the 6661 articles were indexed in pharmacy journals without a single pharmacyspecific MeSH, being Ann Pharmacother and Pharmacotherapy the journals that presented the highest percentage of articles indexed without pharmacy-specific MeSH terms, respectively 87.0% and 86.4%. Of the total number of MeSH terms assigned to articles, the mean percentage of those that were pharmacy-specific was 10.5% (SD ¼ 13.9). A statistically significant difference in the percentage of pharmacy-specific MeSH across journals was identified (Kruskal– Wallis P ! 0.001), with Ann Pharmacother and Pharmacotherapy containing a mean of 2.3% (SD ¼ 7.2) and 2.6% (SD ¼ 8.0), respectively,
Mean (SD) Percentage of pharmacy MeSH per article
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
690
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
Fig. 1. Box plot representing the distribution of the percentage of pharmacy-specific MeSH terms among the articles studied published over a 5-year period across the 10 pharmacy journals.
and the Am J Pharm Educ as much as 25.6% (SD ¼ 12.8) (Table 2 and Fig. 1). Overall, 67193 MeSH terms corresponding to 4835 different MeSH terms were extracted from the 6661 fully indexed articles. The prevalence of MeSH terms (top-20) used to index articles is shown on Table 3. The 5 most prevalent MeSH terms used were: ‘Humans’ (95.4%), ‘Pharmacists’ (23.6%), ‘Middle Aged’ (21.0%), ‘Adult’ (19.5%) and ‘United States’ (17.5%). Of the 67193 MeSH terms retrieved, 6187 (9.2%) were pharmacyspecific terms identified in previous research9 and the 5 most frequently used were: ‘Pharmacists’ (23.6%), ‘Education, Pharmacy’ (13.3%), ‘Students, Pharmacy’ (10.9%), ‘Pharmacy Service, Hospital’ (8.9%) and ‘Pharmaceutical Services’ (8.1%) (Table 4). At the journal level, the prevalence of the use of the broader MeSH ‘Pharmacists’ was 8.6% and 7.8% in Ann Pharmacother and Pharmacotherapy, as opposed to the Int J Pharm Pract, J Am Pharm Assoc, Res Social Adm Pharm and Consult Pharm which was over or nearly 50% (Table 5). Pharmacy-specific MeSH terms less frequently used (under 5%) to index articles across pharmacy journals were: ‘Behind-the-Counter Drugs,’ ‘Clinical Pharmacy Information Systems,’ ‘Drug Compounding,’ ‘Education, Pharmacy, Continuing,’ ‘Ethics, Pharmacy,’ ‘Fees, Pharmaceutical,’ ‘Insurance, Pharmaceutical Services,’ ‘Legislation, Pharmacy,’
‘Licensure, Pharmacy,’ ‘Pharmaceutical Services, Online,’ ‘Pharmacies,’ ‘Pharmacists’ Aides,’ ‘Pharmacy,’ ‘Pharmacy Administration,’ ‘Pharmacy and Therapeutics Committee,’ ‘Schools, Pharmacy’ and ‘Societies, Pharmaceutical’ (Table 5). Of the 895
Table 3 Top-20 most prevalent MeSH terms used to index articles in pharmacy journals MeSH
n (%)
Humans Pharmacists Middle aged Adult United States Aged Education, pharmacy Professional role Students, pharmacy Pharmacy service, hospital Pharmaceutical services Community pharmacy services Retrospective studies Aged, 80 and over Young adult Questionnaires Treatment outcome Curriculum Time factors Adolescent
6351 1574 1396 1298 1168 1164 884 743 729 593 540 515 510 459 442 430 413 412 375 346
(95.4) (23.6) (21.0) (19.5) (17.5) (17.5) (13.3) (11.2) (10.9) (8.9) (8.1) (7.7) (7.7) (6.9) (6.6) (6.5) (6.2) (6.2) (5.6) (5.2)
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695 Table 4 Pharmacy-specific MeSH terms most frequently used to index articles in pharmacy journals Pharm-MeSH
n (%)
Pharmacists Education, pharmacy Students, pharmacy Pharmacy service, hospital Pharmaceutical services Community pharmacy services Schools, pharmacy Societies, pharmaceutical Pharmacies Education, pharmacy, graduate Pharmacy Education, pharmacy, continuing Drug compounding Pharmacists’ aides Insurance, pharmaceutical services Clinical pharmacy information systems Pharmacy administration Legislation, pharmacy Pharmacy and Therapeutics Committee Ethics, pharmacy Licensure, pharmacy Fees, pharmaceutical Behind-the-counter drugs Pharmaceutical services, online
1574 884 729 593 540 515 272 217 155 151 143 86 75 62 57 32 25 21 18 12 12 10 3 1
(23.6) (13.3) (10.9) (8.9) (8.1) (7.7) (4.1) (3.3) (2.3) (2.3) (2.2) (1.3) (1.1) (0.9) (0.9) (0.5) (0.4) (0.3) (0.3) (0.2) (0.2) (0.2) (0.05) (0.02)
articles analyzed from the Am J Pharm Educ, 787 (87.9%) were indexed with one of the education specific MeSH terms (‘Education, Pharmacy,’ ‘Education, Pharmacy, Continuing,’ ‘Education, Pharmacy, Graduate’).
Discussion The aim of the present study was to assess the quality of assignment of the currently existing pharmacy-specific MeSH terms9 to articles published in pharmacy journals. This research presents two main strengths: first, pharmacy journals were selected through an objective method; and secondly, a 5-year period was analyzed as a means of gathering a wide spectrum of articles without being heavily influenced by the yearly MeSH thesaurus update. In addition, articles selected had been included in PubMed for more than 6 months, allowing sufficient time for complete cataloging by the NLM staff.30 During the journal selection process, a wide scattering of pharmacy articles containing the MeSH ‘Pharmacists’ across journals was noted: 667 articles in 194 journals in 2012. About 20% of
691
these articles were published in journals with one single article indexed under the MeSH ‘Pharmacists’ in the year of 2012. This suggests that researchers are publishing pharmacy studies in journals that are not pharmacy-related, therefore contributing to the scattering of the pharmacy literature and the non-creation of a strong body of knowledge, although this may be controversial. One of the main causes for this dispersion may result from an attempt to publish in higher impact factor journals. This study demonstrated that less than half of the articles published in journals with a pharmacy scope were indexed with a pharmacy-specific MeSH and over 20% of these were indexed with the broad MeSH ‘Pharmacists’ which does not specifically reflect the type of research being reported. Being the journals selected for this study the 10 journals with the highest number of articles indexed under the MeSH ‘Pharmacists,’ it would be expected that articles be published with at least one pharmacy-specific MeSH. Although the number of MeSH terms assigned to each article was considered to be within the average,18,19 only about 10% were pharmacy-specific. In addition, less than 10% of the articles indexed with a pharmacy-specific MeSH contained the terms ‘Pharmaceutical Services’ or ‘Community Pharmacy Services’ which, according to the MeSH definitions, refer to the clinical aspects of the pharmacy profession related to the “provision of pharmaceutical services by qualified pharmacists that include consultative services in addition to the preparation and distribution of medical products.” Of note, is that the pharmacy-specific MeSH terms less frequently used (under 5%) were used to index a total of 1201 articles, whereas a previous research showed that over 2000 pharmacy articles, on average, would be indexed under the 17 proposed pharmacy-specific MeSH terms.9 Another example of a potential inappropriateness in MeSH assignment in the pharmacy field is that about 12% (n ¼ 107) of the articles published in the Am J Pharm Educ, whose purpose is “to document and advance pharmaceutical education in the United States and internationally” (as defined by the journal’s editorial board), were not indexed with any of the education specific MeSH terms: ‘Education, Pharmacy,’ ‘Education, Pharmacy, Continuing,’ or ‘Education, Pharmacy, Graduate.’ A similar study on the field of Pharmacoepidemiology/Drug Utilization analyzed the indexation of published articles under the MeSH terms “Pharmacoepidemiology,”
Pharmacy-specific MeSH
Behind-the-counter drugs 0 Clinical pharmacy 20 (1.1) information systems Community pharmacy 18 (1.0) services Drug compounding 45 (2.5) Education, pharmacy 76 (4.3) Education, pharmacy, 9 (0.5) continuing Education, pharmacy, 56 (3.2) graduate Ethics, pharmacy 4 (0.2) Fees, pharmaceutical 3 (0.2) Insurance, pharmaceutical 10 (0.6) services Legislation, pharmacy 8 (0.5) Licensure, pharmacy 0 Pharmaceutical services 108 (6.1) Pharmaceutical services, 1 (0.1) online Pharmacies 13 (0.7) Pharmacists 367 (20.7) Pharmacists’ aides 23 (1.3) Pharmacy 33 (1.9) Pharmacy administration 14 (0.8) Pharmacy service, hospital 370 (20.9) Pharmacy and 5 (0.3) Therapeutics Committee Schools, pharmacy 19 (1.1) Societies, pharmaceutical 89 (5.0) Students, pharmacy 64 (3.6)
0 0
Ann Consult Pharmacother Pharm n (%) n (%)
1 (0.1) 0
0 0
0 1 (0.4)
39 (4.4)
41 (3.1)
10 (3.6)
8 (0.9) 686 (76.6) 20 (2.2)
3 (0.2) 16 (1.2) 4 (0.3)
1 (0.4) 2 (0.7) 0
82 (9.2)
1 (0.1)
5 (0.6) 0 3 (0.3) 6 (0.7) 6 (0.7) 93 (10.4) 0 14 175 3 38 5 28 7
J Am Pharm Assoc n (%) 2 (0.4) 3 (0.5)
66 (29.5) 137 (24.9)
J Pharm Int J Pharmacotherapy Res Pract Clin n (%) Social n (%) Pharm/Pharm Adm World Sci Pharm n (%) n (%) 0 0
0 6 (1.2)
0 0
0 2 (0.9)
5 (0.7)
70 (32.9)
515 (7.7)
1 (0.5) 8 (3.8) 3 (1.4)
75 (1.1) 884 (13.3) 86 (1.3)
0
151 (2.3)
3 (1.4) 11 (5.2) 6 (2.8)
3 (0.6) 13 (2.6) 14 (2.8)
1 (0.1) 20 (2.8) 1 (0.1)
0
1 (0.4)
5 (0.9)
1 (0.5)
1 (0.2)
4 (0.6)
0 0 7 (0.5)
0 0 3 (1.1)
0 0 1 (0.4)
0 4 (0.7) 11 (2.0)
0 0 1 (0.5)
2 (0.4) 2 (0.4) 1 (0.2)
1 (0.1) 1 (0.1) 4 (0.6)
0 0 16 (7.5)
0 0 38 (2.9) 0
0 0 51 (18.3) 0
3 (1.4) 1 (0.5) 6 (2.8) 0
0 0 35 (7.0) 0
0 0 21 (3.0%) 0
0 0 32 (15.0) 0
6 (0.5) 4 (0.3) 11 (0.8)
3 120 1 5 0 2 0
(1.1) 10 (4.5) 42 (7.6) 7 (3.3) 34 (6.8) (43.0) 125 (55.8) 277 (50.4) 53 (25.0) 185 (37.1) (0.4) 7 (3.1) 16 (2.9) 0 3 (0.6) (1.8) 3 (1.3) 32 (5.8) 4 (1.9) 8 (1.6) 1 (0.4) 1 (0.2) 0 0 (0.7) 31 (13.8) 16 (2.9) 14 (6.6) 76 (15.2) 0 0 0 5 (1.0)
0 12 (4.3) 4 (1.4)
3 (1.3) 2 (0.9) 12 (5.4)
11 (2.0) 51 (9.3) 45 (8.2)
3 (0.05) 32 (0.5)
125 (25.1)
8 (1.5) 39 (7.1) 18 (3.3)
1 (0.4) 3 (0.5) 1 (0.4) 4 (0.7) 50 (22.3) 106 (19.3) 0 0
Total n (%)
4 (1.9)
2 (0.9) 13 (5.8) 11 (4.9)
(1.6) 10 (0.8) (19.6) 113 (8.6) (0.3) 3 (0.2) (4.2) 7 (0.5) (0.6) 0 (3.1) 27 (2.1) (0.8) 1 (0.1)
223 (24.9) 38 (4.2) 568 (63.5)
Int J Pharm Pract n (%)
4 (1.9) 3 (1.4) 8 (3.8)
0 3 (0.6) 7 (1.4)
1 (0.1) 55 (7.8) 0 7 (1.0) 1 (0.1) 18 (2.5) 0
3 (0.4) 14 (2.0) 4 (0.6)
21 104 6 6 3 11 0
12 (0.2) 10 (0.2) 57 (0.9) 21 12 540 1
(0.3) (0.2) (8.1) (0.02)
(9.9) 155 (2.3) (48.8) 1574 (23.6) (2.8) 62 (0.9) (2.8) 143 (2.1) (1.4) 25 (0.4) (5.2) 593 (8.9) 18 (0.3)
3 (1.4) 1 (0.5) 6 (2.8)
272 (4.1) 217 (3.3) 729 (10.9)
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
Am J Health Am J Syst Pharm Pharm n (%) Educ n (%)
692
Table 5 Prevalence of pharmacy-specific MeSH terms used to index articles in each of the pharmacy journals analyzed
9.2% 10.8% 2.0% 8.4% 6.1% 14.4% 15.5% 7.9% 23.3% 9.0%
2.2%
8775 2047
Total MeSH Total pharmacy-specific MeSH Percentage of pharmacyspecific MeSH
15139 1355
13500 293
2711 214
2201 341
5757 831
2107 129
6234 523
8061 161
2708 293
67193 6187
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
693
“Drug Utilization” and “Drug Utilization Review.”17 The authors found that the journal Pharmacoepidemiol Drug Saf, which is specifically devoted to this field, presented only 47.5% of its articles included under the studied MeSH terms.17 Inaccurate indexing of articles may compromise the search of the literature for concepts rather than text words, therefore not accounting for variability in language which is one of the advantages of using MeSH terms. In addition to the utility of MeSH as a way to facilitate literature retrieval and to refine search strategies,11 the analysis of MeSH assignment can serve other purposes. Based on the present analysis different MeSH assignment patterns have been identified across the 10 journals. A distinction in the journals’ scope was objectively noted, with some being more pharmacotherapy-focused and containing a low proportion of pharmacyspecific MeSH terms in indexed articles (Ann Pharmacother and Pharmacotherapy), while others were more pharmacy practice-oriented and articles were indexed with a higher proportion of pharmacy-specific MeSH terms (Am J Pharm Educ, Int J Pharm Pract or J Am Pharm Assoc). A third category of journals presented a broader focus (Am J Health Syst Pharm, Consult Pharm, J Pharm Pract, Pharm World & Sci/Int J Clin Pharm, Res Soc Adm Pharm). These findings contrast with what has been claimed by some authors when justifying the choice of only the journals Am J Health Syst Pharm, Ann Pharmacother and Pharmacotherapy to analyze MeSH assignment delay: they considered that these journals were representative of pharmacy practice literature.30 These findings suggest that the categories used to group journals, such as the Subject Categories created for the Journal Citation Reports, should be analyzed on the basis of objective criteria such as the ones employed in this study. Assuming that MeSH terms were accurately assigned during article indexation, the analysis of MeSH would also enable the identification of specific areas within a given field through the recognition of “sentinel” MeSH. This would be of great value in guiding authors to submit their works to journals which targeted a specific area better than others. On the other hand, analyzing MeSH could shed some light into areas that are currently under-researched, specifically in this case: ethics in pharmacy, the role of Pharmacy and Therapeutics Committees, or pharmacy legislation. Likewise, the creation of MeSH terms within a specific discipline has also been studied
694
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
as a way of predicting new topic trends.31 Additionally, recent authors developed a robust and unbiased expert finding system based on the use of MeSH to quantitatively measure expertise.32 This could be used as a means of finding experts in a given subject to serve as peer-reviewers for journals or finding subjects associated with an expert.32 This study presents some limitations. First of all, the analysis included only 10 of the existing pharmacy journals. Nevertheless, these 10 journals included more than 50% of the articles published and indexed with the MeSH ‘Pharmacists.’ Secondly, articles indexed with descendant MeSH of pharmacy-specific MeSH terms in the MeSH tree hierarchy were not considered in the analysis. This means that we might have missed articles that could have been retrieved by automatic explosion (i.e., the process by which MeSH terms are automatically expanded to enable the retrieval of not only citations that carry the specified MeSH but also citations that carry any of the more specific MeSH terms, or descendants, included beneath the parent MeSH in the tree structure).33 However, some of the child MeSH terms that have a pharmacy-specific parent do not correspond to activities exclusively performed by pharmacists as defined in a previous study.9
Conclusions The findings of this study suggest that there is room for improvement with regards to the quality of assignment of the existing pharmacy-specific MeSH terms to articles indexed in pharmacy journals by the NLM catalogers. To date, over half of the articles published in the top-10 journals publishing pharmacy literature were indexed without a single pharmacy-specific MeSH. Significant differences in pharmacy-specific MeSH assignment patterns were found across journals suggesting distinct journals’ scopes. References 1. Doig GS, Simpson F. Efficient literature searching: a core skill for the practice of evidence-based medicine. Intensive Care Med 2003;29:2119–2127. 2. Young S, Duffull SB. A learning-based approach for performing an in-depth literature search using ME DLINE. J Clin Pharm Ther 2011;36:504–512. 3. Lawrence DW. Controlled search term vocabularies for finding articles relevant to injury prevention and safety promotion. Inj Prev 2011;17:260–265.
4. Meneghini R, Packer AL, Nassi-Calo L. Articles by Latin American authors in prestigious journals have fewer citations. PLoS One 2008;3:e3804. 5. Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ. Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital. BMC Med Inform Decis Mak 2008;8:42. 6. The National Library of Medicine. Pubmed Home Page, 2014. [cited 03-Nov-2014]; Available from: http://www.ncbi.nlm.nih.gov/pubmed. 7. The National Library of Medicine. Fact Sheet: Medical Subject Headings (MeSHÒ), 2014. [cited 03Nov-2014]; Available from: http://www.nlm.nih. gov/pubs/factsheets/mesh.html. 8. Lipscomb CE. Medical Subject Headings (MeSH). Bull Med Libr Assoc 2000;88:265–266. 9. Minguet F, Van Den Boogerd L, Salgado TM, Correr CJ, Fernandez-Llimos F. Characterization of the Medical Subject Headings thesaurus for pharmacy. Am J Health Syst Pharm 2014;71:1965–1972. 10. The National Library of Medicine. Principles of MEDLINE Subject Indexing, 2014. [cited 03-Nov-2014]; Available from: http://www.nlm.nih.gov/bsd/disted/ meshtutorial/principlesofmedlinesubjectindexing/ principles/index.html. 11. Shultz M. Comparing test searches in PubMed and Google Scholar. J Med Libr Assoc 2007;95:442–445. 12. Jenuwine ES, Floyd JA. Comparison of Medical Subject Headings and text-word searches in MEDLINE to retrieve studies on sleep in healthy individuals. J Med Libr Assoc 2004;92:349–353. 13. Rana GK, Bradley DR, Hamstra SJ, et al. A validated search assessment tool: assessing practicebased learning and improvement in a residency program. J Med Libr Assoc 2011;99:77–81. 14. Jia X, Guo X, Li H, An X, Zhao Y. Characteristics and popular topics of latest researches into the effects of air particulate matter on cardiovascular system by bibliometric analysis. Inhal Toxicol 2013;25:211–218. 15. Zheng HC, Yan L, Cui L, Guan YF, Takano Y. Mapping the history and current situation of research on John Cunningham virus – a bibliometric analysis. BMC Infect Dis 2009;9:28. 16. Li HY, Cui L, Cui M. Hot topics in Chinese herbal drugs research documented in PubMed/MEDLINE by authors inside China and outside of China in the past 10 years: based on co-word cluster analysis. J Altern Complement Med 2009;15:779–785. 17. Sanz EJ. Pharmacoepidemiology and the “impact factor”. Eur J Clin Pharmacol 2005;60:765–772. 18. de Granda Orive JI, Garcia Rio F, Roig Vazquez F, Escobar Sacristan J, Gutierrez Jimenez T, Callol Sanchez L. Key words, essential tools for bibliographic research: analysis of usage in Archivos de Bronconeumologia for respiratory system knowledge areas. Arch Bronconeumol 2005;41:78–83. 19. Allison JJ, Kiefe CI, Weissman NW, Carter J, Centor RM. The art and science of searching
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
20.
21.
22.
23.
24.
25.
MEDLINE to answer clinical questions. Finding the right number of articles. Int J Technol Assess Health Care 1999;15:281–296. The National Library of Medicine. Depth of Indexing and the Rule of Three, 2014. [cited 03-Nov-2014]; Available from: http://www.nlm.nih.gov/bsd/disted/ meshtutorial/principlesofmedlinesubjectindexing/dept hofindexing/02.html. Ugolini D, Neri M, Knudsen LE, Bonassi S, Merlo DF. Searching PubMed for molecular epidemiology studies: the case of chromosome aberrations. Environ Mol Mutagen 2006;47:227–229. Sogaard M, Andersen JP, Schonheyder HC. Searching PubMed for studies on bacteremia, bloodstream infection, septicemia, or whatever the best term is: a note of caution. Am J Infect Control 2012;40:237–240. Mavropoulos A, Kiliaridis S. Orthodontic literature: an overview of the last 2 decades. Am J Orthod Dentofacial Orthop 2003;124:30–40. Portaluppi F. Consistency and accuracy of the Medical Subject Headings thesaurus for electronic indexing and retrieval of chronobiologic references. Chronobiol Int 2007;24:1213–1229. Wilczynski NL, Haynes RB. Consistency and accuracy of indexing systematic review articles and meta-analyses in Medline. Health Info Libr J 2009; 26:203–210.
695
26. Sampson M, Horsley T, Doja A. A bibliometric analysis of evaluative medical education studies: characteristics and indexing accuracy. Acad Med 2013;88: 421–427. 27. Funk ME, Reid CA. Indexing consistency in MEDLINE. Bull Med Libr Assoc 1983;71:176–183. 28. The National Library of Medicine. Frequently Asked Questions about Indexing for MEDLINE, 2014. [cited 03-Nov-2014]; Available from: http://www.nlm.nih. gov/bsd/indexfaq.html. 29. Richter RR, Austin TM. Using MeSH (medical subject headings) to enhance PubMed search strategies for evidence-based practice in physical therapy. Phys Ther 2012;92:124–132. 30. Rodriguez RW. Delay in indexing articles published in major pharmacy practice journals. Am J Health Syst Pharm 2014;71:321–324. 31. Moerchen F, Fradkin D, Dejori M, Wachmann B. Emerging trend prediction in biomedical literature. AMIA Annu Symp Proc; 2008::485–489. 32. Singh H, Singh R, Malhotra A, Kaur M. Developing a biomedical expert finding system using medical subject headings. Healthc Inform Res 2013;19:243–249. 33. The National Library of Medicine. Glossary, 2013. [cited 03-Nov-2014]; Available from: http://www.nlm. nih.gov/bsd/disted/pubmedtutorial/glossary.html.
Original Research
Quality of pharmacy-specific Medical Subject Headings (MeSH) assignment in pharmacy journals indexed in MEDLINE Fernando Minguet, M.Sc., Pharm.D.a, Teresa M. Salgado, Ph.D.b, Lucienne van den Boogerd, Pharm.D.c, Fernando Fernandez-Llimos, Ph.D., M.B.A.d,* a Valencian Pharmacy Practice Research Group, Valencia, Spain Department of Clinical, Social, and Administrative Sciences, College of Pharmacy, University of Michigan, MI, USA c Department of Pharmaceutical Sciences, Utrecht University, Utrecht, The Netherlands d Research Institute for Medicines (iMed.ULisboa), Department of Social-Pharmacy, Faculty of Pharmacy, University of Lisbon, Lisbon, Portugal b
Abstract Background: The Medical Subject Headings (MeSH) is the National Library of Medicine (NLM) controlled vocabulary for indexing articles. Inaccuracies in the MeSH thesaurus have been reported for several areas including pharmacy. Objectives: To assess the quality of pharmacy-specific MeSH assignment to articles indexed in pharmacy journals. Methods: The 10 journals containing the highest number of articles published in 2012 indexed under the MeSH ‘Pharmacists’ were identified. All articles published over a 5-year period (2008–2012) in the 10 previously selected journals were retrieved from PubMed. MeSH terms used to index these articles were extracted and pharmacy-specific MeSH terms were identified. The frequency of use of pharmacy-specific MeSH terms was calculated across journals. Results: A total of 6989 articles were retrieved from the 10 pharmacy journals, of which 328 (4.7%) were articles not fully indexed and therefore did not contain any MeSH terms assigned. Among the 6661 articles fully indexed, the mean number of MeSH terms was 10.1 (SD ¼ 4.0), being 1.0 (SD ¼ 1.3) considered as Major MeSH. Both values significantly varied across journals. The mean number of pharmacy-specific MeSH terms per article was 0.9 (SD ¼ 1.2). A total of 3490 (52.4%) of the 6661 articles were indexed in pharmacy journals without a single pharmacy-specific MeSH. Of the total 67193 MeSH terms assigned to articles, on average 10.5% (SD ¼ 13.9) were pharmacy-specific MeSH. A statistically significant different pattern of pharmacy-specific MeSH assignment was identified across journals (Kruskal–Wallis P ! 0.001). Conclusions: The quality of assignment of the existing pharmacy-specific MeSH terms to articles indexed in pharmacy journals can be improved to further enhance evidence gathering in pharmacy. Over half of the
Conflicts of interest disclosure: None to declare. * Corresponding author. Departamento de Socio-Farma´cia, Faculdade de Farmacia, Universidade de Lisboa, Av. Prof. Gama Pinto, 1649-003 Lisboa, Portugal. Tel.: þ351 217 946 400; fax: þ351 217 946 470. E-mail address: f-llimos@ff.ul.pt (F. Fernandez-Llimos). 1551-7411/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.sapharm.2014.11.004
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
687
articles published in the top-10 journals publishing pharmacy literature were indexed without a single pharmacy-specific MeSH. Ó 2015 Elsevier Inc. All rights reserved. Keywords: Evidence-based practice; Medical subject headings; Periodicals as topic; Pharmaceutical services; Pharmacists; Pharmacy
Introduction Searching the literature when attempting to answer a given clinical or research question can sometimes be a difficult process, as the many existing databases nowadays provide access to millions of records. An efficient literature search is an essential skill for evidence-based practice1 and searching in electronic databases offers the most effective use of time and the greatest potential yield.2 To facilitate article retrieval, databases adopt controlled vocabularies as a means of cataloging articles. An appropriate assignment of a controlled vocabulary defines the quality of an information retrieval system.3 Therefore, problems with controlled vocabulary assignment can hinder an article’s diffusion or even cause it to fall into oblivion.4 Common user errors in database searches stem from inappropriate selection of search terms.5 MEDLINE is the National Library of Medicine’s (NLM) journal citation database and contains citations and abstracts for biomedical literature from around the world. PubMed is a free available search engine that provides access to MEDLINE, among other databases, and it contains over 23 million citations.6 PubMed uses the NLM’s controlled vocabulary thesaurus – the Medical Subject Headings (MeSH) – for indexing and cataloging articles.7,8 Although criticized for not appropriately covering some areas, such as pharmacy,9 the MeSH database is continuously updated and revised annually.7 The intent of using the MeSH terminology is to facilitate search retrieval by eliminating (or accounting for) the use of variant terminology for the same concept.10 MeSH indexing facilitates literature retrieval as it provides a powerful method of narrowing search results.11 In fact, literature searches performed using MeSH terms were shown to retrieve a fewer number of irrelevant citations.12 However, knowledge and appropriate use of MeSH are critical elements to conduct an effective MEDLINE search.13 Besides providing a consistent way of retrieving information, MeSH can also be
indicative of popular topics within a scientific area.14–16 Articles indexed under the same set of MeSH terms should refer to the same research field.17 During the indexing process in PubMed, articles are assigned an average of 10 MeSH terms by professional catalogers to reflect their content.18,19 The amount of MeSH terms assigned to one article represents the depth of indexing.20 NLM’s MEDLINE catalogers use the MeSH Browser, an online vocabulary look-up aid with virtually complete MeSH records, to find the term that best describes the concept to be indexed. The catalogers restrict their job to indexing, refraining from any interpretations or evaluations of the content of the articles.10 The philosophy behind indexing is that the content and format of each item are fully and adequately described, and articles are indexed by their most specific term, with Major MeSH terms representing the main topic of the article and other MeSH terms representing additional concepts.10 Although a standardized procedure for indexing articles in MEDLINE is in place, indexing inconsistencies have been found both for MeSH term assignment3,21–24 and for study-type indexing,25,26 essentially in result of the indexing process being ultimately subject to individual interpretation. This was shown in a report that identified a 48% consistency rate among catalogers in the assignment of main MeSH terms.27 Therefore, it is possible that articles describing the same concept do not possess equivalent MeSH indexing profiles.27 Furthermore, the NLM does not retrospectively re-index citations as new MeSH concepts are created,28 which results in discrepancies not being reconciled. Several scientific areas analyzed the use of MeSH. Portaluppi et al drew attention to the fact that the MeSH thesaurus was inaccurate and incomplete for retrieving chronobiologic references.24 Similarly, Richter et al found that several entry terms commonly used by physical therapists did not map to an appropriate MeSH.29 In the pharmacy field, an imbalance was found in
688
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
MeSH coverage of pharmacy concepts relative to MeSH terminology specific to the nursing and dentistry professions.9 Only 26 pharmacy-specific MeSH terms currently existing in the MeSH thesaurus were identified, as opposed to 94 and 145 nursing- and dentistry-specific, respectively.9 Additionally the inclusion of 17 new MeSH terms based on a systematic and objective method was proposed.9 Considering that the quality of indexing and consistent use of terminology are of major importance to increase the chance of successful literature retrieval and citing,22 the aim of the present study was to assess the quality of assignment of the existing pharmacy-specific MeSH terms to articles indexed in pharmacy journals.
Methods As a means of objectively selecting a set of 10 pharmacy journals to include in the analyses, MEDLINE was searched through PubMed in May 2013 to identify journals containing the highest number of articles published in 2012 indexed under the MeSH ‘Pharmacists’ (search strategy: pharmacists[MESH] AND 2012[DP]). The 10 journals with the highest number of articles indexed under the MeSH ‘Pharmacists’ were identified. A second search was performed in June 2013 to retrieve all articles published over a 5-year period (January 2008 to December 2012) in the 10 previously selected journals. PubMed records of these articles were automatically imported using Endnote and then exported into Microsoft ExcelÒ files. The 5-year timeframe was selected to avoid the impact of the delay in article indexation described elsewhere.30 MeSH terms and Major MeSH terms, those representing the main topic of the article, were isolated from their subheadings and MeSH terms relating to specific pharmacy activities (pharmacy-specific terms) described in previous research9 were identified. The pharmacy-specific MeSH ‘Technology, Pharmaceutical’ included in that study was excluded from the current analysis to improve specificity. The prevalence of the use of general MeSH and pharmacy-specific MeSH terms to index articles in each of the journals analyzed was calculated. Descriptive statistical analyses were carried out, with absolute values and relative frequencies reported for categorical variables and central tendency with dispersion measures being reported for continuous or discrete variables. Normality
was assessed using the Kolmogorov–Smirnov test. Bivariate analysis was performed by applying the Kruskal–Wallis test to compare means and Chisquare test to estimate the association between categorical variables. All statistics were performed using SPSS version 16.
Results The search performed in MEDLINE in May 2013 yielded a total of 667 articles indexed with the MeSH ‘Pharmacists’ and published in 194 different journals during the year of 2012. The mean (standard deviation, SD) number of articles per journal was 3.4 (SD ¼ 8.6), with an asymmetrical distribution where the median value was 1. Of the 667 articles, 21.3% (n ¼ 142) were the only articles published in a given journal in 2012 that were indexed under the MeSH ‘Pharmacists’. The 10 pharmacy journals that contained the highest number of articles indexed with the MeSH ‘Pharmacists,’ and which were therefore included in our analysis, are described in Table 1. Of the total 667 articles, 53.5% (n ¼ 357) were published in the 10 journals selected. Over the 5-year period considered to gather the sample of articles, a total of 6989 articles were retrieved in June 2013 from the 10 pharmacy journals previously identified. The number of articles was evenly distributed throughout the study period, ranging from 1289 in 2008 to 1531 in 2012. The frequency of articles retrieved was unevenly distributed across journals with the Am J Health Syst Pharm and Ann Pharmacother Table 1 Pharmacy journals containing the highest number of articles published in 2012 indexed under the MeSH ‘Pharmacists’ as of May 2013 Pharmacy journal
Number of articles indexed with the MeSH ‘pharmacists’
1. Am J Health Syst Pharm 2. J Am Pharm Assoc (2003) 3. Int J Clin Pharm/Pharm World Sci 4. Int J Pharm Pract 5. Am J Pharm Educ 6. Consult Pharm 7. Res Social Adm Pharm 8. J Pharm Pract 9. Ann Pharmacother 10. Pharmacotherapy
75 44 41 40 39 29 29 26 22 12
2.6 (8.0) 12.1 (10.7) 10.5 (13.9) 612 (86.4) 61 (28.6) 3489 0.2 (0.7) 1.4 (1.1) 0.9 (1.2) 0.6 (0.9) 1.8 (1.6) 1.0 (1.3) 11.4 (4.1) 12.7 (5.0) 10.1 (4.0) 126.0 (57.2) 211.4 (148.4) 133.6 (89.8) 24 (3.3) 35 (14.1) 328 708 (96.7) 213 (85.9) 6661
(14.5) (12.8) (7.2) (11.1) (12.3) (14.2) (11.6) (10.3) 10.5 25.6 2.3 9.0 17.2 16.4 6.6 9.1 (55.2) (3.0) (87.0) (49.5) (21.0) (23.6) (66.0) (43.3) 976 27 1142 138 47 130 140 216 (1.0) (1.0) (0.7) (0.9) (1.0) (1.1) (1.0) (1.1) 0.8 2.3 0.2 0.8 1.5 1.5 0.6 1.1 (1.1) (1.6) (0.9) (1.1) (1.1) (1.4) (1.1) (1.5) 0.9 2.0 0.5 0.9 1.1 1.2 0.8 1.4 (3.2) (3.8) (3.8) (3.8) (3.6) (3.9) (3.8) (4.5) 8.6 9.8 10.3 9.7 9.8 10.5 9.9 12.5 (69.7) (76.9) (64.2) (40.7) (52.3) (45.8) (143.7) (88.6) 108.8 133.1 134.2 47.8 70.5 101.6 271.8 251.6 (0.2) (2.6) (4.5) (18.9) (5.5) (8.9) (12.4) (3.3) 4 24 62 65 13 54 30 17 (99.8) (97.4) (95.5) (81.1) (94.5) (91.1) (87.6) (96.7) 1769 895 1312 279 224 550 212 499
689
Am J Health Syst Pharm 1773 Am J Pharm Educ 919 Ann Pharmacother 1374 Consult Pharm 344 Int J Pharm Pract 237 J Am Pharm Assoc 604 J Pharm Pract 242 Int J Clin Pharm/Pharm 516 World Sci Pharmacotherapy 732 Res Social Adm Pharm 248 Total 6989
Articles with no pharmacy MeSH terms assigned n (%) Mean (SD) Number of pharmacy MeSH terms per article Mean (SD) Number of major MeSH terms per article Mean (SD) Number of MeSH terms per article Mean (SD) time for MeSH assignment (days) Articles with no MeSH terms assigned n (%) Articles Articles retrieved fully indexed over 5 n (%) years (n) Journal
Table 2 Summary of overall results per pharmacy journal
publishing the greatest number of articles (1773 and 1374, respectively) and the Int J Pharm Pract, J Pharm Pract and Res Social Adm Pharm publishing the fewest (237, 242, and 248, respectively). The mean delay for MeSH assignment was 134 days (SD ¼ 90), with a median of 113 days. Delay for MeSH assignment significantly varied across journals (Kruskal–Wallis P ! 0.001), ranging from 272 (SD ¼ 144) or 252 (SD ¼ 88) days for the J Pharm Pract and Int J Clin Pharm/Pharm World Sci, respectively, to 48 (SD ¼ 41) days for Consult Pharm. Of the 6989 articles, 328 (4.7%) had not been fully indexed by NLM catalogers and therefore did not have any MeSH terms assigned. A statistically significant difference in the number of articles without any MeSH terms assigned was found (Chi-square P ! 0.001), with differences ranging from 0.2% in the Am J Health Syst Pharm to 18.9% in Consult Pharm (Table 2). Among the 6661 articles fully indexed, the mean number of MeSH per article was 10.1 (SD ¼ 4.0). The number of MeSH terms per article did not follow a normal distribution (Kolmogorov–Smirnov P ! 0.001), ranging from 1 to 30. A significant difference (Kruskal– Wallis P ! 0.001) in the mean number of MeSH used to index articles was found across journals, which varied from 8.6 (SD ¼ 3.2) in the Am J Health Syst Pharm to 12.7 (SD ¼ 5.0) in Res Social Adm Pharm. The mean (SD) number of Major MeSH terms per article was 1.0 (SD ¼ 1.3) and this number also presented significant differences across journals (Kruskal–Wallis P ! 0.001) varying between 0.5 (SD ¼ 0.9) in the Ann Pharmacother and 2.0 (SD ¼ 1.6) in the Am J Pharm Educ (Table 2). The mean number of pharmacy-specific MeSH terms per article was 0.9 (SD ¼ 1.2). A total of 3490 (52.4%) of the 6661 articles were indexed in pharmacy journals without a single pharmacyspecific MeSH, being Ann Pharmacother and Pharmacotherapy the journals that presented the highest percentage of articles indexed without pharmacy-specific MeSH terms, respectively 87.0% and 86.4%. Of the total number of MeSH terms assigned to articles, the mean percentage of those that were pharmacy-specific was 10.5% (SD ¼ 13.9). A statistically significant difference in the percentage of pharmacy-specific MeSH across journals was identified (Kruskal– Wallis P ! 0.001), with Ann Pharmacother and Pharmacotherapy containing a mean of 2.3% (SD ¼ 7.2) and 2.6% (SD ¼ 8.0), respectively,
Mean (SD) Percentage of pharmacy MeSH per article
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
690
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
Fig. 1. Box plot representing the distribution of the percentage of pharmacy-specific MeSH terms among the articles studied published over a 5-year period across the 10 pharmacy journals.
and the Am J Pharm Educ as much as 25.6% (SD ¼ 12.8) (Table 2 and Fig. 1). Overall, 67193 MeSH terms corresponding to 4835 different MeSH terms were extracted from the 6661 fully indexed articles. The prevalence of MeSH terms (top-20) used to index articles is shown on Table 3. The 5 most prevalent MeSH terms used were: ‘Humans’ (95.4%), ‘Pharmacists’ (23.6%), ‘Middle Aged’ (21.0%), ‘Adult’ (19.5%) and ‘United States’ (17.5%). Of the 67193 MeSH terms retrieved, 6187 (9.2%) were pharmacyspecific terms identified in previous research9 and the 5 most frequently used were: ‘Pharmacists’ (23.6%), ‘Education, Pharmacy’ (13.3%), ‘Students, Pharmacy’ (10.9%), ‘Pharmacy Service, Hospital’ (8.9%) and ‘Pharmaceutical Services’ (8.1%) (Table 4). At the journal level, the prevalence of the use of the broader MeSH ‘Pharmacists’ was 8.6% and 7.8% in Ann Pharmacother and Pharmacotherapy, as opposed to the Int J Pharm Pract, J Am Pharm Assoc, Res Social Adm Pharm and Consult Pharm which was over or nearly 50% (Table 5). Pharmacy-specific MeSH terms less frequently used (under 5%) to index articles across pharmacy journals were: ‘Behind-the-Counter Drugs,’ ‘Clinical Pharmacy Information Systems,’ ‘Drug Compounding,’ ‘Education, Pharmacy, Continuing,’ ‘Ethics, Pharmacy,’ ‘Fees, Pharmaceutical,’ ‘Insurance, Pharmaceutical Services,’ ‘Legislation, Pharmacy,’
‘Licensure, Pharmacy,’ ‘Pharmaceutical Services, Online,’ ‘Pharmacies,’ ‘Pharmacists’ Aides,’ ‘Pharmacy,’ ‘Pharmacy Administration,’ ‘Pharmacy and Therapeutics Committee,’ ‘Schools, Pharmacy’ and ‘Societies, Pharmaceutical’ (Table 5). Of the 895
Table 3 Top-20 most prevalent MeSH terms used to index articles in pharmacy journals MeSH
n (%)
Humans Pharmacists Middle aged Adult United States Aged Education, pharmacy Professional role Students, pharmacy Pharmacy service, hospital Pharmaceutical services Community pharmacy services Retrospective studies Aged, 80 and over Young adult Questionnaires Treatment outcome Curriculum Time factors Adolescent
6351 1574 1396 1298 1168 1164 884 743 729 593 540 515 510 459 442 430 413 412 375 346
(95.4) (23.6) (21.0) (19.5) (17.5) (17.5) (13.3) (11.2) (10.9) (8.9) (8.1) (7.7) (7.7) (6.9) (6.6) (6.5) (6.2) (6.2) (5.6) (5.2)
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695 Table 4 Pharmacy-specific MeSH terms most frequently used to index articles in pharmacy journals Pharm-MeSH
n (%)
Pharmacists Education, pharmacy Students, pharmacy Pharmacy service, hospital Pharmaceutical services Community pharmacy services Schools, pharmacy Societies, pharmaceutical Pharmacies Education, pharmacy, graduate Pharmacy Education, pharmacy, continuing Drug compounding Pharmacists’ aides Insurance, pharmaceutical services Clinical pharmacy information systems Pharmacy administration Legislation, pharmacy Pharmacy and Therapeutics Committee Ethics, pharmacy Licensure, pharmacy Fees, pharmaceutical Behind-the-counter drugs Pharmaceutical services, online
1574 884 729 593 540 515 272 217 155 151 143 86 75 62 57 32 25 21 18 12 12 10 3 1
(23.6) (13.3) (10.9) (8.9) (8.1) (7.7) (4.1) (3.3) (2.3) (2.3) (2.2) (1.3) (1.1) (0.9) (0.9) (0.5) (0.4) (0.3) (0.3) (0.2) (0.2) (0.2) (0.05) (0.02)
articles analyzed from the Am J Pharm Educ, 787 (87.9%) were indexed with one of the education specific MeSH terms (‘Education, Pharmacy,’ ‘Education, Pharmacy, Continuing,’ ‘Education, Pharmacy, Graduate’).
Discussion The aim of the present study was to assess the quality of assignment of the currently existing pharmacy-specific MeSH terms9 to articles published in pharmacy journals. This research presents two main strengths: first, pharmacy journals were selected through an objective method; and secondly, a 5-year period was analyzed as a means of gathering a wide spectrum of articles without being heavily influenced by the yearly MeSH thesaurus update. In addition, articles selected had been included in PubMed for more than 6 months, allowing sufficient time for complete cataloging by the NLM staff.30 During the journal selection process, a wide scattering of pharmacy articles containing the MeSH ‘Pharmacists’ across journals was noted: 667 articles in 194 journals in 2012. About 20% of
691
these articles were published in journals with one single article indexed under the MeSH ‘Pharmacists’ in the year of 2012. This suggests that researchers are publishing pharmacy studies in journals that are not pharmacy-related, therefore contributing to the scattering of the pharmacy literature and the non-creation of a strong body of knowledge, although this may be controversial. One of the main causes for this dispersion may result from an attempt to publish in higher impact factor journals. This study demonstrated that less than half of the articles published in journals with a pharmacy scope were indexed with a pharmacy-specific MeSH and over 20% of these were indexed with the broad MeSH ‘Pharmacists’ which does not specifically reflect the type of research being reported. Being the journals selected for this study the 10 journals with the highest number of articles indexed under the MeSH ‘Pharmacists,’ it would be expected that articles be published with at least one pharmacy-specific MeSH. Although the number of MeSH terms assigned to each article was considered to be within the average,18,19 only about 10% were pharmacy-specific. In addition, less than 10% of the articles indexed with a pharmacy-specific MeSH contained the terms ‘Pharmaceutical Services’ or ‘Community Pharmacy Services’ which, according to the MeSH definitions, refer to the clinical aspects of the pharmacy profession related to the “provision of pharmaceutical services by qualified pharmacists that include consultative services in addition to the preparation and distribution of medical products.” Of note, is that the pharmacy-specific MeSH terms less frequently used (under 5%) were used to index a total of 1201 articles, whereas a previous research showed that over 2000 pharmacy articles, on average, would be indexed under the 17 proposed pharmacy-specific MeSH terms.9 Another example of a potential inappropriateness in MeSH assignment in the pharmacy field is that about 12% (n ¼ 107) of the articles published in the Am J Pharm Educ, whose purpose is “to document and advance pharmaceutical education in the United States and internationally” (as defined by the journal’s editorial board), were not indexed with any of the education specific MeSH terms: ‘Education, Pharmacy,’ ‘Education, Pharmacy, Continuing,’ or ‘Education, Pharmacy, Graduate.’ A similar study on the field of Pharmacoepidemiology/Drug Utilization analyzed the indexation of published articles under the MeSH terms “Pharmacoepidemiology,”
Pharmacy-specific MeSH
Behind-the-counter drugs 0 Clinical pharmacy 20 (1.1) information systems Community pharmacy 18 (1.0) services Drug compounding 45 (2.5) Education, pharmacy 76 (4.3) Education, pharmacy, 9 (0.5) continuing Education, pharmacy, 56 (3.2) graduate Ethics, pharmacy 4 (0.2) Fees, pharmaceutical 3 (0.2) Insurance, pharmaceutical 10 (0.6) services Legislation, pharmacy 8 (0.5) Licensure, pharmacy 0 Pharmaceutical services 108 (6.1) Pharmaceutical services, 1 (0.1) online Pharmacies 13 (0.7) Pharmacists 367 (20.7) Pharmacists’ aides 23 (1.3) Pharmacy 33 (1.9) Pharmacy administration 14 (0.8) Pharmacy service, hospital 370 (20.9) Pharmacy and 5 (0.3) Therapeutics Committee Schools, pharmacy 19 (1.1) Societies, pharmaceutical 89 (5.0) Students, pharmacy 64 (3.6)
0 0
Ann Consult Pharmacother Pharm n (%) n (%)
1 (0.1) 0
0 0
0 1 (0.4)
39 (4.4)
41 (3.1)
10 (3.6)
8 (0.9) 686 (76.6) 20 (2.2)
3 (0.2) 16 (1.2) 4 (0.3)
1 (0.4) 2 (0.7) 0
82 (9.2)
1 (0.1)
5 (0.6) 0 3 (0.3) 6 (0.7) 6 (0.7) 93 (10.4) 0 14 175 3 38 5 28 7
J Am Pharm Assoc n (%) 2 (0.4) 3 (0.5)
66 (29.5) 137 (24.9)
J Pharm Int J Pharmacotherapy Res Pract Clin n (%) Social n (%) Pharm/Pharm Adm World Sci Pharm n (%) n (%) 0 0
0 6 (1.2)
0 0
0 2 (0.9)
5 (0.7)
70 (32.9)
515 (7.7)
1 (0.5) 8 (3.8) 3 (1.4)
75 (1.1) 884 (13.3) 86 (1.3)
0
151 (2.3)
3 (1.4) 11 (5.2) 6 (2.8)
3 (0.6) 13 (2.6) 14 (2.8)
1 (0.1) 20 (2.8) 1 (0.1)
0
1 (0.4)
5 (0.9)
1 (0.5)
1 (0.2)
4 (0.6)
0 0 7 (0.5)
0 0 3 (1.1)
0 0 1 (0.4)
0 4 (0.7) 11 (2.0)
0 0 1 (0.5)
2 (0.4) 2 (0.4) 1 (0.2)
1 (0.1) 1 (0.1) 4 (0.6)
0 0 16 (7.5)
0 0 38 (2.9) 0
0 0 51 (18.3) 0
3 (1.4) 1 (0.5) 6 (2.8) 0
0 0 35 (7.0) 0
0 0 21 (3.0%) 0
0 0 32 (15.0) 0
6 (0.5) 4 (0.3) 11 (0.8)
3 120 1 5 0 2 0
(1.1) 10 (4.5) 42 (7.6) 7 (3.3) 34 (6.8) (43.0) 125 (55.8) 277 (50.4) 53 (25.0) 185 (37.1) (0.4) 7 (3.1) 16 (2.9) 0 3 (0.6) (1.8) 3 (1.3) 32 (5.8) 4 (1.9) 8 (1.6) 1 (0.4) 1 (0.2) 0 0 (0.7) 31 (13.8) 16 (2.9) 14 (6.6) 76 (15.2) 0 0 0 5 (1.0)
0 12 (4.3) 4 (1.4)
3 (1.3) 2 (0.9) 12 (5.4)
11 (2.0) 51 (9.3) 45 (8.2)
3 (0.05) 32 (0.5)
125 (25.1)
8 (1.5) 39 (7.1) 18 (3.3)
1 (0.4) 3 (0.5) 1 (0.4) 4 (0.7) 50 (22.3) 106 (19.3) 0 0
Total n (%)
4 (1.9)
2 (0.9) 13 (5.8) 11 (4.9)
(1.6) 10 (0.8) (19.6) 113 (8.6) (0.3) 3 (0.2) (4.2) 7 (0.5) (0.6) 0 (3.1) 27 (2.1) (0.8) 1 (0.1)
223 (24.9) 38 (4.2) 568 (63.5)
Int J Pharm Pract n (%)
4 (1.9) 3 (1.4) 8 (3.8)
0 3 (0.6) 7 (1.4)
1 (0.1) 55 (7.8) 0 7 (1.0) 1 (0.1) 18 (2.5) 0
3 (0.4) 14 (2.0) 4 (0.6)
21 104 6 6 3 11 0
12 (0.2) 10 (0.2) 57 (0.9) 21 12 540 1
(0.3) (0.2) (8.1) (0.02)
(9.9) 155 (2.3) (48.8) 1574 (23.6) (2.8) 62 (0.9) (2.8) 143 (2.1) (1.4) 25 (0.4) (5.2) 593 (8.9) 18 (0.3)
3 (1.4) 1 (0.5) 6 (2.8)
272 (4.1) 217 (3.3) 729 (10.9)
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
Am J Health Am J Syst Pharm Pharm n (%) Educ n (%)
692
Table 5 Prevalence of pharmacy-specific MeSH terms used to index articles in each of the pharmacy journals analyzed
9.2% 10.8% 2.0% 8.4% 6.1% 14.4% 15.5% 7.9% 23.3% 9.0%
2.2%
8775 2047
Total MeSH Total pharmacy-specific MeSH Percentage of pharmacyspecific MeSH
15139 1355
13500 293
2711 214
2201 341
5757 831
2107 129
6234 523
8061 161
2708 293
67193 6187
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
693
“Drug Utilization” and “Drug Utilization Review.”17 The authors found that the journal Pharmacoepidemiol Drug Saf, which is specifically devoted to this field, presented only 47.5% of its articles included under the studied MeSH terms.17 Inaccurate indexing of articles may compromise the search of the literature for concepts rather than text words, therefore not accounting for variability in language which is one of the advantages of using MeSH terms. In addition to the utility of MeSH as a way to facilitate literature retrieval and to refine search strategies,11 the analysis of MeSH assignment can serve other purposes. Based on the present analysis different MeSH assignment patterns have been identified across the 10 journals. A distinction in the journals’ scope was objectively noted, with some being more pharmacotherapy-focused and containing a low proportion of pharmacyspecific MeSH terms in indexed articles (Ann Pharmacother and Pharmacotherapy), while others were more pharmacy practice-oriented and articles were indexed with a higher proportion of pharmacy-specific MeSH terms (Am J Pharm Educ, Int J Pharm Pract or J Am Pharm Assoc). A third category of journals presented a broader focus (Am J Health Syst Pharm, Consult Pharm, J Pharm Pract, Pharm World & Sci/Int J Clin Pharm, Res Soc Adm Pharm). These findings contrast with what has been claimed by some authors when justifying the choice of only the journals Am J Health Syst Pharm, Ann Pharmacother and Pharmacotherapy to analyze MeSH assignment delay: they considered that these journals were representative of pharmacy practice literature.30 These findings suggest that the categories used to group journals, such as the Subject Categories created for the Journal Citation Reports, should be analyzed on the basis of objective criteria such as the ones employed in this study. Assuming that MeSH terms were accurately assigned during article indexation, the analysis of MeSH would also enable the identification of specific areas within a given field through the recognition of “sentinel” MeSH. This would be of great value in guiding authors to submit their works to journals which targeted a specific area better than others. On the other hand, analyzing MeSH could shed some light into areas that are currently under-researched, specifically in this case: ethics in pharmacy, the role of Pharmacy and Therapeutics Committees, or pharmacy legislation. Likewise, the creation of MeSH terms within a specific discipline has also been studied
694
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
as a way of predicting new topic trends.31 Additionally, recent authors developed a robust and unbiased expert finding system based on the use of MeSH to quantitatively measure expertise.32 This could be used as a means of finding experts in a given subject to serve as peer-reviewers for journals or finding subjects associated with an expert.32 This study presents some limitations. First of all, the analysis included only 10 of the existing pharmacy journals. Nevertheless, these 10 journals included more than 50% of the articles published and indexed with the MeSH ‘Pharmacists.’ Secondly, articles indexed with descendant MeSH of pharmacy-specific MeSH terms in the MeSH tree hierarchy were not considered in the analysis. This means that we might have missed articles that could have been retrieved by automatic explosion (i.e., the process by which MeSH terms are automatically expanded to enable the retrieval of not only citations that carry the specified MeSH but also citations that carry any of the more specific MeSH terms, or descendants, included beneath the parent MeSH in the tree structure).33 However, some of the child MeSH terms that have a pharmacy-specific parent do not correspond to activities exclusively performed by pharmacists as defined in a previous study.9
Conclusions The findings of this study suggest that there is room for improvement with regards to the quality of assignment of the existing pharmacy-specific MeSH terms to articles indexed in pharmacy journals by the NLM catalogers. To date, over half of the articles published in the top-10 journals publishing pharmacy literature were indexed without a single pharmacy-specific MeSH. Significant differences in pharmacy-specific MeSH assignment patterns were found across journals suggesting distinct journals’ scopes. References 1. Doig GS, Simpson F. Efficient literature searching: a core skill for the practice of evidence-based medicine. Intensive Care Med 2003;29:2119–2127. 2. Young S, Duffull SB. A learning-based approach for performing an in-depth literature search using ME DLINE. J Clin Pharm Ther 2011;36:504–512. 3. Lawrence DW. Controlled search term vocabularies for finding articles relevant to injury prevention and safety promotion. Inj Prev 2011;17:260–265.
4. Meneghini R, Packer AL, Nassi-Calo L. Articles by Latin American authors in prestigious journals have fewer citations. PLoS One 2008;3:e3804. 5. Hoogendam A, Stalenhoef AF, Robbe PF, Overbeke AJ. Analysis of queries sent to PubMed at the point of care: observation of search behaviour in a medical teaching hospital. BMC Med Inform Decis Mak 2008;8:42. 6. The National Library of Medicine. Pubmed Home Page, 2014. [cited 03-Nov-2014]; Available from: http://www.ncbi.nlm.nih.gov/pubmed. 7. The National Library of Medicine. Fact Sheet: Medical Subject Headings (MeSHÒ), 2014. [cited 03Nov-2014]; Available from: http://www.nlm.nih. gov/pubs/factsheets/mesh.html. 8. Lipscomb CE. Medical Subject Headings (MeSH). Bull Med Libr Assoc 2000;88:265–266. 9. Minguet F, Van Den Boogerd L, Salgado TM, Correr CJ, Fernandez-Llimos F. Characterization of the Medical Subject Headings thesaurus for pharmacy. Am J Health Syst Pharm 2014;71:1965–1972. 10. The National Library of Medicine. Principles of MEDLINE Subject Indexing, 2014. [cited 03-Nov-2014]; Available from: http://www.nlm.nih.gov/bsd/disted/ meshtutorial/principlesofmedlinesubjectindexing/ principles/index.html. 11. Shultz M. Comparing test searches in PubMed and Google Scholar. J Med Libr Assoc 2007;95:442–445. 12. Jenuwine ES, Floyd JA. Comparison of Medical Subject Headings and text-word searches in MEDLINE to retrieve studies on sleep in healthy individuals. J Med Libr Assoc 2004;92:349–353. 13. Rana GK, Bradley DR, Hamstra SJ, et al. A validated search assessment tool: assessing practicebased learning and improvement in a residency program. J Med Libr Assoc 2011;99:77–81. 14. Jia X, Guo X, Li H, An X, Zhao Y. Characteristics and popular topics of latest researches into the effects of air particulate matter on cardiovascular system by bibliometric analysis. Inhal Toxicol 2013;25:211–218. 15. Zheng HC, Yan L, Cui L, Guan YF, Takano Y. Mapping the history and current situation of research on John Cunningham virus – a bibliometric analysis. BMC Infect Dis 2009;9:28. 16. Li HY, Cui L, Cui M. Hot topics in Chinese herbal drugs research documented in PubMed/MEDLINE by authors inside China and outside of China in the past 10 years: based on co-word cluster analysis. J Altern Complement Med 2009;15:779–785. 17. Sanz EJ. Pharmacoepidemiology and the “impact factor”. Eur J Clin Pharmacol 2005;60:765–772. 18. de Granda Orive JI, Garcia Rio F, Roig Vazquez F, Escobar Sacristan J, Gutierrez Jimenez T, Callol Sanchez L. Key words, essential tools for bibliographic research: analysis of usage in Archivos de Bronconeumologia for respiratory system knowledge areas. Arch Bronconeumol 2005;41:78–83. 19. Allison JJ, Kiefe CI, Weissman NW, Carter J, Centor RM. The art and science of searching
Minguet et al. / Research in Social and Administrative Pharmacy 11 (2015) 686–695
20.
21.
22.
23.
24.
25.
MEDLINE to answer clinical questions. Finding the right number of articles. Int J Technol Assess Health Care 1999;15:281–296. The National Library of Medicine. Depth of Indexing and the Rule of Three, 2014. [cited 03-Nov-2014]; Available from: http://www.nlm.nih.gov/bsd/disted/ meshtutorial/principlesofmedlinesubjectindexing/dept hofindexing/02.html. Ugolini D, Neri M, Knudsen LE, Bonassi S, Merlo DF. Searching PubMed for molecular epidemiology studies: the case of chromosome aberrations. Environ Mol Mutagen 2006;47:227–229. Sogaard M, Andersen JP, Schonheyder HC. Searching PubMed for studies on bacteremia, bloodstream infection, septicemia, or whatever the best term is: a note of caution. Am J Infect Control 2012;40:237–240. Mavropoulos A, Kiliaridis S. Orthodontic literature: an overview of the last 2 decades. Am J Orthod Dentofacial Orthop 2003;124:30–40. Portaluppi F. Consistency and accuracy of the Medical Subject Headings thesaurus for electronic indexing and retrieval of chronobiologic references. Chronobiol Int 2007;24:1213–1229. Wilczynski NL, Haynes RB. Consistency and accuracy of indexing systematic review articles and meta-analyses in Medline. Health Info Libr J 2009; 26:203–210.
695
26. Sampson M, Horsley T, Doja A. A bibliometric analysis of evaluative medical education studies: characteristics and indexing accuracy. Acad Med 2013;88: 421–427. 27. Funk ME, Reid CA. Indexing consistency in MEDLINE. Bull Med Libr Assoc 1983;71:176–183. 28. The National Library of Medicine. Frequently Asked Questions about Indexing for MEDLINE, 2014. [cited 03-Nov-2014]; Available from: http://www.nlm.nih. gov/bsd/indexfaq.html. 29. Richter RR, Austin TM. Using MeSH (medical subject headings) to enhance PubMed search strategies for evidence-based practice in physical therapy. Phys Ther 2012;92:124–132. 30. Rodriguez RW. Delay in indexing articles published in major pharmacy practice journals. Am J Health Syst Pharm 2014;71:321–324. 31. Moerchen F, Fradkin D, Dejori M, Wachmann B. Emerging trend prediction in biomedical literature. AMIA Annu Symp Proc; 2008::485–489. 32. Singh H, Singh R, Malhotra A, Kaur M. Developing a biomedical expert finding system using medical subject headings. Healthc Inform Res 2013;19:243–249. 33. The National Library of Medicine. Glossary, 2013. [cited 03-Nov-2014]; Available from: http://www.nlm. nih.gov/bsd/disted/pubmedtutorial/glossary.html.
