DOI: 10.1111/hir.12046
A comparison of searching the Cochrane library databases via CRD, Ovid and Wiley: implications for systematic searching and information services Jenny Craven, Jayne Jefferies, Jenny Kendrick, Dave Nicholls, Janette Boynton & Ruth Frankish National Institute for Health and Care Excellence (NICE), UK
Abstract Background: The Cochrane Library databases are available via different interfaces; evidence in the literature, together with anecdotal evidence, shows interfaces perform differently. To ensure the quality of searches, a study was undertaken to systematically explore the functionality of interfaces. Objectives: To demonstrate differences in functionality when searching the same databases across different interfaces; to discuss the implications this may have on searching; and in a wider context, to suggest a ‘best match’ for comparable searching. Methods: Detailed cross-comparisons of a selection of search functions including MeSH terms, free text, proximity operators and truncation were undertaken in databases accessed via CRD, Wiley and Ovid. Up to three terms per function were selected and analysed. Results: Differences were identified in the way searches for MeSH headings are executed, which fields are searched, how proximity operators perform, the word order searched and where terms are searched. This adds to a body of evidence demonstrating a lack of consistency in searching across different interfaces. Conclusions: A ‘best match’ for comparable searching is suggested. Differences between interfaces offering the same database content can have implications for the success of a search, on user education, and on database evaluation and purchasing decisions. Keywords: database searching; information retrieval; library and information professionals; review and systematic search; search strategies Key Messages
• • • • •
For a truly systematic search, it is preferable to be able to search across a range of databases with confidence that the search is a comparable one. This study identifies both similarities and differences in the functionality and syntax used across different interfaces. The differences identified across interfaces for the same database have a significant impact on information professionals’ practice across a range of responsibilities. Information professionals should be aware of these differences and apply a ‘best match’ for search syntax, as outlined in this study. Information professionals should also refer to interface Help pages to ensure any changes to the interface are taken into account.
Introduction and background Systematic literature searching is a core function in the service delivery of health library and information
Correspondence: Jenny Craven, Level 1A, City Tower, Piccadilly Plaza Manchester, M1 4BD Manchester, UK, E-mail: jenny.craven@ nice.org.uk
54
services. It involves a ‘thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible’.1 Systematic literature searching is undertaken by the Guidance Information Services (gIS) team at the National Institute for Health and Care Excellence (NICE), the UK organisation responsible for providing independent, authoritative and
© 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Group Health Information & Libraries Journal, 31, pp. 54–63
Comparison of searching Cochrane library databases, Jenny Craven et al.
evidence-based guidance on the most effective ways to prevent, diagnose and treat disease and ill health. The gIS team offers tailored support to the guidance-producing teams within NICE, providing professional expertise to enable access to highquality information to support the development of guidance. The gIS team carries out systematic literature searching using a variety of bibliographic databases, including MEDLINE, MEDLINE In-Process, Embase, and the Cochrane and Centre for Reviews and Dissemination (CRD) databases. Alternative searching techniques (e.g. citation searching) are also employed as appropriate. A number of the databases searched by the gIS team are accessible via more than one interface, for example Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE). This raises the question of which interface to use in practice and, in particular, whether there are differences between the interfaces in terms of search functionality and syntax and, in turn, retrieval. Aim Two internal investigations (referred to in this article as the studies) were undertaken during 2011 and 2012, prompted by the various upgrades to the interfaces used for the Cochrane Library and CRD databases: one comparing the OVID and Wiley online interfaces for searching the CDSR and Central Register of Controlled Trials (CENTRAL) databases, the second to compare the CRD, OVID and Wiley online interfaces for searching the DARE, Health Technology Assessment (HTA) and NHS Economic Evaluation Database (NHS EED) databases. Although access to alternative interfaces was available, the gIS team was aware of differences in the search functionality and syntax across each, which led to concerns about comparable searching. However, this awareness was not fully documented and often anecdotal. The aim of the studies was to identify and provide documented evidence of a comparable search syntax or ‘best match’ across the different interfaces for each of the databases. The studies were undertaken for the purpose of supporting searching practice within the gIS team,
to enable individuals to interrogate different interfaces effectively and comparably. The studies did not aim to identify an interface of ‘preference’ or to identify which was the ‘best’ interface to search. The results of the studies are now being disseminated for the purposes of sharing practice. This study aims to add to the body of evidence identified in the literature, demonstrating a lack of consistency in searching across different interfaces, the impact this has on trained and untrained users and to provide suggestions for achieving comparable searches. Literature review A review of the literature was conducted to identify published studies comparing search functionality for the same database within different interfaces and to compare this with the results of our studies. For practical reasons, the searches were limited to a subjectively derived set of databases; these included MEDLINE, Library and Information Science Abstracts (LISA) and Library, Information Science & Technology Abstracts (LISTA). Citation searching and more general website searching were also undertaken. No date limit was set, but searches were limited to English language. This ensured the literature review was manageable within the resources available for the studies. The majority of the studies identified appraised comparisons between databases, rather than comparisons of the same database within different interfaces, as presented in this study. Furthermore, the majority of studies concentrate on ease-of-use evaluations, rather than on assessment of the differences in search functionality and syntax offered by the various vendor products. However, although these studies are not comparing search functionality and syntax differences, the results can inform institutional purchasing decisions and the scope of information skills training.2–4 The literature review revealed studies relating to differences in functionality when searching the same database across different interfaces, and these are discussed further in this section. Studies discussing searching within alternative interfaces, whether dealing with medical2,3,5–12 or non-medical databases,4,13–17 have uncovered a number of inconsistencies. For example, in the
© 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Group Health Information & Libraries Journal, 31, pp. 54–63
55
56
Comparison of searching Cochrane library databases, Jenny Craven et al.
reporting of studies around MEDLINE usage,3,6,9,10,12 how MeSH terms are made available to the user has been shown to vary, with differences in mapping capability. Some map to MeSH automatically and others do not or are unable to map from truncated words or multiple MeSH terms. Inconsistencies in truncation, pluralisation, synonym capture and syntax usage generally, have been identified2,3,5–7,9,10,13,15 with Younger and Boddy’s study of AMED via EBSCOhost, Ovid and Dialog noting that after the initial untailored search strategy had failed, ‘It was necessary to amend the search terms, truncation and subject headings to return identical sets of results across all three interfaces’.2 Other studies have reported on the working capacity required to be able to learn the idiosyncrasies of each database and how this can impact on both trained and untrained searchers.2,5–7,11 Variations are documented, such as term sequencing, spelling and capitalisation, and differences in the field codes used in each of the vendor products (for example, PN or PER being used for ‘Personal Name as Subject’ in OVID and ProQuest, respectively),5 whilst other studies show how disparities in stopwords across interfaces can impact on search consistency.8,10 Different products also have a variety of default settings, for example, some searching all fields as standard, whilst others trained upon subject-rich fields,2,8,10 which might not be perceived by the inexperienced searcher, and, as Allison stated in a review of CINAHL interfaces, ‘the user needs to know the attributes of the specific interface being used – that seemingly equivalent terms and functions are not necessarily consistent across interfaces’.5 Time was also identified as a factor, both in terms of the currency of the available references and the period of coverage offered.6,12–14,17 Each interface has a different schedule for uploading new references to the databases, and it was noted in several studies that this staggering of importation will impact on the results retrieved by the end-user, depending on when they undertake their search and in which interface. With this in mind, Crawford noted in a study on the availability of databases, that ‘due to the differences in time coverage between the different versions and in the indexing and retrieval algorithms of the
individual databases, it is difficult to declare a clear winner’.15 It is clear that the differences identified have implications for the information professional in terms of developing the knowledge and skills to search databases via different interfaces and ensuring the quality of a literature search regardless of the interface used. In addition, the different functionality and syntax offered across different interfaces has an impact on how user education sessions are planned and delivered, and on database evaluation and purchasing decisions. Methods During 2011 and 2012, the following online database versions were searched to examine the available search functionality and to determine the best match search syntax for common search functions: For CDSR and CENTRAL: • Ovid CDSR (2005 – Jan 2011) and CENTRAL (First Quarter 2011) • Wiley CDSR (Issue 1 of 12, Jan 2011) and CENTRAL (Issue 1 of 4, Jan 2011) For DARE, NHS EED and HTA database: • CRD DARE, NHS EED and HTA database, between 10 Nov 2011 and 9 Feb 2012 • Ovid DARE, NHS EED and HTA database (Fourth Quarter 2011) • Wiley DARE, NHS EED and HTA database (Issue 4 of 4 2011) The search syntax of the CRD, OVID and Wiley online interfaces was compared and any discrepancies in the results retrieved across the interfaces were identified, investigated and explained. Any implications these discrepancies may have on searching and in a wider context were explored and suggestions for a ‘best match’ for comparable searching put forward. Detailed cross-comparisons of search syntax between interfaces were undertaken on the search functions listed in Table 1, which also includes examples of some of the search terms tested. Appropriate terms were selected from existing search strategies developed by gIS [from the interventional procedures (IP), guideline review (GR) and diagnostic assessment (DAP) programmes]. The searches were undertaken by four members of the gIS team, who selected and investigated
© 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Group Health Information & Libraries Journal, 31, pp. 54–63
Comparison of searching Cochrane library databases, Jenny Craven et al. Table 1 Search functions tested with examples of search terms used Search function
Example search terms tested
Truncation MeSH terms Exploded MeSH terms
Placenta* Angioplasty Surgical Procedures, Minimally Invasive Hip* adj pain* (Percutan* or transcath*) adj (clos* or shut* or plug*)
Adjacency (full text) Adjacency (title, abstract, keyword search) Free text single term (full text) Free text single term (title, abstract, keyword) Free text multiple terms (full text) Free text multiple terms (title, abstract, keyword search)
Hypercholesterol* Urokinase* Stomach* or gastric* or duodenal* or duodenum* Signature* OR profil* OR assay*
around 16 search terms each per search function and then undertook deeper analysis on one or two terms per function. For practical reasons, terms were selected that did not retrieve a large number of results (ideally
A comparison of searching the Cochrane library databases via CRD, Ovid and Wiley: implications for systematic searching and information services Jenny Craven, Jayne Jefferies, Jenny Kendrick, Dave Nicholls, Janette Boynton & Ruth Frankish National Institute for Health and Care Excellence (NICE), UK
Abstract Background: The Cochrane Library databases are available via different interfaces; evidence in the literature, together with anecdotal evidence, shows interfaces perform differently. To ensure the quality of searches, a study was undertaken to systematically explore the functionality of interfaces. Objectives: To demonstrate differences in functionality when searching the same databases across different interfaces; to discuss the implications this may have on searching; and in a wider context, to suggest a ‘best match’ for comparable searching. Methods: Detailed cross-comparisons of a selection of search functions including MeSH terms, free text, proximity operators and truncation were undertaken in databases accessed via CRD, Wiley and Ovid. Up to three terms per function were selected and analysed. Results: Differences were identified in the way searches for MeSH headings are executed, which fields are searched, how proximity operators perform, the word order searched and where terms are searched. This adds to a body of evidence demonstrating a lack of consistency in searching across different interfaces. Conclusions: A ‘best match’ for comparable searching is suggested. Differences between interfaces offering the same database content can have implications for the success of a search, on user education, and on database evaluation and purchasing decisions. Keywords: database searching; information retrieval; library and information professionals; review and systematic search; search strategies Key Messages
• • • • •
For a truly systematic search, it is preferable to be able to search across a range of databases with confidence that the search is a comparable one. This study identifies both similarities and differences in the functionality and syntax used across different interfaces. The differences identified across interfaces for the same database have a significant impact on information professionals’ practice across a range of responsibilities. Information professionals should be aware of these differences and apply a ‘best match’ for search syntax, as outlined in this study. Information professionals should also refer to interface Help pages to ensure any changes to the interface are taken into account.
Introduction and background Systematic literature searching is a core function in the service delivery of health library and information
Correspondence: Jenny Craven, Level 1A, City Tower, Piccadilly Plaza Manchester, M1 4BD Manchester, UK, E-mail: jenny.craven@ nice.org.uk
54
services. It involves a ‘thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible’.1 Systematic literature searching is undertaken by the Guidance Information Services (gIS) team at the National Institute for Health and Care Excellence (NICE), the UK organisation responsible for providing independent, authoritative and
© 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Group Health Information & Libraries Journal, 31, pp. 54–63
Comparison of searching Cochrane library databases, Jenny Craven et al.
evidence-based guidance on the most effective ways to prevent, diagnose and treat disease and ill health. The gIS team offers tailored support to the guidance-producing teams within NICE, providing professional expertise to enable access to highquality information to support the development of guidance. The gIS team carries out systematic literature searching using a variety of bibliographic databases, including MEDLINE, MEDLINE In-Process, Embase, and the Cochrane and Centre for Reviews and Dissemination (CRD) databases. Alternative searching techniques (e.g. citation searching) are also employed as appropriate. A number of the databases searched by the gIS team are accessible via more than one interface, for example Cochrane Database of Systematic Reviews (CDSR) and Database of Abstracts of Reviews of Effects (DARE). This raises the question of which interface to use in practice and, in particular, whether there are differences between the interfaces in terms of search functionality and syntax and, in turn, retrieval. Aim Two internal investigations (referred to in this article as the studies) were undertaken during 2011 and 2012, prompted by the various upgrades to the interfaces used for the Cochrane Library and CRD databases: one comparing the OVID and Wiley online interfaces for searching the CDSR and Central Register of Controlled Trials (CENTRAL) databases, the second to compare the CRD, OVID and Wiley online interfaces for searching the DARE, Health Technology Assessment (HTA) and NHS Economic Evaluation Database (NHS EED) databases. Although access to alternative interfaces was available, the gIS team was aware of differences in the search functionality and syntax across each, which led to concerns about comparable searching. However, this awareness was not fully documented and often anecdotal. The aim of the studies was to identify and provide documented evidence of a comparable search syntax or ‘best match’ across the different interfaces for each of the databases. The studies were undertaken for the purpose of supporting searching practice within the gIS team,
to enable individuals to interrogate different interfaces effectively and comparably. The studies did not aim to identify an interface of ‘preference’ or to identify which was the ‘best’ interface to search. The results of the studies are now being disseminated for the purposes of sharing practice. This study aims to add to the body of evidence identified in the literature, demonstrating a lack of consistency in searching across different interfaces, the impact this has on trained and untrained users and to provide suggestions for achieving comparable searches. Literature review A review of the literature was conducted to identify published studies comparing search functionality for the same database within different interfaces and to compare this with the results of our studies. For practical reasons, the searches were limited to a subjectively derived set of databases; these included MEDLINE, Library and Information Science Abstracts (LISA) and Library, Information Science & Technology Abstracts (LISTA). Citation searching and more general website searching were also undertaken. No date limit was set, but searches were limited to English language. This ensured the literature review was manageable within the resources available for the studies. The majority of the studies identified appraised comparisons between databases, rather than comparisons of the same database within different interfaces, as presented in this study. Furthermore, the majority of studies concentrate on ease-of-use evaluations, rather than on assessment of the differences in search functionality and syntax offered by the various vendor products. However, although these studies are not comparing search functionality and syntax differences, the results can inform institutional purchasing decisions and the scope of information skills training.2–4 The literature review revealed studies relating to differences in functionality when searching the same database across different interfaces, and these are discussed further in this section. Studies discussing searching within alternative interfaces, whether dealing with medical2,3,5–12 or non-medical databases,4,13–17 have uncovered a number of inconsistencies. For example, in the
© 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Group Health Information & Libraries Journal, 31, pp. 54–63
55
56
Comparison of searching Cochrane library databases, Jenny Craven et al.
reporting of studies around MEDLINE usage,3,6,9,10,12 how MeSH terms are made available to the user has been shown to vary, with differences in mapping capability. Some map to MeSH automatically and others do not or are unable to map from truncated words or multiple MeSH terms. Inconsistencies in truncation, pluralisation, synonym capture and syntax usage generally, have been identified2,3,5–7,9,10,13,15 with Younger and Boddy’s study of AMED via EBSCOhost, Ovid and Dialog noting that after the initial untailored search strategy had failed, ‘It was necessary to amend the search terms, truncation and subject headings to return identical sets of results across all three interfaces’.2 Other studies have reported on the working capacity required to be able to learn the idiosyncrasies of each database and how this can impact on both trained and untrained searchers.2,5–7,11 Variations are documented, such as term sequencing, spelling and capitalisation, and differences in the field codes used in each of the vendor products (for example, PN or PER being used for ‘Personal Name as Subject’ in OVID and ProQuest, respectively),5 whilst other studies show how disparities in stopwords across interfaces can impact on search consistency.8,10 Different products also have a variety of default settings, for example, some searching all fields as standard, whilst others trained upon subject-rich fields,2,8,10 which might not be perceived by the inexperienced searcher, and, as Allison stated in a review of CINAHL interfaces, ‘the user needs to know the attributes of the specific interface being used – that seemingly equivalent terms and functions are not necessarily consistent across interfaces’.5 Time was also identified as a factor, both in terms of the currency of the available references and the period of coverage offered.6,12–14,17 Each interface has a different schedule for uploading new references to the databases, and it was noted in several studies that this staggering of importation will impact on the results retrieved by the end-user, depending on when they undertake their search and in which interface. With this in mind, Crawford noted in a study on the availability of databases, that ‘due to the differences in time coverage between the different versions and in the indexing and retrieval algorithms of the
individual databases, it is difficult to declare a clear winner’.15 It is clear that the differences identified have implications for the information professional in terms of developing the knowledge and skills to search databases via different interfaces and ensuring the quality of a literature search regardless of the interface used. In addition, the different functionality and syntax offered across different interfaces has an impact on how user education sessions are planned and delivered, and on database evaluation and purchasing decisions. Methods During 2011 and 2012, the following online database versions were searched to examine the available search functionality and to determine the best match search syntax for common search functions: For CDSR and CENTRAL: • Ovid CDSR (2005 – Jan 2011) and CENTRAL (First Quarter 2011) • Wiley CDSR (Issue 1 of 12, Jan 2011) and CENTRAL (Issue 1 of 4, Jan 2011) For DARE, NHS EED and HTA database: • CRD DARE, NHS EED and HTA database, between 10 Nov 2011 and 9 Feb 2012 • Ovid DARE, NHS EED and HTA database (Fourth Quarter 2011) • Wiley DARE, NHS EED and HTA database (Issue 4 of 4 2011) The search syntax of the CRD, OVID and Wiley online interfaces was compared and any discrepancies in the results retrieved across the interfaces were identified, investigated and explained. Any implications these discrepancies may have on searching and in a wider context were explored and suggestions for a ‘best match’ for comparable searching put forward. Detailed cross-comparisons of search syntax between interfaces were undertaken on the search functions listed in Table 1, which also includes examples of some of the search terms tested. Appropriate terms were selected from existing search strategies developed by gIS [from the interventional procedures (IP), guideline review (GR) and diagnostic assessment (DAP) programmes]. The searches were undertaken by four members of the gIS team, who selected and investigated
© 2014 The authors. Health Information and Libraries Journal © 2014 Health Libraries Group Health Information & Libraries Journal, 31, pp. 54–63
Comparison of searching Cochrane library databases, Jenny Craven et al. Table 1 Search functions tested with examples of search terms used Search function
Example search terms tested
Truncation MeSH terms Exploded MeSH terms
Placenta* Angioplasty Surgical Procedures, Minimally Invasive Hip* adj pain* (Percutan* or transcath*) adj (clos* or shut* or plug*)
Adjacency (full text) Adjacency (title, abstract, keyword search) Free text single term (full text) Free text single term (title, abstract, keyword) Free text multiple terms (full text) Free text multiple terms (title, abstract, keyword search)
Hypercholesterol* Urokinase* Stomach* or gastric* or duodenal* or duodenum* Signature* OR profil* OR assay*
around 16 search terms each per search function and then undertook deeper analysis on one or two terms per function. For practical reasons, terms were selected that did not retrieve a large number of results (ideally
