This article was downloaded by: [Nipissing University] On: 18 October 2014, At: 16:20 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Medical Reference Services Quarterly Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/wmrs20
An Introduction to Altmetrics Tara J. Brigham
a
a
Winn-Dixie Foundation Medical Library, Mayo Clinic , Jacksonville , Florida , USA Published online: 14 Oct 2014.
To cite this article: Tara J. Brigham (2014) An Introduction to Altmetrics, Medical Reference Services Quarterly, 33:4, 438-447, DOI: 10.1080/02763869.2014.957093 To link to this article: http://dx.doi.org/10.1080/02763869.2014.957093
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Medical Reference Services Quarterly, 33(4):438–447, 2014 Published with license by Taylor & Francis ISSN: 0276-3869 print=1540-9597 online DOI: 10.1080/02763869.2014.957093
EMERGING TECHNOLOGIES Matthew B. Hoy and Tara J. Brigham, Column Editors
An Introduction to Altmetrics Downloaded by [Nipissing University] at 16:20 18 October 2014
TARA J. BRIGHAM Winn-Dixie Foundation Medical Library, Mayo Clinic, Jacksonville, Florida, USA
Altmetrics, or alternative metrics, are forging a new way to capture the impact of not only articles, but also scholarly or research ‘‘products’’ by tracking them when they are mentioned online, such as in blogs or social media platforms. While altmetrics have a lot of potential, there are also some limitations preventing their full acceptance alongside traditional citation metrics. This column will explain the basics of altmetrics and altmetric tools, discuss some of the ways they can be used in libraries, and explore some possible concerns with this new metric. A list of resources for additional information is also included. KEYWORDS Altmetrics, citation metrics, social media
INTRODUCTION Researchers and scholars are often interested in the impact their findings have made, not only in their field but also in the wider context of the science community and even in mainstream media. Increasingly, administrators and granting organizations are also interested in this sort of information, usually
# Tara J. Brigham Comments and suggestions should be sent to the Column Editors: Matthew B. Hoy ([email protected]) and Tara J. Brigham ([email protected]). Address correspondence to Tara J. Brigham, Winn-Dixie Foundation Medical Library, Mayo Clinic, 4500 San Pablo Road, Jacksonville, FL 32224. E-mail: [email protected] Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/wmrs. 438
Downloaded by [Nipissing University] at 16:20 18 October 2014
Emerging Technologies
439
to determine tenure or to justify how departmental time or public funds are being spent. Traditionally, research or scholarly impact has been measured using a range of different citation metrics. Some examples of these metrics are the number of peer-reviewed articles published by an author, the number of citations accumulated from those published articles, and the ranking of the journals in which those articles were published. However, there are limitations that prevent these measurements from being comprehensive, foolproof, or even directly correlative. As observed by the number of individuals and organizations that have now signed the San Francisco Declaration on Research Assessment (DORA), there is an obvious discontent of using journal-based metrics to measure the impact of individual papers and researchers.1 Altmetrics presents academics, researchers, or any creators of scholarly content the opportunity to showcase previously unknown and unrecognized impact metrics. This column will explain what altmetrics are and the tools associated with them, concerns surrounding their use, as well as their application in the library setting.
WHAT ARE ALTMETRICS? The term altmetrics—alternative metrics or alternative to conventional metric measures—was first coined by Jason Priem in a 2010 tweet.2 Priem, along with Dario Taraborelli, Paul Groth, and Cameron Neylon, went on to write a manifesto that further defined altmetrics and its relevance in a world that was increasingly online.3 Three years later, Priem would describe altmetrics as the following: Altmetrics is the study and use of scholarly impact measures based on activity in online tools and environments. The term has also been used to describe the metrics themselves–one could propose in plural a ‘set of new altmetrics.’ Altmetrics is in most cases a subset of both scientometrics and webometrics; it is a subset of the latter in that it focuses more narrowly on scholarly influence as measured in online tools and environments, rather than on the Web more generally.4
Altmetrics cover more types of scholarly information since they not only measure article-level metrics but can also capture metrics on other research ‘‘products’’ as well. These other research products, such as data sets and software, are often overlooked in traditional citation metrics. And while most large funding organizations such as the U.S. National Institutes of Health still request a list of publications as validation of a researcher’s influence and skill on a grant application, the National Science Foundation made headlines in 2013 when it changed the wording in its policy from ‘‘Publications’’ to
Downloaded by [Nipissing University] at 16:20 18 October 2014
440
Emerging Technologies
‘‘Products.’’5 This change signaled to many researchers that other aspects of their work would now be recognized if it was citable and accessible. Altmetrics are proving to be one of the ways that researchers are now able to track the impact of these other research products. Due to a broader coverage of scholarly activity, altmetrics might also be helpful for clinicians and other health professionals who are more actively producing videos, blogs, books, and even posters, instead of journal articles. Since altmetrics aggregators are pulling data daily or weekly, they can produce a quicker turn-around time to reflect the influence an article or piece of research is exerting on a particular field. Altmetrics can also reveal how it affects a wider diversity of individuals, such as practitioners, clinicians, educators, and the general public.6 For scholars and researchers who would like to show how far their research stretches into the mainstream, altmetrics can be that link. In addition, altmetrics can provide a greater understanding of how a publication or product is being used. They can disclose which scholarly products are being read, discussed, saved, and recommended as well as cited.6 Tracking the variance in usage has led to the concept in altmetric literature called ‘‘impact flavors.’’ Impact flavors are a way to ‘‘understand the distinctive patterns in the diverse impacts of individual products.’’7 The impact flavor of a scholarly or research product featured in the mainstream media is going to be much different than one heavily saved in an online citation manager or cited in a research paper. Altmetrics capture the dynamism that permeates the online environment and attempt to marry it to the established process of measuring impact.
ALTMETRICS TOOLS While the tools used to gather altmetrics are in flux, currently there are three main altmetrics aggregators that anyone can use to explore them.
Altmetric.com Altmetric.com or Altmetric LLC reveals the impact of anything with a digital object identifier (DOI) or other standard identifier. It can find mentions of papers, books, datasets in social media sites (e.g., Twitter, Facebook, Pinterest, Googleþ), science blogs, many mainstream media outlets (including the NY Times, The Guardian, and special interest publications like Scientific American and New Scientist), government policy documents, and reference managers for mentions of academic papers.8 Its trademark Altmetric ‘‘donuts’’ are a recognizable feature of this service. The colors that make up the donut reflect the mix of sources mentioning the Altmetric score—blue for Twitter, yellow for blogs, red for mainstream media sources, and so on. The Altmetric score is an aggregate metric that is created by using
Emerging Technologies
441
the count from all the relevant mentions from a set of online sources (see above), along with the relative influence of each source.9 The Altmetric is a fee-based, subscription tool. It is geared toward publishers and institutions with various plans and pricing. Nature Publishing Group (nature.com) and Scopus are two examples that have incorporated Altmetric’s information into their website (see Figure 1). Individuals can use a free Bookmarklet to instantly view the altmetrics of certain papers. There are a few restrictions to using the Altmetric Bookmarklet, but it can produce altmetric figures if used in PubMed or any web page with a DOI.
Downloaded by [Nipissing University] at 16:20 18 October 2014
Impactstory.org ImpactStory.org is an open source and open access tool which is currently funded by the Alfred P. Sloan Foundation. Individuals can sign up for a free 30-day trial and can continue to use ImpactStory for $60 per year after the trial ends. Co-founded by Jason Priem and Heather Piwowar, ImpactStory was originally called ‘‘total-impact’’ and was a hackathon project developed
FIGURE 1 Screenshot of Altmetric donut in the Scopus database. (Data provided by Altmetric.com as of July 29, 2014).
442
Emerging Technologies
at the UK Beyond Impact workshop in 2011.10 ImpactStory tracks the impact of articles, data sets, software, videos, presentations, posters, and lab websites by monitoring a wide variety of data sources.10,11 At present, metrics are computed based on the following sources: . . . . . .
Downloaded by [Nipissing University] at 16:20 18 October 2014
. . . . . . . . . . .
Altmetric.com (which includes, blog posts, Facebook public posts, Googleþ posts, Twitter tweets), arXiv.org, CiteULike, Crossref, Delicious, Dryad, figshare, GitHub, Mendeley, PLoS search and article-level metrics, PubMed, Scopus, SlideShare, Vimeo, Web pages, Wikipedia, and YouTube.
Scholars or researchers can create a collection by uploading articles or products using Google Scholar or Open Researcher and Contributor ID (ORCID), or by providing PubMed IDs, DOI, or URL identifiers. ImpactStory then gathers and provides information on the author’s profile that conveys the altmetrics of each article or research product, detailing the influence within the context of the year it was published or created, and based on engagement type and audience.
Plum Analytics Plum Analytics was founded in 2011 by former Microsoft librarian, Mike Buschman and entrepreneur Andrea Michalek. They developed an analysis tool called PlumX that tracks more than 20 different types of what it calls ‘‘artifacts,’’ which includes ‘‘journal articles, books, videos, presentations, conference proceedings, datasets, source code, cases, and more.’’12 On these research products or artifacts, it collects impact metrics in five major categories: usage, captures, mentions, social media, and citations. The ‘‘Sunburst’’ visualization illustrates artifact types, individual artifacts, and the data from each source. It is included to help scholars, researchers, and administrators understand the large amount of data that is collected. Similar to Altmetric.com,
Emerging Technologies
443
PlumX is a fee-based, tiered subscription tool. However, it is geared mainly towards institutions or organizations, and pricing is based on the number of researchers tracked by the system. It offers a free trial of PlumX profile for individual researchers. In January 2014, Plum Analytics became a part of EBSCO Information Services.
Downloaded by [Nipissing University] at 16:20 18 October 2014
CONCERNS ABOUT ALTMETRICS As with traditional citation metrics, altmetrics can succumb to the same problems. Author disambiguation, gaming, and new issues, such as a lack of authority and regulation, hinder the widespread use of altmetrics. These issues have caused many scholars and researchers to be hesitant to use them in grant applications or promotion material. One of the difficulties of gathering accurate altmetrics is author or creator disambiguation. While a new initiative, called the ORCID is working to accurately attribute articles and products to authors by assigning each author a unique identifier, it is still not widespread.13 ORCID is free for anyone to register and is supported by many publishers and research universities, but is self-driven which means that the authors typically have to sign up themselves. Gaming is another concern surrounding the usefulness of altmetrics. Librarian and predatory publisher watchdog, Jeffrey Beall, warns of the very real potential for altmetrics or article-level metrics to be gamed. He points out that there are already companies which sell Facebook likes and those that will promote an article for a price.14 Individuals with enough technical skill and malicious intent could create multiple social media profiles and program them to regularly endorse the links to a certain article, leading to inflated altmetric values. This type of gaming leads to another problem: lack of regulation and authority. Unlike the regulated computations of scholarly influence such as the journal impact factor, it is unclear how altmetrics would distinguish between a poorly written article receiving a high number of mentions in the mainstream media and an article of higher quality discussed among leaders in that particular field. ‘‘[A] new paper that receives Facebook likes from a dozen Nobel laureates may be of higher impact than another paper that receives 50 likes from undergraduates at the local junior college.’’15 Important distinctions in the quality of feedback and recognition are currently lacking in most altmetric calculations. Another limitation of using altmetric figures is the inability to use them comparatively between different disciplines. Some disciplines are more active than others online, and involvement can also vary depending on the social media tool. Whereas one discipline has adopted Twitter as a platform for discussing new and relevant information, another might have a stronger following on Mendeley. The popularity evolution among the social media
Downloaded by [Nipissing University] at 16:20 18 October 2014
444
Emerging Technologies
platforms can also be a limitation of altmetrics. Social media tools like MySpace and Friendster were much more active eight years ago, but have since diminished in popularity and have been replaced with other tools like Twitter and Facebook. This means that altmetric measures should be regularly updated and normalized if they are to remain useful.15 Those currently working on altmetrics tools have recognized and are addressing some of the concerns identified above. In regards to gaming, Altmetric.com employees point out that ‘‘right now such gaming of the system is rare, but simple to spot algorithmically; in the case of Twitter spam, where hundreds of fake accounts will suddenly engage in meaningless, random retweets, all of the accounts are quite new, follow each other and have never mentioned a scholarly article before.’’9 There are also experienced groups like SSRN (Social Science Research Network) and COUNTER (Counting ONline Usage of NeTworked Electronic Resources) who regularly deal with these types of issues and could provide future assistance.9 There is also the effort to benchmark and put altmetrics into proper context. Many of the altmetrics tools have incorporated a ranking system that makes comparisons between other articles within the same journal, written during the same time frame, and across the whole database: As an example, the most popular article in the Altmetric database received an incredible amount of online attention relative to other items appearing in the same journal (Canadian Medical Association Journal). Accordingly, the article-level metrics page included a context statement, indicating that the article’s Altmetric score ‘‘is one of the highest ever scores in this journal (ranked #1 of 940).’’9
While there are clearly inroads being made to remedy some of the more pressing problems, until these issues are resolved and quality improves, altmetrics may not be taken as seriously as traditional citation metrics.
LIBRARIES AND ALTMETRICS As academics, researchers, or any creator of scholarly content endeavors to find new ways of providing evidence of their impact value, librarians or information professionals can offer to assist them in a number of ways. Librarians are the perfect educators and disseminators of information about altmetrics. They can help scholars and researchers negotiate the landscape of both altmetrics and traditional impact metrics. This could prove helpful to staff if they are interested in using altmetrics for career advancement or for pursuing grant funding. Additionally, librarians can provide an objective perspective of the pros and cons of altmetrics and explain the nuances that are important in understanding how altmetrics work. Since altmetrics cover a
Downloaded by [Nipissing University] at 16:20 18 October 2014
Emerging Technologies
445
wider range of ‘‘products’’ other than just articles, they could prove useful in hospital or other medical libraries, in addition to those that serve academic staff and researchers. For example, a local physician produces a YouTube video regarding a public health issue and is curious how useful it is in the wider community. In addition to the physician running a study, a librarian could offer to track the altmetrics of the video. Librarians can also help administrators who are interested in learning more about altmetrics as well. At many institutions, administrators are becoming more focused on quantitative data versus other measures when judging the worth of staff, research groups, or departments. Librarians can help administrators understand the limitations of using altmetrics for certain purposes and how they compare with other measures of influence. This data are valuable to administrators seeking to get a picture about how groups compare to one another and to their peer groups in other institutions—but the truth is that decision-makers want quantifiable data for making decisions. Promotion, hiring, and grant funding processes will continue to evolve, but those changes will not be prerequisites for including more holistic measurements.16
ALTMETRIC RESOURCES The following resources contain additional information about altmetrics: Association for Information Science and Technology (ASIS&T). ‘‘Altmetrics: What, Why and Where?’’ ASIS&T Bulletin 39, no. 4. (April=May 2013). http://www.asis.org/Bulletin/Apr-13/. Special issue dedicated to the topic of altmetrics. Galloway, L.M., J.L. Pease, and A.E. Rauh. ‘‘Introduction to Altmetrics for Science, Technology, Engineering, and Mathematics (STEM) Librarians.’’ Science & Technology Libraries 32, no. 4 (2013): 335–345. doi: 10.1080= 0194262X.2013.829762. Helpful overview of altmetrics; geared toward librarians in the science, technology, engineering and mathematics fields. Holmes, K. ‘‘Going Beyond Bibliometric and Altmetric Counts to Understand Impact.’’ Library Connect Newsletter, May 27, 2014. http:// libraryconnect.elsevier.com/articles/2014-05/going-beyond-bibliometric-andaltmetric-counts-understand-impact#sthash.9AP9vPBJ.dpuf. Short but effective overview of how bibliometrics and altmetrics can work together to reveal research impact. Watch the webinar presentation as well. PLOS. ‘‘PLOS Collections. Table of Contents: Altmetrics Collection.’’ 2012. http://www.ploscollections.org/altmetrics.
446
Emerging Technologies
A collection of articles on the emerging body of research to seed further study and use of altmetrics.
Downloaded by [Nipissing University] at 16:20 18 October 2014
Priem, J., H.A. Piwowar, and B.M. Hemminger. ‘‘Altmetrics In the Wild: Using Social Media to Explore Scholarly Impact.’’ arXiv.org. March 20, 2012. http:// arxiv.org/html/1203.4745v1. Focuses on questions such as: Is there enough data available to construct meaningful metrics (from social media outlets)? How is data distributed across tools, users, and time? How do altmetrics relate to accepted citation measures? Priem, J., D. Taraborelli, P. Groth, and C. Neylon. ‘‘Altmetrics: A Manifesto. (v.1.01).’’ Altmetrics. September 28, 2011. http://altmetrics.org/manifesto. The manifesto written by Jason Priem, Dario Taraborelli, Paul Groth, and Cameron Neylon on why altmetrics should be taken seriously.
CONCLUSION The continued growth of altmetrics seems inevitable, although their widespread application appears less certain. Acceptance of altmetrics will largely depend on their usefulness and quality. Librarians can help others understand the complexities that come with using altmetrics and how to properly incorporate them with other impact measures.
REFERENCES 1. San Francisco Declaration on Research Assessment (DORA) 2012. http:// am.ascb.org/dora/. 2. Priem, J. (jasonpriem). ‘‘I like the term #articlelevelmetrics, but it fails to imply diversity of measures. Lately, I’m liking #altmetrics.’’ September 28, 2010, 7:28 p.m. https:==twitter.com=jasonpriem=status=25844968813. Tweet. 3. Priem, J., D. Taraborelli, P. Groth, and C. Neylon. ‘‘Altmetrics: A Manifesto. (v.1.01).’’ Altmetrics. September 28, 2011. http://altmetrics.org/manifesto. 4. Priem, J., P. Groth, and D. Taraborelli. ‘‘The Altmetrics Collection.’’ PLoS ONE 7, no. 11: e48753. doi:10.1371=journal.pone.0048753. 5. National Science Foundation. NSF 13–1, Grant Proposal Guide Summary of Changes. January 2013. http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/ gpg_sigchanges.jsp. 6. Piwowar, H. ‘‘Altmetrics: What, Why and Where?’’ ASIS&T Bulletin 39, no. 4 (April=May 2013): 8–9. Library, Information Science & Technology Abstracts, EBSCOhost. 7. Lapinski, S., H. Piwowar, and J. Priem. ‘‘Riding the Crest of the Altmetrics Wave.’’ C&RL News 74, no. 6 (June 2013): 292–300. Library, Information Science & Technology Abstracts, EBSCOhost.
Downloaded by [Nipissing University] at 16:20 18 October 2014
Emerging Technologies
447
8. Altmetric, LLC. ‘‘What Does Altmetric Do?’’ Accessed July 13, 2014. http:// www.altmetric.com/whatwedo.php. 9. Liu, J., and E. Adie. ‘‘Five Challenges in Altmetrics: A Toolmaker’s Perspective.’’ ASIS&T Bulletin 39, no. 4. (April=May 2013): 31–34. Library, Information Science & Technology Abstracts, EBSCOhost. 10. ImpactStory. ‘‘FAQ.’’ Accessed July 13, 2014. http://impactstory.org/faq. 11. Piwowar, H. ‘‘Value All Research Products.’’ Nature 493 (January 10, 2013): 159. 12. Plum Analytics. ‘‘Overview: Plum Metrics.’’ Accessed July 14, 2014. http:// www.plumanalytics.com/metrics.html. 13. Galloway, L.M., J.L. Pease, and A.E. Rauh. ‘‘Introduction to Altmetrics for Science, Technology, Engineering, and Mathematics (STEM) Librarians.’’ Science & Technology Libraries 32, no. 4 (2013): 335–345. Library, Information Science & Technology Abstracts, EBSCOhost. 14. Beall, J. ‘‘Article-Level Metrics: An Ill-Conceived and Meretricious Idea.’’ Scholarly Open Access (blog). August 1, 2013. http://scholarlyoa.com/2013/08/01/ article-level-metrics/. 15. Tananbaum, G. ‘‘Article-Level Metrics: A SPARC Primer.’’ SPARC. April 2013. http://www.sparc.arl.org/resource/sparc-article-level-metrics-primer. 16. Buschman, M., and A. Michalek. ‘‘Are Alternative Metrics Still Alternative?’’ ASIS&T Bulletin 39, no. 4 (April=May 2013): 35–39. Library, Information Science & Technology Abstracts, EBSCOhost.
ABOUT THE AUTHOR Tara J. Brigham ([email protected]) is Librarian, Winn-Dixie Foundation Medical Library, Mayo Clinic, 4500 San Pablo Road, Jacksonville, FL 32224.
Medical Reference Services Quarterly Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/wmrs20
An Introduction to Altmetrics Tara J. Brigham
a
a
Winn-Dixie Foundation Medical Library, Mayo Clinic , Jacksonville , Florida , USA Published online: 14 Oct 2014.
To cite this article: Tara J. Brigham (2014) An Introduction to Altmetrics, Medical Reference Services Quarterly, 33:4, 438-447, DOI: 10.1080/02763869.2014.957093 To link to this article: http://dx.doi.org/10.1080/02763869.2014.957093
PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Medical Reference Services Quarterly, 33(4):438–447, 2014 Published with license by Taylor & Francis ISSN: 0276-3869 print=1540-9597 online DOI: 10.1080/02763869.2014.957093
EMERGING TECHNOLOGIES Matthew B. Hoy and Tara J. Brigham, Column Editors
An Introduction to Altmetrics Downloaded by [Nipissing University] at 16:20 18 October 2014
TARA J. BRIGHAM Winn-Dixie Foundation Medical Library, Mayo Clinic, Jacksonville, Florida, USA
Altmetrics, or alternative metrics, are forging a new way to capture the impact of not only articles, but also scholarly or research ‘‘products’’ by tracking them when they are mentioned online, such as in blogs or social media platforms. While altmetrics have a lot of potential, there are also some limitations preventing their full acceptance alongside traditional citation metrics. This column will explain the basics of altmetrics and altmetric tools, discuss some of the ways they can be used in libraries, and explore some possible concerns with this new metric. A list of resources for additional information is also included. KEYWORDS Altmetrics, citation metrics, social media
INTRODUCTION Researchers and scholars are often interested in the impact their findings have made, not only in their field but also in the wider context of the science community and even in mainstream media. Increasingly, administrators and granting organizations are also interested in this sort of information, usually
# Tara J. Brigham Comments and suggestions should be sent to the Column Editors: Matthew B. Hoy ([email protected]) and Tara J. Brigham ([email protected]). Address correspondence to Tara J. Brigham, Winn-Dixie Foundation Medical Library, Mayo Clinic, 4500 San Pablo Road, Jacksonville, FL 32224. E-mail: [email protected] Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/wmrs. 438
Downloaded by [Nipissing University] at 16:20 18 October 2014
Emerging Technologies
439
to determine tenure or to justify how departmental time or public funds are being spent. Traditionally, research or scholarly impact has been measured using a range of different citation metrics. Some examples of these metrics are the number of peer-reviewed articles published by an author, the number of citations accumulated from those published articles, and the ranking of the journals in which those articles were published. However, there are limitations that prevent these measurements from being comprehensive, foolproof, or even directly correlative. As observed by the number of individuals and organizations that have now signed the San Francisco Declaration on Research Assessment (DORA), there is an obvious discontent of using journal-based metrics to measure the impact of individual papers and researchers.1 Altmetrics presents academics, researchers, or any creators of scholarly content the opportunity to showcase previously unknown and unrecognized impact metrics. This column will explain what altmetrics are and the tools associated with them, concerns surrounding their use, as well as their application in the library setting.
WHAT ARE ALTMETRICS? The term altmetrics—alternative metrics or alternative to conventional metric measures—was first coined by Jason Priem in a 2010 tweet.2 Priem, along with Dario Taraborelli, Paul Groth, and Cameron Neylon, went on to write a manifesto that further defined altmetrics and its relevance in a world that was increasingly online.3 Three years later, Priem would describe altmetrics as the following: Altmetrics is the study and use of scholarly impact measures based on activity in online tools and environments. The term has also been used to describe the metrics themselves–one could propose in plural a ‘set of new altmetrics.’ Altmetrics is in most cases a subset of both scientometrics and webometrics; it is a subset of the latter in that it focuses more narrowly on scholarly influence as measured in online tools and environments, rather than on the Web more generally.4
Altmetrics cover more types of scholarly information since they not only measure article-level metrics but can also capture metrics on other research ‘‘products’’ as well. These other research products, such as data sets and software, are often overlooked in traditional citation metrics. And while most large funding organizations such as the U.S. National Institutes of Health still request a list of publications as validation of a researcher’s influence and skill on a grant application, the National Science Foundation made headlines in 2013 when it changed the wording in its policy from ‘‘Publications’’ to
Downloaded by [Nipissing University] at 16:20 18 October 2014
440
Emerging Technologies
‘‘Products.’’5 This change signaled to many researchers that other aspects of their work would now be recognized if it was citable and accessible. Altmetrics are proving to be one of the ways that researchers are now able to track the impact of these other research products. Due to a broader coverage of scholarly activity, altmetrics might also be helpful for clinicians and other health professionals who are more actively producing videos, blogs, books, and even posters, instead of journal articles. Since altmetrics aggregators are pulling data daily or weekly, they can produce a quicker turn-around time to reflect the influence an article or piece of research is exerting on a particular field. Altmetrics can also reveal how it affects a wider diversity of individuals, such as practitioners, clinicians, educators, and the general public.6 For scholars and researchers who would like to show how far their research stretches into the mainstream, altmetrics can be that link. In addition, altmetrics can provide a greater understanding of how a publication or product is being used. They can disclose which scholarly products are being read, discussed, saved, and recommended as well as cited.6 Tracking the variance in usage has led to the concept in altmetric literature called ‘‘impact flavors.’’ Impact flavors are a way to ‘‘understand the distinctive patterns in the diverse impacts of individual products.’’7 The impact flavor of a scholarly or research product featured in the mainstream media is going to be much different than one heavily saved in an online citation manager or cited in a research paper. Altmetrics capture the dynamism that permeates the online environment and attempt to marry it to the established process of measuring impact.
ALTMETRICS TOOLS While the tools used to gather altmetrics are in flux, currently there are three main altmetrics aggregators that anyone can use to explore them.
Altmetric.com Altmetric.com or Altmetric LLC reveals the impact of anything with a digital object identifier (DOI) or other standard identifier. It can find mentions of papers, books, datasets in social media sites (e.g., Twitter, Facebook, Pinterest, Googleþ), science blogs, many mainstream media outlets (including the NY Times, The Guardian, and special interest publications like Scientific American and New Scientist), government policy documents, and reference managers for mentions of academic papers.8 Its trademark Altmetric ‘‘donuts’’ are a recognizable feature of this service. The colors that make up the donut reflect the mix of sources mentioning the Altmetric score—blue for Twitter, yellow for blogs, red for mainstream media sources, and so on. The Altmetric score is an aggregate metric that is created by using
Emerging Technologies
441
the count from all the relevant mentions from a set of online sources (see above), along with the relative influence of each source.9 The Altmetric is a fee-based, subscription tool. It is geared toward publishers and institutions with various plans and pricing. Nature Publishing Group (nature.com) and Scopus are two examples that have incorporated Altmetric’s information into their website (see Figure 1). Individuals can use a free Bookmarklet to instantly view the altmetrics of certain papers. There are a few restrictions to using the Altmetric Bookmarklet, but it can produce altmetric figures if used in PubMed or any web page with a DOI.
Downloaded by [Nipissing University] at 16:20 18 October 2014
Impactstory.org ImpactStory.org is an open source and open access tool which is currently funded by the Alfred P. Sloan Foundation. Individuals can sign up for a free 30-day trial and can continue to use ImpactStory for $60 per year after the trial ends. Co-founded by Jason Priem and Heather Piwowar, ImpactStory was originally called ‘‘total-impact’’ and was a hackathon project developed
FIGURE 1 Screenshot of Altmetric donut in the Scopus database. (Data provided by Altmetric.com as of July 29, 2014).
442
Emerging Technologies
at the UK Beyond Impact workshop in 2011.10 ImpactStory tracks the impact of articles, data sets, software, videos, presentations, posters, and lab websites by monitoring a wide variety of data sources.10,11 At present, metrics are computed based on the following sources: . . . . . .
Downloaded by [Nipissing University] at 16:20 18 October 2014
. . . . . . . . . . .
Altmetric.com (which includes, blog posts, Facebook public posts, Googleþ posts, Twitter tweets), arXiv.org, CiteULike, Crossref, Delicious, Dryad, figshare, GitHub, Mendeley, PLoS search and article-level metrics, PubMed, Scopus, SlideShare, Vimeo, Web pages, Wikipedia, and YouTube.
Scholars or researchers can create a collection by uploading articles or products using Google Scholar or Open Researcher and Contributor ID (ORCID), or by providing PubMed IDs, DOI, or URL identifiers. ImpactStory then gathers and provides information on the author’s profile that conveys the altmetrics of each article or research product, detailing the influence within the context of the year it was published or created, and based on engagement type and audience.
Plum Analytics Plum Analytics was founded in 2011 by former Microsoft librarian, Mike Buschman and entrepreneur Andrea Michalek. They developed an analysis tool called PlumX that tracks more than 20 different types of what it calls ‘‘artifacts,’’ which includes ‘‘journal articles, books, videos, presentations, conference proceedings, datasets, source code, cases, and more.’’12 On these research products or artifacts, it collects impact metrics in five major categories: usage, captures, mentions, social media, and citations. The ‘‘Sunburst’’ visualization illustrates artifact types, individual artifacts, and the data from each source. It is included to help scholars, researchers, and administrators understand the large amount of data that is collected. Similar to Altmetric.com,
Emerging Technologies
443
PlumX is a fee-based, tiered subscription tool. However, it is geared mainly towards institutions or organizations, and pricing is based on the number of researchers tracked by the system. It offers a free trial of PlumX profile for individual researchers. In January 2014, Plum Analytics became a part of EBSCO Information Services.
Downloaded by [Nipissing University] at 16:20 18 October 2014
CONCERNS ABOUT ALTMETRICS As with traditional citation metrics, altmetrics can succumb to the same problems. Author disambiguation, gaming, and new issues, such as a lack of authority and regulation, hinder the widespread use of altmetrics. These issues have caused many scholars and researchers to be hesitant to use them in grant applications or promotion material. One of the difficulties of gathering accurate altmetrics is author or creator disambiguation. While a new initiative, called the ORCID is working to accurately attribute articles and products to authors by assigning each author a unique identifier, it is still not widespread.13 ORCID is free for anyone to register and is supported by many publishers and research universities, but is self-driven which means that the authors typically have to sign up themselves. Gaming is another concern surrounding the usefulness of altmetrics. Librarian and predatory publisher watchdog, Jeffrey Beall, warns of the very real potential for altmetrics or article-level metrics to be gamed. He points out that there are already companies which sell Facebook likes and those that will promote an article for a price.14 Individuals with enough technical skill and malicious intent could create multiple social media profiles and program them to regularly endorse the links to a certain article, leading to inflated altmetric values. This type of gaming leads to another problem: lack of regulation and authority. Unlike the regulated computations of scholarly influence such as the journal impact factor, it is unclear how altmetrics would distinguish between a poorly written article receiving a high number of mentions in the mainstream media and an article of higher quality discussed among leaders in that particular field. ‘‘[A] new paper that receives Facebook likes from a dozen Nobel laureates may be of higher impact than another paper that receives 50 likes from undergraduates at the local junior college.’’15 Important distinctions in the quality of feedback and recognition are currently lacking in most altmetric calculations. Another limitation of using altmetric figures is the inability to use them comparatively between different disciplines. Some disciplines are more active than others online, and involvement can also vary depending on the social media tool. Whereas one discipline has adopted Twitter as a platform for discussing new and relevant information, another might have a stronger following on Mendeley. The popularity evolution among the social media
Downloaded by [Nipissing University] at 16:20 18 October 2014
444
Emerging Technologies
platforms can also be a limitation of altmetrics. Social media tools like MySpace and Friendster were much more active eight years ago, but have since diminished in popularity and have been replaced with other tools like Twitter and Facebook. This means that altmetric measures should be regularly updated and normalized if they are to remain useful.15 Those currently working on altmetrics tools have recognized and are addressing some of the concerns identified above. In regards to gaming, Altmetric.com employees point out that ‘‘right now such gaming of the system is rare, but simple to spot algorithmically; in the case of Twitter spam, where hundreds of fake accounts will suddenly engage in meaningless, random retweets, all of the accounts are quite new, follow each other and have never mentioned a scholarly article before.’’9 There are also experienced groups like SSRN (Social Science Research Network) and COUNTER (Counting ONline Usage of NeTworked Electronic Resources) who regularly deal with these types of issues and could provide future assistance.9 There is also the effort to benchmark and put altmetrics into proper context. Many of the altmetrics tools have incorporated a ranking system that makes comparisons between other articles within the same journal, written during the same time frame, and across the whole database: As an example, the most popular article in the Altmetric database received an incredible amount of online attention relative to other items appearing in the same journal (Canadian Medical Association Journal). Accordingly, the article-level metrics page included a context statement, indicating that the article’s Altmetric score ‘‘is one of the highest ever scores in this journal (ranked #1 of 940).’’9
While there are clearly inroads being made to remedy some of the more pressing problems, until these issues are resolved and quality improves, altmetrics may not be taken as seriously as traditional citation metrics.
LIBRARIES AND ALTMETRICS As academics, researchers, or any creator of scholarly content endeavors to find new ways of providing evidence of their impact value, librarians or information professionals can offer to assist them in a number of ways. Librarians are the perfect educators and disseminators of information about altmetrics. They can help scholars and researchers negotiate the landscape of both altmetrics and traditional impact metrics. This could prove helpful to staff if they are interested in using altmetrics for career advancement or for pursuing grant funding. Additionally, librarians can provide an objective perspective of the pros and cons of altmetrics and explain the nuances that are important in understanding how altmetrics work. Since altmetrics cover a
Downloaded by [Nipissing University] at 16:20 18 October 2014
Emerging Technologies
445
wider range of ‘‘products’’ other than just articles, they could prove useful in hospital or other medical libraries, in addition to those that serve academic staff and researchers. For example, a local physician produces a YouTube video regarding a public health issue and is curious how useful it is in the wider community. In addition to the physician running a study, a librarian could offer to track the altmetrics of the video. Librarians can also help administrators who are interested in learning more about altmetrics as well. At many institutions, administrators are becoming more focused on quantitative data versus other measures when judging the worth of staff, research groups, or departments. Librarians can help administrators understand the limitations of using altmetrics for certain purposes and how they compare with other measures of influence. This data are valuable to administrators seeking to get a picture about how groups compare to one another and to their peer groups in other institutions—but the truth is that decision-makers want quantifiable data for making decisions. Promotion, hiring, and grant funding processes will continue to evolve, but those changes will not be prerequisites for including more holistic measurements.16
ALTMETRIC RESOURCES The following resources contain additional information about altmetrics: Association for Information Science and Technology (ASIS&T). ‘‘Altmetrics: What, Why and Where?’’ ASIS&T Bulletin 39, no. 4. (April=May 2013). http://www.asis.org/Bulletin/Apr-13/. Special issue dedicated to the topic of altmetrics. Galloway, L.M., J.L. Pease, and A.E. Rauh. ‘‘Introduction to Altmetrics for Science, Technology, Engineering, and Mathematics (STEM) Librarians.’’ Science & Technology Libraries 32, no. 4 (2013): 335–345. doi: 10.1080= 0194262X.2013.829762. Helpful overview of altmetrics; geared toward librarians in the science, technology, engineering and mathematics fields. Holmes, K. ‘‘Going Beyond Bibliometric and Altmetric Counts to Understand Impact.’’ Library Connect Newsletter, May 27, 2014. http:// libraryconnect.elsevier.com/articles/2014-05/going-beyond-bibliometric-andaltmetric-counts-understand-impact#sthash.9AP9vPBJ.dpuf. Short but effective overview of how bibliometrics and altmetrics can work together to reveal research impact. Watch the webinar presentation as well. PLOS. ‘‘PLOS Collections. Table of Contents: Altmetrics Collection.’’ 2012. http://www.ploscollections.org/altmetrics.
446
Emerging Technologies
A collection of articles on the emerging body of research to seed further study and use of altmetrics.
Downloaded by [Nipissing University] at 16:20 18 October 2014
Priem, J., H.A. Piwowar, and B.M. Hemminger. ‘‘Altmetrics In the Wild: Using Social Media to Explore Scholarly Impact.’’ arXiv.org. March 20, 2012. http:// arxiv.org/html/1203.4745v1. Focuses on questions such as: Is there enough data available to construct meaningful metrics (from social media outlets)? How is data distributed across tools, users, and time? How do altmetrics relate to accepted citation measures? Priem, J., D. Taraborelli, P. Groth, and C. Neylon. ‘‘Altmetrics: A Manifesto. (v.1.01).’’ Altmetrics. September 28, 2011. http://altmetrics.org/manifesto. The manifesto written by Jason Priem, Dario Taraborelli, Paul Groth, and Cameron Neylon on why altmetrics should be taken seriously.
CONCLUSION The continued growth of altmetrics seems inevitable, although their widespread application appears less certain. Acceptance of altmetrics will largely depend on their usefulness and quality. Librarians can help others understand the complexities that come with using altmetrics and how to properly incorporate them with other impact measures.
REFERENCES 1. San Francisco Declaration on Research Assessment (DORA) 2012. http:// am.ascb.org/dora/. 2. Priem, J. (jasonpriem). ‘‘I like the term #articlelevelmetrics, but it fails to imply diversity of measures. Lately, I’m liking #altmetrics.’’ September 28, 2010, 7:28 p.m. https:==twitter.com=jasonpriem=status=25844968813. Tweet. 3. Priem, J., D. Taraborelli, P. Groth, and C. Neylon. ‘‘Altmetrics: A Manifesto. (v.1.01).’’ Altmetrics. September 28, 2011. http://altmetrics.org/manifesto. 4. Priem, J., P. Groth, and D. Taraborelli. ‘‘The Altmetrics Collection.’’ PLoS ONE 7, no. 11: e48753. doi:10.1371=journal.pone.0048753. 5. National Science Foundation. NSF 13–1, Grant Proposal Guide Summary of Changes. January 2013. http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/ gpg_sigchanges.jsp. 6. Piwowar, H. ‘‘Altmetrics: What, Why and Where?’’ ASIS&T Bulletin 39, no. 4 (April=May 2013): 8–9. Library, Information Science & Technology Abstracts, EBSCOhost. 7. Lapinski, S., H. Piwowar, and J. Priem. ‘‘Riding the Crest of the Altmetrics Wave.’’ C&RL News 74, no. 6 (June 2013): 292–300. Library, Information Science & Technology Abstracts, EBSCOhost.
Downloaded by [Nipissing University] at 16:20 18 October 2014
Emerging Technologies
447
8. Altmetric, LLC. ‘‘What Does Altmetric Do?’’ Accessed July 13, 2014. http:// www.altmetric.com/whatwedo.php. 9. Liu, J., and E. Adie. ‘‘Five Challenges in Altmetrics: A Toolmaker’s Perspective.’’ ASIS&T Bulletin 39, no. 4. (April=May 2013): 31–34. Library, Information Science & Technology Abstracts, EBSCOhost. 10. ImpactStory. ‘‘FAQ.’’ Accessed July 13, 2014. http://impactstory.org/faq. 11. Piwowar, H. ‘‘Value All Research Products.’’ Nature 493 (January 10, 2013): 159. 12. Plum Analytics. ‘‘Overview: Plum Metrics.’’ Accessed July 14, 2014. http:// www.plumanalytics.com/metrics.html. 13. Galloway, L.M., J.L. Pease, and A.E. Rauh. ‘‘Introduction to Altmetrics for Science, Technology, Engineering, and Mathematics (STEM) Librarians.’’ Science & Technology Libraries 32, no. 4 (2013): 335–345. Library, Information Science & Technology Abstracts, EBSCOhost. 14. Beall, J. ‘‘Article-Level Metrics: An Ill-Conceived and Meretricious Idea.’’ Scholarly Open Access (blog). August 1, 2013. http://scholarlyoa.com/2013/08/01/ article-level-metrics/. 15. Tananbaum, G. ‘‘Article-Level Metrics: A SPARC Primer.’’ SPARC. April 2013. http://www.sparc.arl.org/resource/sparc-article-level-metrics-primer. 16. Buschman, M., and A. Michalek. ‘‘Are Alternative Metrics Still Alternative?’’ ASIS&T Bulletin 39, no. 4 (April=May 2013): 35–39. Library, Information Science & Technology Abstracts, EBSCOhost.
ABOUT THE AUTHOR Tara J. Brigham ([email protected]) is Librarian, Winn-Dixie Foundation Medical Library, Mayo Clinic, 4500 San Pablo Road, Jacksonville, FL 32224.
