CrossRef Text and Data Mining
•Descargar como PPT, PDF•
6 recomendaciones•3,961 vistas
After a successful pilot under the name "Prospect," CrossRef will provide a means for publishers to simplify text and data mining access for researchers. Both researchers and publishers will benefit from support of standard APIs and data representations to enable text and data mining across open access and subscription-based publishers, and this is what CrossRef is aiming to provide. This webinar was held on October 28, 2014.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (13)
Similar a CrossRef Text and Data Mining
Similar a CrossRef Text and Data Mining (20)
Más de Crossref
Más de Crossref (20)
Último
Último (20)
CrossRef Text and Data Mining
- 1. Rachael Lammey Product Manager, CrossRef 28 October 2014
- 2. Not-for-profit association of scholarly publishers All subjects, all business models 4,000+ organizations from all over the world 83 non-publisher affiliates, 2000 library affiliates 68 million content items
- 5. User clicks on CrossRef DOI reference link in Journal A Tani, N., N. Tomaru, M. Araki, AND K. Ohba. 1996. Genetic diversity and differentiation in populations of Japanese stone pine (Pinus pumila) in Japan. Canadian Journal of Forest Research 26: 1454–1462.[CrossRef] DOI directory returns URL User accesses cited article in Journal B
- 6. 90,000,000
- 7. Services • Cross-publisher reference linking • Cross-publisher Cited-by linking • Cross-publisher metadata feeds • Cross-publisher plagiarism screening • Cross-publisher update identification • Cross-publisher funder identification • Cross-publisher text and data mining Powered by iThenticate
- 8. A Text and Data Mining Hub for Researchers
- 9. What is text and data mining? Text Mining is an interdisciplinary field combining techniques from linguistics, computer science and statistics to build tools that can efficiently retrieve and extract information from digital text. http://blogs.plos.org/everyone/2013/04/17/announcing-the-plos-text-mining-collection/ It uses powerful computers to find links between drugs and side effects, or genes and diseases, that are hidden within the vast scientific literature. These are discoveries that a person scouring through papers one by one may never notice. http://www.theguardian.com/science/2012/may/23/text-mining-research-tool-forbidden
- 10. http://www.jisc.ac.uk/media/documents/publications/textminingbp_rtf.rtf Marc Weeber and colleagues used automated text mining tools to infer that the drug thalidomide could treat several diseases it had not been associated with before. Thalidomide was taken off the market 40 years ago, but is still the subject of research because it seems to benefit leprosy patients via their immune systems. Weeber and Grietje Molema, an immunologist, used text mining tools to search the literature for papers on thalidomide and then pick out those containing concepts related to immunology. One concept, concerning thalidomide’s ability to inhibit Interleukin-12 (IL-12), a chemical involved in the launch of an immune response, struck Molema as particularly interesting. A second automated search for diseases that improve when the action of IL-12 is blocked, revealed several not previously linked with thalidomide, including chronic hepatitis, myasthenia gravis and a type of gastritis. “Type in thalidomide and you get 2-3000 hits. Type in disease and you get 40,000 hits. With automated text mining tools we only had to read 100-200 abstracts and 20 or 30 full papers. We’ve created hypotheses for others to follow up” says Weeber. Weeber et al. J Am Med Inform Assoc. 2003 10 252-259
- 12. Why? • Researchers find it impractical to negotiate multiple bilateral agreements with hundreds of subscription- based publishers in order to authorize TDM of subscribed content. • Subscription-based publishers find it impractical to negotiate multiple bilateral agreements with thousands of researchers and institutions in order to authorize TDM of subscribed content. • All parties would benefit from support of standard APIs and data representations in order to enable TDM across both open access and subscription-based publishers.
- 13. * Chinese Geoscience Union * Chinese Institute Of Automation Engineers (Ciae) * Chinese Journal Of Mechanical Engineering * Chinese Mathematical Society * Chinese Physical Society * Chinese Physiological Society * Chinese Society Of Theoretical And Applied Mechanics * Chonnam National University Medical School (Kamje) * Christ University Bangalore * Cic Edizioni Internazionali * Cig Media Group * Cilip Information Literacy Group * Civil-Comp, Ltd. * Claremont Colleges Library * Classical Association Of The Middle West And South, Inc. (Camws) * Clawar Association Limited * Clay Minerals Society * Cleo Revues.Org * Cleveland Clinic Journal Of Medicine * Clinical Autonomic Research Society * Clinical Laboratory Publications * Clinics Cardive Publishing * Clockss Archive * Cnps * Cnrs France * Cnu Journal Of Agricultural Science
- 15. Using the DOI as the basis for a common text and data mining API provides several benefits. For example, the DOI provides: •An easy way to de-duplicate documents that may be found on several sites. •Persistent provenance information. •An easy way to document, share and compare coropra without having to exchange the actual documents •A mechanism to ensure the reproducibility of TDM results using the source documents. •A mechanism to track the impact of updates, corrections retractions and withdrawls on corpora. Why use the DOI?
- 16. The TDM Workflow
- 22. CrossRef TDM HTTP Headers CR-TDM-Rate-Limit: 1500 (the rate limit ceiling per window on requests) CR-TDM-Rate-Limit-Remaining: 1387 (number of requests left for the current window) CR-TDM-Rate-Limit-Reset: 1378072800 (the remaining time in UTC epoch seconds before the rate limit resets and a new window is started) *this is a technique used by many APIs, including Twitter’s
- 23. Common API Summary • Content Negotiation (Required) • New Metadata (Required) • Full text URIs • License URIs • Rate Limiting Headers (optional)
- 24. New Metadata
- 25. 1. Full Text Link https://apps.crossref.org/docs/tdm/full-text- uris-technical-details/
- 27. Example from Hindawi <ai:program name="AccessIndicators"> <ai:license_ref>http://creativecommons.org/licenses/by/3.0/</ai:license_ref> </ai:program> <doi_data> <doi>10.1155/2014/969265</doi> <timestamp>20140401090031</timestamp> <resource>http://www.hindawi.com/journals/aaa/2014/969265/</resource> <collection property="text-mining"> <item> <resource mime_type="application/pdf"> http://downloads.hindawi.com/journals/aaa/2014/969265.pdf </resource> </item> <item> <resource mime_type="application/xml"> http://downloads.hindawi.com/journals/aaa/2014/969265.xml </resource> </item>
- 28. Stop here if • You are an open access publisher • You include TDM as a part of your subscription license/T&Cs.
- 31. Researcher View
- 32. Publisher View
- 33. Researcher queries DOI using CN + API token Publisher verifies API token If token verified AND access control allows, publisher returns full text (frequency at publisher discretion)
- 34. Benefits • Streamlines researcher access to distributed full text for TDM • Enables machine-to-machine, automated access for recognized TDM (i.e. researchers won’t be locked out of publisher sites) • Enables article-level licensing info and easy mechanism for supplemental T&Cs for text and data mining (publishers discussing model license via STM)
- 36. Publishers There are two additional metadata elements that publishers will need to deposit to support TDM via CrossRef. These are: •Full Text URIs: One or more URIs that point to full text representations of the content identified by your CrossRef DOIs. •License URIs: One or more URIs pointing at licenses that govern how the full text content can be used. •OPTIONAL: Add publisher TDM terms and conditions to the click-through service
- 37. Researchers • Modify TDM tools to make use of the API token • Modify TDM tools to look for <lic_ref> elements • Register with the click-through service and accept/decline licenses (if applicable)
- 39. Progress to date • DOI content negotiation • CrossRef support for recording links to full text • CrossRef metadata support for: • ORCIDS • FundRef • License information • CrossRef Metadata Search for Discovery: http://search.labs.crossref.org/ • Click-through license service • Publisher API for verifying and managing tokens • Launched as live service 29th May 2014
- 40. Publishers Articles with full-text links and license information deposited: 998,416 Cost? Free to researchers and the public No cost for publishers through 2014, 2015 tbc Register interest at: http://www.crossref.org/tdm/contact_form.html
Notas del editor
- Questions at end. Talk a little bit about what CrossRef is then move on to talk about our text and data mining service.
- First just a few words about CrossRef for anyone who isn’t a member or might not be familiar with us as an organisation. CrossRef is a not-for-profit membership organisation of international scholarly publishers. We have 4000 member publishers, representing all disciplines - not just STM, and comprising commercial publishers, academic societies, open access publishers, university presses. We also have 83 affiliate members and 2000 library affiliates - these libraries and other organisations make use of the CrossRef database to look up DOIs and metadata. We are the largest DOI registration agency and have assigned nearly 63 million DOIs to date.
- CrossRef was founded 14 years ago to solve the problem of broken links. The web is all about links, but links break. This is annoying if you’re browsing the web and want to follow an interesting link, but in the context of scholarly publishing it becomes more than annoying - if you can’t follow a citation from one paper to another you’re being hampered in your research. CItation linking is one of the greatest benefits of online publishing, but it really does need to be reliable
- and publishers were finding that web sites changed, content moved, and links that they had put into their articles stopped working. So they started a multi-publisher initiative to solve this problem of broken links. This is done using the DOI - the Digital Object Identifier, which I’m sure many of you are familiar with. A CrossRef DOI is simply a unique identifier for a piece of content. Once assigned, it doesn’t change. It is to all intents and purposes a meaningless number, but it allows that piece of content to be located on the web.
- And it works like this: publishers use CrossRef DOIs to link to content, usually from the references at the end of articles. Users click on those DOI-based links and are referred via the CrossRef database to the cited article at it’s correct location on the web. If content moves the publisher only has to update the CrossRef database once, and all of the publishers that are linking to their content using CrossRef DOIs will be redirected to the content in its new location.
- Every month there are around 90 million clicks on CrossRef DOI links, so 90 million citations resolved to content.
- The issue of Text and Data Mining has become very important and we feel that CrossRef is in a unique position to expand its current infrastructure (a registry of unique identifiers and metadata for scholarly content and thousands of members) to make TDM easier for researchers and their institutions and publishers. Technical solution - we aren’t addressing the issue of licencing.
- Looking at positives. Finding treatments to diseases that may not have been found before.
- But urge caution – Google Flu!
- Why did CrossRef develop this service? Applies to OA content too. Let’s just illustrate these issues.
- Researcher to illustrate that plus some of the publishers we represent. TDM is about scale.
- Bilateral agreements aspect - In the past, researchers who wish to text and data mine published literature have no common or simple way of accessing the full text for the content they wish to mine. This is true both of subscription-based content as well as of open access content. Consequently, TDM users access the content in one or two ways: Negotiating with publishers to have the content delivered to them, either via physical media or bulk data transfer (e.g. FTP) “Screen-scraping” the publisher’s website. The first option doesn’t scale well across multiple Publishers and Researchers. It also presents synchronisation problems if the researchers want an ongoing feed of refreshed content. The issue with the second option is that “screen scraping” is an inefficient, fragile and error prone mechanism for identifying and downloading full text. Screen scrapers put a large performance burden on web sites and, at the same time, any slight changes to the web site can break the tool that is doing the screen scraping. CrossRef Text and Data Mining provides a common solution which works across Open Access and subscription-based publishers and is free for anyone to use.
- Processing the same document on multiple sites could easily skew text and data mining results and traditional techniques for eliminating duplicates (e.g. hashes, etc.) will not work reliably if the document in question exists in several representations (e.g. PDF, HTML, ePub ) and/or versions (e.g. accepted manuscript, version of record) Using the DOI as a key will allow researchers to retrieve and verify the provenance of the items in the TDM corpus, many years into the future when traditional HTTP URLs will have already broken
- The CrossRef Common API is the main aspect of this service and is designed to allow researchers to easily harvest full text documents from all participating publishers regardless of their business model (e.g. open access, subscription). It makes use of CrossRef DOI content negotiation to provide researchers with links to the full text of content located on the publisher’s site. The publisher remains responsible for actually delivering the full text of the content requested. Thus, open access publishers can simply deliver the requested content while subscription based publishers continue to support subscriptions using their existing access control systems.
- API works with content negotiation – what is content negotiation
- Content negotiation allows a user to request a particular representation of a web resource. DOI resolvers use content negotation to provide different representations of metadata associated with DOIs.A content negotiated request to a DOI resolver is much like a standard HTTP request, except server-driven negotiation will take place based on the list of acceptable content types a client provides. Here, they’re asking for text
- Here they’re asking for XML – and can also request PDF too as we know a lot of publishers may only have back content in PDF and that’s fine.
- Set of standard HTTP headers that can be used by servers to convey rate-limiting information to automated TDM tools. Well-behaved TDM tools can simply look for these headers when they query publisher sites in order to understand how best to adjust their behaviour so as not to effect the performance of the site. The headers allow a publisher to define a “rate limit window”- which is basically a time span (e.g. a minute, and hour, a day).
- In order for researchers to use the CrossRef API, Publishers need to add new metadata to their CrossRef DOI deposits.
- One or more URIs pointing at licenses that govern how the full text content can be used.
- This needs to be added to the publisher XML – license information at the article-level. Examples on our support site.
- Publishers who require researchers to agree to a specific set of Terms and Conditions (T&Cs) before they are allowed to text and data mine content that they otherwise have access to (e.g. through an existing subscription) will need to make use of the click-through service.
- So to put it all together…
- If you are an open access publisher or if your existing subscription licenses already allow TDM of subscribed full text, then the registration of the above metadata deposit is the ONLY thing you need to do in order to enable TDM of your content via the CrossRef Metadata API. Rate limiting.
- Rate limiting too
- Support site with info. Info on rate limiting on there too.
- LAUNCH later this month. More deposits from OA publishers at the moment as it’s easier as their licenses already allow text and data mining. Piloted last year and got approval from the board to move it into Production in November which we’ve done to prepare for launch in March.
- Working group which will migrate to a full CrossRef Committee when the service is officially launched seen over 100,000 deposits of full text links and license information, mainly from Hindawi but some from AIP and IEEE as well.
- Eric Lease Morgan
- Publishers and researchers in pilot. Launch in May