A Critique of the Proposed National Education Policy Reform
10 years Agricultural Ontology Initiative: Building Blocks for a Linked Data Infrastructure
1. 10 years Agricultural
Ontology Initiative:
Building Blocks for a
Linked Data Infrastructure
Dr. Johannes Keizer
FAO of the United Nations
Office of Knowledge Exchange, Research and Extension
Team Leader “Knowledge Standards and Services”
10/2010
2. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
The Internet!
3. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
4. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Aggregation States of Knowledge
5. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Data Flows and Repositories in
Research
6. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
“... FAO’s principle task is to work
to ensure that the world’s
knowledge of food and
agriculture is available to those
who need it when they need it
and in a form which they can
access and use ...”
7. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
AOS Vision in 2001
8. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
..from thesaurus to Ontologies….
9. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Our push of AGROVOC to the Semantic Web had
enormous positive effects, among others
From 4 to 20 language versions
Defacto standard for indexing in many areas
More than 2000 downloads only in 2009
SKOS incorporated all our requirements
For many purposes we need semantics on a lower
level than of a fully elaborated ontology – but we
need them urgently
The development of specific Ontologies should be
always application driven – a demand economy
Lessons Learned
10. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
AOS - Today
SemanticsTools
Linked DataCommunity
11. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Community
12. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
The AOS Community
13. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
http://aims.fao.org/community/home
14. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Better Semantics
15. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
around 30,000 concepts
600000 labels in around 20 languages.
one-stop shop for terminological knowledge related to
agriculture in general
a knowledge base of related concepts organized in
ontological relationships (hierarchical, associative,
equivalence)
Is a concept/term/string based system
Concepts may be organized in multiple categories.
AGROVOC concept scheme
16. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Further schemes
in FAO
skos:broader
:bar
has_synonym
has_translation
skos:literalForm “maize”
:foo
maïs (fr)
:foo
has_synonym
skos:literalForm “corn”
:bar
8171
1474
skosxl:altLabel
skosxl:prefLabel
skos:broader
has_synonym
SKOS Label
The AGROVOC concept scheme
SKOS
Concept
rdf:type
rdf:type
6211
skos:broader
AGROVOC
Concept
Scheme
skos:topConceptOf
skos:inScheme
Another scheme
in FAO
Other scheme
in FAO
skos:inScheme
12332
17. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Work Expression Manifestation Item
Subjects
Corporate
Bodies
Conferences Journals
Series
FAO Projects
FAO FRBR Model & Authority Data
18. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
isSpatiallyIncludedInisSpatiallyIncludedIn
isPublishedByisPublishedBy
isOtherLanguageEditionOfisOtherLanguageEditionOf
isFollowedBy/FollowsisFollowedBy/Follows
EXAMPLE: JOURNAL CONTENT
MODEL
19. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
A Model to create Linked Data
Rich set of relationships
Easier implementation of concept-based thesauri and
authority data
20. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Geopolitical Ontology
OEKM FAO of the UN 20
21. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Fishery Ontologies
22. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
What Partners have done…..
Rice Knowledge Models
Rice Production Ontology
ASFA Thesaurus
VIVO Ontologies
Look to the many presentations on this
workshop
23. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Tools
24. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
The Concept Scheme Work Bench
25. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Drupal
AgriDrupal is a “suite of solutions” for agricultural information
management and dissemination, built on the Drupal CMS by different
Institutions and individuals who are now sharing their experiences in the
AgriDrupal community
Drupal, a semantic web enabled CMS
General purpose CMS necessary
Drupal has flexibility to manage all information types
Drupal has a strong user community
Version 7 natively with RDF backbone, but semantic applications
already possible with version 6
Drupal can be a producer and consumer of Linked Data
26. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Drupal Query run on a Drupal website
from a Virtuoso test environment
at http://demo.openlinksw.com/sparql_demo/
SPARQL endpoint
RDF triples
Drupal website
27. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
• Objectives:
• Assure quality in metadata creation
• Sharing information in a standardized manner
• Use of common semantics and interoperable
syntaxes
• Use of more sophisticated and specialized
metadata
• Use of controlled, multilingual vocabularies
• Requirements:
• AGRIS AP compliancy
• AGROVOC
AgriOcean Dspace – a tool for Repositories
28. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
AgriOceanDspace – Thesaurus plug in
• Developed by Kasetsart University (Bangkok,
Thailand)
• Thesaurus plug-in
o
Web services: use local or remote version of AGROVOC
thesaurus/SKOS
29. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Linking Data
30. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
The Linked Data Universe:
http://www.linkeddata.org (july 2010)
31. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
http://agris.fao.org/agris-search/search/display.do?f=2004/ZA/ZA04002.xml;ZA2004000049
32. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
AGROVOC EUROVOC UNBIS Relationship
http://aims.fao.
org/aos/agrovoc
/c_207
http://eurovoc
.europa.eu/21
9055
agroforestry skos:exactMatch
/ owl:sameAs
http://aims.fao.
org/aos/agrovoc
/c_4826
http://eurovoc
.europa.eu/22
0018
MILK skos:exactMatch
/ owl:sameAs
http://aims.fao.
org/aos/agrovoc
/c_12332
http://eurovoc
.europa.eu/21
9871
MAIZE skos:exactMatch
/ owl:sameAs
Linking vocabularies
33. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
http://aims.fao.org/aos/agrovoc/c_7825
http://eurovoc.europa.eu/218754
34. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
http://eurovoc.europa.eu/
219871
Maize
skosxl: literalForm
Maize
http://aims.fao.org/ao
s/agrovoc/c_12332
AGROVOC
skosxl: literalForm
Maize
http://aims.fao.org/aos/agrovoc/c_12332 owl:sameAs http://eurovoc.europa.eu/219871
owl:sameAs/exactMatch
http://agris.fao.org/agris-
search/search/display.do?
f=1996/TR/TR96001.xml;TR96000
26
Linking data through common URIs
skosxl: literalForm
owl:sameAs/exactMatch
http://eur-
lex.europa.eu/LexUriServ/LexUriSe
rv.do?
uri=OJ:L:2010:202:0011:0015:EN:
PDF
http://unbisnet.un.org:8080/ipac20/ipac.j
sp?
session=128F308557F34.283092&profil
e=bib&uri=full=3100001~!685149~!
1&ri=1&aspect=subtab124&menu=searc
h&source=~!horizon
Maize
Eurovoc
UNBIS
35. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
What are we doing with unstructured data?
• We have enormous amounts of unstructured
material
• Still most of the documents that we are producing
are mostly semantically unstructured
• Human work to catalogue and index is becoming
always more rare
• We need machines to do automatic semantic mark
ups of text
• If machines are trained and based on concept
schemes, ther are able to do so
36. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
37. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
• Does Concept identification in unstructured
texts
• Uses Agrovoc as a controlled vocabulary
• Prototype under testing with excellent results
(entire repository of ICARDA indexed)
• Will produce in future Structured RDF files
that can be used to link data like “open
Calais”
•
AgroTagger
38. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
39. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
40. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
41. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
Life Demo: Semantic mark ups:
http://viewer.opencalais.com/
http://agropedialabs.iitk.ac.in/Tagger/Agrotagger_text.ph
p
42. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
The CIARD RING
43. dr johannes keizer - FAO of the United Nations - knowledge and capacity for development
Chinese Agricultural Ontology Service WS, Beijing 2010-10-30
AGRIS Linked Data
This is the AGROVOC SKOS model that has been developed and decided in April 2010 under active collaboration from Tom Baker, who was member of the W3C SKOS working group.
AgriOcean Dspace release: 09/2010
Setting up of AgriOcean Dspace Community on AIMS: 09-10/2010
exchange of experiences
Pilot Testing: “Bangladesh Agricultural Universities’ Institutional Repository”
Cooperation with DURASPACE
Informing about the new AgriOcean Dspace implementations
Collaborating in the Dspace Ambassadors Program
This is a snapshot one year later. The growth is enormous. A central point is DBPedia, “triplified” information from Wikipedia. The different colours represent the different information types, being “life sciences” and “publications” the most populated areas, but with the area “government” strongly growing
Interesting newcomers in the last months are the two VIVO datasets from the United States descriping expertise in Science. Vivo is actually a project that started the agricultural library of Cornell University
In a bibliographical record there is much more hidden information than displayed with the metadata. Many of the highly structured data are linking to other information on the web. In AGRIS we have now introduced something what we call “naive linking”. An AGRIS record links automatically to Google Maps for the location of the center and to Google to retrieve the full text of the resource, citation lists or other publications from the authors.
This often works, but clearly not alway, s as it is not controlled by semantics, but only through identy of strings. For an uneducated machine unfortunately COW and C.O.W. are the same, whereas peanuts and groundnuts are something different.
The table shows 3 descriptors that are in AGROVOC, EUROVOC and UNBIS. In AGROVOC and EUROVOC they are already encoded as URIs. Easily we could establish relationships like owl.sameAs between the concepts or skos:exactMatch between labels.
If resources are marked up with semantically defined and machine readable concepts, they can be linked and mashed up precisely as we have seen in the example from the BBC.
In this example we start with an AGRIS record on Hazardous waste, which is indexed with AGROVOC. Already now we can easily link to material indexed with Eurovoc, here an example from EuroLex. If the UNBIS thesaurus would be restructured to a concept scheme and published as LOD, related UN documents could be attached automatically by the machine.
How does this work: A resource is connected with each concept URI in the web. The concepts between three vocabularies are having same literal which is connected with owl:sameAS/exactMatch relationship. As we are speaking about thesauri and not ontologies we kept the relation to be chosen purposely vague. The concepts could be matched with owl:sameAS or the terms could be matches with SKOS:exactMatch. A lot of discussion on this is ongoing
One of the groundbreaking enterprises in this area is Thomson Reuters “Open Calais”. This is a webservice that provides semantic mark up for any unstructured text that you feed into their service The service is free of Charge. Why? I will show you later.
My team in collaboration with the Indian Institute of Technology in Kanpur is developing a similar service for our subject area.
We have here a text from 1964 without a bibliographic record at hand about a plant protection issue
Open Calais is very good in those areas, in which they have their own elaborated conceptscheme against which the texts are analyzed: “Places”, “Persons”, “Business Processes” , “Industry Terms”, but it is weak in the specific topic analysis, what they call “social tags”
AgroTagger still lacks many of the sophisticated features of “Open Calais” , but is much, much better in the subject analysis of the text