Importing life science at a into Neo4j

Simon Jupp
Simon JuppEuropean Bioinformatics Institute
Importing linked life science databases into
Neo4j
Simon Jupp
Sample Phenotypes and Ontologies Team
European Bioinformatics Institute
jupp@ebi.ac.uk
Purpose of the workshop
• Introduce two alternate graph models
• RDF graphs
• Property graph
• Demonstrate a simple data integration use-case
• Show how Neo4j data import tools can be used to rapidly
import life science data from public APIs
• Example Cypher for querying biological data
• Introduction to Neo4j sandboxes, Apoc procedures and
tips for creating your own Neo4j guide
Some biological questions
“Differentially expressed genes in adult mice, bred in oxygen rich vs
oxygen poor environments? Of this set, which biological processes
(GO) are enriched?”
“Where are genes with antigen binding function differentially
expressed, which disease and which associated pathways?”
“Get metformin associated pathways with differentially expressed
genes, find any proteins that are targets for known diabetes drugs”
How do you go about answering these kinds of questions?...
… you go to the data
Literature & ontologies
•Experimental Factor Ontology
•Gene Ontology
•BioStudies
•Europe PMC
Chemical biology
•ChEBI
•ChEMBL
•SureChEMBL
Molecular structures
•Protein Data Bank in Europe
•Electron Microscopy Data Bank
Gene, protein & metabolite expression
•Expression Atlas
•Metabolights
•PRIDE
•RNA Central
Protein sequences,
families & motifs
•InterPro
•Pfam
•UniProt
Genes, genomes & variation
•Ensembl
•Ensembl Genomes
•GWAS Catalog
•Metagenomics portal
Systems
•BioModels
•BioSamples
•Enzyme Portal
•IntAct
•Reactome
Molecular Archives
•European Nucleotide Archive
•European Variation Archive
•European Genome-phenome Archive
•ArrayExpress
Data integration challenges
• Heterogeneous formats and identifiers
• We invest heavily in mapping and cross-linking
resources, but it’s still hard to integrate and query across
internal/external resources.
• Lots of effort doing mapping, each groups duplicate these
efforts
Standardise data publishing
• What is we could standardise the way we publish data?
• Global identification systems (so we can identify the things
in our data)
• Common semantics (talking about the same things)
• A common query language to the data
Original vision of the Web
Information Management: A Proposal, Tim Berners-Lee, CERN, March 1989, May 1990,
http://www.w3.org/History/1989/proposal.html
Relations
“Things”
Vocabularies
Early Web
Semantic Web ( or Linked Data)
"The Semantic Web is a webby way to
link data"
“Turning the web into a global API”
“The existing web links documents, the
semantic web links data”
“Shared meaning through ontologies”
The Linking Open Data cloud 2017
http://lod-cloud.net
RDF is for describing graphs
• 1995-2004 W3C develop specification for a vocabulary
for Web meta-data called Resource Description
Framework (RDF)
http://en.wikipedia.org/wiki/Barack_Obama
Web
Document
Structured
dataPublishing data as a graph
dbpedia:Barack_Obama
Human President of the United States
Honolulu
1961-08-04
birthplace
birthdate
position_held
type
Anatomy of a triple statement
• All triples are composed of a subject, predicate and an
object
Barack Obama
Honolulu
birth place
Subject
Predicate
Object
Identify things on the web
• Build on existing Web technology
• global identifiers for resources (things) using URIs
• URIs should resolve
http://dbpedia.org/page/Barack_Obama
http://dbpedia.org/page/Honolulu
http://dbpedia.org/property/birthPlace
Subject
Predicate
Object
Turning relational data to RDF –
EBI Gene Expression Atlas database
Relational Data to RDF graph conversion
•Give “things” URIs
•Type “things” with ontologies
•Link “things” to other related “things”
Stardog
Apache Jena
SesameVirtuoso
Allegrograph
OWLIM
Storing and querying RDF
• Optimized databases for RDF data
• SPARQL query language
Querying RDF with SPARQL
• W3C standard query language for querying RDF data
• Query language for matching graph patterns in RDF
• SPARQL endpoints – common API to query RDF data
• ”Get all presidents of the united states?” from
https://query.wikidata.org/
PREFIX position_held: http://www.wikidata.org/prop/direct/P39
PREFIX potus: http://www.wikidata.org/entity/Q11696
SELECT ?label WHERE {
?subject position_held: potus: .
?subject rdfs:label ?label .
filter (lang(?label) = "en")
}
RDF and the Property graph
RDF graphs
dbpedia:Barack_Obama
Human President of the United States
Honolulu
“1961-08-04”xsd:datetime
birthplace
birthdate
position_held
type
Every statement adds a new edge to the graph
All nodes are resources (with URIs) or literals (with types)
Property graphs (Neo4j)
“Barack_Obama”xsd:string
name
dbpedia:Barack_Obam
a
{
name: “Barack Obama”
Type: “Human”
}
Nodes and edges have internal structure
Honolulu
Birthplace
{ Birthdate: “1961-08-04” }
Working with RDF and Neo4j
• RDF great for publishing data
• SPARQL gives flexible access to query data
• But…
• RDF schemas are often (necessarily) complex
• Expose the full underlying data semantics
• RDF comes with baggage that can be turn off for newomers
• Neo4j is for graphs
• Easier to grasp for begniners
• Powerful query language (Cypher)
• Excellent third-party tools, community and developer
integrations
Working with RDF and Neo
• In this tutorial we will harness the publishing power of
RDF
• Combine with the simplicity and querying power of Neo4j
• Use Neo4j data import tools to rapidly import data from
public SPARQL endpoints
• Simplify the graph schema to fit a specific use case
Use-case
• Build a simple graph of gene-disease and drug-disease
associations
• Data from public resources (Ensembl, GWAS, ChEMBL)
Setup for workshop
• Sandbox Neo4j instance from https://neo4j.com/sandbox-
v2/
• Optionally run your own local installation, but you’ll need
Apoc procedures installed to run
• Run the Neo4j guide
:play https://guides.neo4j.com/life-science-import
1 de 19

Más contenido relacionado

La actualidad más candente(20)

Linking Data, Linking PeopleLinking Data, Linking People
Linking Data, Linking People
fereiraJ787 vistas
Neo4j and bioinformaticsNeo4j and bioinformatics
Neo4j and bioinformatics
Pablo Pareja Tobes5.1K vistas
20130622 okfn hackathon t220130622 okfn hackathon t2
20130622 okfn hackathon t2
Seonho Kim505 vistas
Semantic Technologies in ST&DLSemantic Technologies in ST&DL
Semantic Technologies in ST&DL
Andrea Nuzzolese366 vistas
OEG-Tools for supporting Ontology EngineeringOEG-Tools for supporting Ontology Engineering
OEG-Tools for supporting Ontology Engineering
María Poveda Villalón639 vistas
OkeOke
Oke
Andrea Nuzzolese1.1K vistas
Paul GrothPaul Groth
Paul Groth
Connected Data World249 vistas
Metadata in the BioSample Online Repository are Impaired by Numerous Anomalie...Metadata in the BioSample Online Repository are Impaired by Numerous Anomalie...
Metadata in the BioSample Online Repository are Impaired by Numerous Anomalie...
CEDAR: Center for Expanded Data Annotation and Retrieval132 vistas
Embracing Semantic Technology for Better Metadata Authoring in Biomedicine (S...Embracing Semantic Technology for Better Metadata Authoring in Biomedicine (S...
Embracing Semantic Technology for Better Metadata Authoring in Biomedicine (S...
CEDAR: Center for Expanded Data Annotation and Retrieval521 vistas
Bio4jBio4j
Bio4j
Pablo Pareja Tobes3K vistas
An Open Repository Model for Acquiring Knowledge About Scientific ExperimentsAn Open Repository Model for Acquiring Knowledge About Scientific Experiments
An Open Repository Model for Acquiring Knowledge About Scientific Experiments
CEDAR: Center for Expanded Data Annotation and Retrieval108 vistas
Genome science intermineGenome science intermine
Genome science intermine
ELIXIR UK997 vistas

Similar a Importing life science at a into Neo4j

Linked Open DataLinked Open Data
Linked Open DataLaura Hollink
254 vistas144 diapositivas
Linked DataLinked Data
Linked DataAnja Jentzsch
653 vistas52 diapositivas
Introduction to linked dataIntroduction to linked data
Introduction to linked dataLaura Po
1.1K vistas75 diapositivas

Similar a Importing life science at a into Neo4j(20)

Open data and linked dataOpen data and linked data
Open data and linked data
Marie Gustafsson Friberger800 vistas
Linked Open DataLinked Open Data
Linked Open Data
Laura Hollink254 vistas
Linked DataLinked Data
Linked Data
Anja Jentzsch653 vistas
Introduction to linked dataIntroduction to linked data
Introduction to linked data
Laura Po1.1K vistas
20141112 courtot big_datasemwebontologies20141112 courtot big_datasemwebontologies
20141112 courtot big_datasemwebontologies
Melanie Courtot901 vistas
Data Designed for DiscoveryData Designed for Discovery
Data Designed for Discovery
OCLC1.1K vistas
20110728 datalift-rpi-troy20110728 datalift-rpi-troy
20110728 datalift-rpi-troy
François Scharffe857 vistas
20140521 sem-tech-biz-guest-lecture20140521 sem-tech-biz-guest-lecture
20140521 sem-tech-biz-guest-lecture
Vladimir Alexiev, PhD, PMP729 vistas
Metadata for researchers Metadata for researchers
Metadata for researchers
Getaneh Alemu242 vistas
LOD/LAM PresentationLOD/LAM Presentation
LOD/LAM Presentation
Hafabe553 vistas

Importing life science at a into Neo4j

  • 1. Importing linked life science databases into Neo4j Simon Jupp Sample Phenotypes and Ontologies Team European Bioinformatics Institute jupp@ebi.ac.uk
  • 2. Purpose of the workshop • Introduce two alternate graph models • RDF graphs • Property graph • Demonstrate a simple data integration use-case • Show how Neo4j data import tools can be used to rapidly import life science data from public APIs • Example Cypher for querying biological data • Introduction to Neo4j sandboxes, Apoc procedures and tips for creating your own Neo4j guide
  • 3. Some biological questions “Differentially expressed genes in adult mice, bred in oxygen rich vs oxygen poor environments? Of this set, which biological processes (GO) are enriched?” “Where are genes with antigen binding function differentially expressed, which disease and which associated pathways?” “Get metformin associated pathways with differentially expressed genes, find any proteins that are targets for known diabetes drugs” How do you go about answering these kinds of questions?...
  • 4. … you go to the data Literature & ontologies •Experimental Factor Ontology •Gene Ontology •BioStudies •Europe PMC Chemical biology •ChEBI •ChEMBL •SureChEMBL Molecular structures •Protein Data Bank in Europe •Electron Microscopy Data Bank Gene, protein & metabolite expression •Expression Atlas •Metabolights •PRIDE •RNA Central Protein sequences, families & motifs •InterPro •Pfam •UniProt Genes, genomes & variation •Ensembl •Ensembl Genomes •GWAS Catalog •Metagenomics portal Systems •BioModels •BioSamples •Enzyme Portal •IntAct •Reactome Molecular Archives •European Nucleotide Archive •European Variation Archive •European Genome-phenome Archive •ArrayExpress
  • 5. Data integration challenges • Heterogeneous formats and identifiers • We invest heavily in mapping and cross-linking resources, but it’s still hard to integrate and query across internal/external resources. • Lots of effort doing mapping, each groups duplicate these efforts
  • 6. Standardise data publishing • What is we could standardise the way we publish data? • Global identification systems (so we can identify the things in our data) • Common semantics (talking about the same things) • A common query language to the data
  • 7. Original vision of the Web Information Management: A Proposal, Tim Berners-Lee, CERN, March 1989, May 1990, http://www.w3.org/History/1989/proposal.html Relations “Things” Vocabularies Early Web
  • 8. Semantic Web ( or Linked Data) "The Semantic Web is a webby way to link data" “Turning the web into a global API” “The existing web links documents, the semantic web links data” “Shared meaning through ontologies” The Linking Open Data cloud 2017 http://lod-cloud.net
  • 9. RDF is for describing graphs • 1995-2004 W3C develop specification for a vocabulary for Web meta-data called Resource Description Framework (RDF) http://en.wikipedia.org/wiki/Barack_Obama Web Document Structured dataPublishing data as a graph dbpedia:Barack_Obama Human President of the United States Honolulu 1961-08-04 birthplace birthdate position_held type
  • 10. Anatomy of a triple statement • All triples are composed of a subject, predicate and an object Barack Obama Honolulu birth place Subject Predicate Object
  • 11. Identify things on the web • Build on existing Web technology • global identifiers for resources (things) using URIs • URIs should resolve http://dbpedia.org/page/Barack_Obama http://dbpedia.org/page/Honolulu http://dbpedia.org/property/birthPlace Subject Predicate Object
  • 12. Turning relational data to RDF – EBI Gene Expression Atlas database Relational Data to RDF graph conversion •Give “things” URIs •Type “things” with ontologies •Link “things” to other related “things”
  • 13. Stardog Apache Jena SesameVirtuoso Allegrograph OWLIM Storing and querying RDF • Optimized databases for RDF data • SPARQL query language
  • 14. Querying RDF with SPARQL • W3C standard query language for querying RDF data • Query language for matching graph patterns in RDF • SPARQL endpoints – common API to query RDF data • ”Get all presidents of the united states?” from https://query.wikidata.org/ PREFIX position_held: http://www.wikidata.org/prop/direct/P39 PREFIX potus: http://www.wikidata.org/entity/Q11696 SELECT ?label WHERE { ?subject position_held: potus: . ?subject rdfs:label ?label . filter (lang(?label) = "en") }
  • 15. RDF and the Property graph RDF graphs dbpedia:Barack_Obama Human President of the United States Honolulu “1961-08-04”xsd:datetime birthplace birthdate position_held type Every statement adds a new edge to the graph All nodes are resources (with URIs) or literals (with types) Property graphs (Neo4j) “Barack_Obama”xsd:string name dbpedia:Barack_Obam a { name: “Barack Obama” Type: “Human” } Nodes and edges have internal structure Honolulu Birthplace { Birthdate: “1961-08-04” }
  • 16. Working with RDF and Neo4j • RDF great for publishing data • SPARQL gives flexible access to query data • But… • RDF schemas are often (necessarily) complex • Expose the full underlying data semantics • RDF comes with baggage that can be turn off for newomers • Neo4j is for graphs • Easier to grasp for begniners • Powerful query language (Cypher) • Excellent third-party tools, community and developer integrations
  • 17. Working with RDF and Neo • In this tutorial we will harness the publishing power of RDF • Combine with the simplicity and querying power of Neo4j • Use Neo4j data import tools to rapidly import data from public SPARQL endpoints • Simplify the graph schema to fit a specific use case
  • 18. Use-case • Build a simple graph of gene-disease and drug-disease associations • Data from public resources (Ensembl, GWAS, ChEMBL)
  • 19. Setup for workshop • Sandbox Neo4j instance from https://neo4j.com/sandbox- v2/ • Optionally run your own local installation, but you’ll need Apoc procedures installed to run • Run the Neo4j guide :play https://guides.neo4j.com/life-science-import

Notas del editor

  1. The slide shows the core resources at the EBI to show the range of data you can access through the EBI.