The document describes the EGI Open Data Platform, which enables easy discovery, access, and usage of reference open data and user data. It federates existing data sources and storage providers, brings data to multiple hybrid clouds, and provides discovery and publishing of data. Key features include being a new data as a service offering based on Onedata; cross-domain federation of storage providers; and a central means for accessing open data. It benefits users by increasing data accessibility, bringing data to computing resources, and providing integrated services like federation and authentication.

1. www.egi.eu
EGI-Engage is co-funded by the Horizon 2020 Framework Programme
of the European Union under grant number 654142
Open Data Platform
Matthew Viljoen
matthew.viljoen@egi.eu
EGI Operations Team

2. 224/01/2018 EGI Open Data Platform
A solution to enable the easy discovery, access and
usage of reference open data and user data
The first open source means to:
– Federating existing data sources and data storage
providers
– Bringing data to the multiple hybrid clouds
– Discovery and publish data
EGI Open Data Platform Description

3. 324/01/2018 EGI Open Data Platform
• New Data as a Service offering from EGI based on Onedata, developed
under EGI-Engage, INDIGO-DataCloud and HelixNebula
• Cross-domain federation of existing storage providers
• Central means for accessing open data
Storage
provider
Domain level
open data
repository
Generic open
data repository
(e.g. AWS)
EGI
FedCloud
User browsing data
V
M
User analyzing data
Introducing the EGI DataHub

4. 424/01/2018 EGI Open Data Platform
• Same technology as the EGI DataHub
• Brings user data to the clouds
• Enables storage providers to easily expose their storage
Storage
provider
V
M
Domain level
open data
repository
Generic open
data repository
(e.g. AWS)
Storage
provider 2
Storage
provider 3
EGI
FedCloud
User browsing data User analyzing data
And the EGI Federated Data Gateway

5. 524/01/2018 EGI Open Data Platform
Private cloud 1
Institutional
cloud 2
Public cloud
V
M
V
M
V
M
V
M
V
M
V
M
V
M
V
M
Shared storage
POSIX/S3 POSIX/S3 POSIX/S3
Bringing data to computing

6. 624/01/2018 EGI Open Data Platform
Domain level
open data repository
EGI FedCloud
V
M
V
M
V
M
Shared storage
POSIX/S3
From isolated domain level data
repository with its users to:
• open discovery by new users
via EGI DataHub
• exploitation of data on EGI
FedCloud
• integrated EGI Check-in
(OpenID Connect, social
media, Token Translation
Service)
• publishing and long term
data archiving
Publishing, discovery and using open data

7. 724/01/2018 EGI Open Data Platform
What benefits does it bring?
• Increased accessibility of data to users
• Bringing data to computing
• Integrated services (federation of data providers,
multiple data access, publishing/DOIs and AAI)
• Scalability

8. 824/01/2018 EGI Open Data Platform
Target users
Who can benefit?
• End users (inc. scientists, researchers, end users)
– accessing/using data
• Industry and SMEs
– services based on data exploitation
• Society and citizens
– increased data reuse = more research
• Service and resource providers
– easier use of cloud and data

9. 924/01/2018 EGI Open Data Platform
Use cases
• ICOS (EGI-EUDAT Joint Usecase)
• Earth Observation Proof of Concept (EGI-Engage)
• PanCancer (EGI + EOSCpilot)
• Molecular Dynamics/INSTRUCT/LHC-CMS ...
(INDIGO-DataCloud)
• Industry e.g. T-Systems (INDIGO-DataCloud)

10. 1024/01/2018 EGI Open Data Platform
Benefits from working with communities
• Validating technology against the real work
• Bugs identifications
• Suggestions for documentation improvements
• Guiding integration activities
• Invaluable user feedback

11. 1124/01/2018 EGI Open Data Platform
Demo
1. Access data via EGI DataHub
2. Instantiating a VM via the AppDB VMops
Dashboard
3. Setting up a Onedata Oneclient via Docker
4. Access the same data via FedCloud
https://www.youtube.com/watch?v=sc5MsOkvgFQ
Developed by EGI as part of INDIGO-DataCloud

12. www.egi.eu
Thank you for your attention.
Questions?
This work by Parties of the EGI-Engage Consortium is licensed under a
Creative Commons Attribution 4.0 International License.