EUDAT Porto Conference 2018

1. www.eudat.euEUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065
EUDAT-EGI collaboration
Welcome and Overview
EUDAT 2018, Porto
Putting the EOSC vision into practice
Michaela BARTH caela@kth.se

2. 2
Joint Access to Data
and HPC Services Commercial
stakeholders
Collaboration with
Commercial
Stakeholders
Joint Access to Data,
HTC and Cloud
Computing Resources
Interoperability
WP 7 Task 7.2
WP 7

3. WP7 Task 7.2: Joint Access to Data, HTC and
Cloud Computing Resources
• EGI-EUDAT collaboration started in March 2015 and officially
continues until end of EUDAT (February 2018).
• Aiming at a production cross-infrastructure service
• provide end-users with a seamless access to an integrated
infrastructure offering both EGI and EUDAT services
• pairing data and high-throughput computing resources together.
• Concrete community pilots
• EPOS
• ICOS
• ENES
• Harmonization on all levels (Technical, Operational, Policies)

4. An end-user driven approach
• Selection of initial set of user communities:
• European Research infrastructures in the fields of
■ Earth Science (EPOS and ICOS),
■ Bioinformatics (BBMRI and ELIXIR) and
■ Space Physics (EISCAT-3D).
• Collecting requirements from user communities
• Getting their indication of prioritization of those requirements
• ⇨ Definition of universal use case
• Demo of the universal use case
• Continuation with user community pilots as early adopters
• Planned joint open call to expand the pilot activity further:
■ Not many new user communities were expected to participate
■ Remaining user communities very productive in providing feedback
■ Instead: Integrate user communities that already previously used both EGI and
EUDAT services, but not yet in combination.

5. • EPOS:
• Fostering worldwide interoperability
in Earth Sciences and provide
services to a broad community of
users
• ICOS:
• Creating web-based service
(“Footprint tool”) at ICOS Carbon
Portal, providing on-demand
computing facilities
• ENES:
• Performing on-demand climate data
analytics for climate research and
climate change impact communities
User community pilots
5

6. Outcomes
• General:
• Valuable feedback to EGI and EUDAT service offerings
• Feedback on data-handling support within the EGI DataHub
• Testing EGI Federated Cloud with automatic submission
• Data transfer tests between the VMs and B2STAGE instances using both OneData
and EGI DataHub to access a common storage for several VMs
• Evaluating the new B2STAGE HTTP API
• Harmonization of access policies
• Agreement on authentication and authorization model (not only
affecting EGI - EUDAT, but AAI Interoperability in general), overview
document created for understanding each other’s AAI layers,
agreements on e.g. RCauth as common link
• Transparent access: See the EGI and EUDAT services as offered by a unique
infrastructure once authenticated
• Access all (web + non-web) EGI and EUDAT services with the same credentials
• Access Delegation from one service to another
• Data privacy considerations and policy harmonisation
• Establishing and revising common roadmap
• Input to AARC questionnaires (still outstanding)

7. Observations
The user-driven approach:
• (+) likely producing the best possible result
• (-) time demanding
• Substantial time and trust investment by the user communities
• steep learning curves
• has to be matched by discipline of the e-infrastructure providers in
• creating the right expectation level on the actual status of the services and
• providing up-to-date documentation
• Pilots are not free beta-users for non-production-ready undocumented
new features eagerly put forward by the developers
• Personal contacts highly appreciated!

8. Further Challenges
• Automatization freedom as unforeseen requirement:
• enabling and facilitating automatization changes the whole design
process for the e-infra services
• clear user feedback influencing the design towards providing added
value to the research communities as end-users of the
e-infrastructure services.
• Technical/Expected challenges
• Co-existing support systems and channels
• Multiplied policy and trust chains
• Scaling up (esp. AAI interoperation)
• 3rd party dependencies and delayed development
• Globus Toolkit GridFTP → B2STAGE HTTPS API
• Support of metadata handling for B2SAFE (GraphDB)
• EGI OneData: Large amount of small files to be used as input for further model runs
problematic

9. Recommendations part I
• Improve documentation:
• From the user’s point of view
• More focused documentation
• Simple recipes
• up2date examples
• up2date
• checking the validation of the documentation should be part of
the release cycle
• Keep documentation in-sync with the current real service offering
(not desired end-vision, not outdated)
• Remove old invalid links

10. Recommendations part II
• Saving time when on-boarding further user communities
• Establish a true common understanding
• Goal: What is to be achieved?
• What tools are already there?
• What resources will be needed (people, time,...)?
• Boundary conditions (platforms, software,..)?
• For all services, platforms and applications:
• How do they work?
• How do they interact?
• Benefits of using them (compared to similar services)?
• Appoint a project enabler (in this case should have been even two: one
from EGI, one from EUDAT)
• Appoint a direct contact person acting as consulting expert for each
of the offered services
• Provide simple entry-point step-by-step guides
• for typical alternative scenarios (based on different underlying service portfolio)

11. Read on
• EUDAT:
• EUDAT2020-DEL-WP7-D7.1. describing initial EGI-EUDAT pilot activity
up to definition of universal use case
• EUDAT2020-DEL-WP7-D7.4. Deliverable D7.4 Final Report EUDAT/EGI
(currently under review, later on https://eudat.eu/deliverables)
• Corresponding reports focusing on the EGI part of the work (also
with input from EUDAT):
• EGI Engage Deliverable D4.8 “Cross-infrastructure case studies
report”
• EGI Engage Deliverable D4.9 “Open Data Platform: Demonstrator,
Experience Report and Use Cases”

12. www.eudat.euEUDAT receives funding from the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065
Thank you and Welcome!
Michaela BARTH caela@kth.se