Dynamic On Demand Analysis Service

1. eosc-hub.eu
@EOSC_eu
EOSC-hub receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777536.
Dynamic On Demand Analysis Service
DODAS

2. 2
Motivation and driving consideration about the service
Service architecture and interfaces: overview
- How the user can access the service
 E.g.: REST, GUI, CLIs, etc.
- Service options and attributes
Access policy and business model
Use cases
Documentation/tutorial/information
11/30/2018
Content

3. 3
Platform as a Service tool to deploy complex and intricate
setup on “any cloud provider” with almost zero effort.
Allows to instantiate on-demand container based cluster to
execute software applications such as:
Batch System as a Service
Machine Learning as a Service
BigData Platforms
Guarantees:
Dynamic extension of existing facilities
Multi cloud cluster deployment
11/30/2018
Motivation

4. 411/30/2018
Service architecture and interfaces
4
Home IdP
AuthN
DODAS
User
IA
M
PaaS
Orchestrator
IM
Private Cloud Public Cloud
Marathon Maste
r
Maste
r
Maste
rSlave
ExecutorExecutorExecutor
App App
Slave
ExecutorExecutorExecutor
MarathonMaste
r
Maste
r
Master
Slave
ExecutorExecutorExecutor
Slave
ExecutorExecutorExecutor
DataAnalysts
Software
Application
App App
Software
Application
PaaSIaaS

5. 5
DODAS access is required to require cluster generation
and software application deployment.
- User require such cluster through the PaaS Orchestrator
client interface
Users (e.g. data analysts) access generated clusters
either through GUIs and/or REST APIs
User access to the software application depend on the
specific case
- E.g. a regular batch as a services implemented through
HTCondor will grant access through CLI
11/30/2018
Service access

6. 6
Service options relate both to Storage and Compute
- Data & Storage service:
 Online Storage is supported
- Service attributes:
 Storage capacity (GB), Start of Service (date), Duration (num.
of days)
- Compute service:
 Service attributes:
 Number of Cores, RAM (GB), Start of Service (date), Duration
(num. of days)
11/30/2018
Service options and attributes

7. 7
Access policies:
- Policy-based,
Payment model:
- Market-driven
11/30/2018
Access policies and Funding models

8. 811/30/2018
Featured use cases:
CMS Opportunistic Computing
Squid Proxy
CVMFS
CVMFS
CVMFS
Master
Load
Balancer
SlaveSlave
Slave
HTCondo
r
HTCondo
r
HTCondo
r
HTCondo
r
HTCondo
r
CMS
Distributed
Storages
 Completely transparent to CMS
physicists
 Seamlessly integrating the global
infrastructure
CertCach
e
DODAS
ephemeral site
Token Translation
X.509
OpenID Connect

9. 911/30/2018
Opportunistic Resources
DODAS within top 6
CMS Tier2s for 10
days
CMS users analysis: hh—>2b2tau
Skimming minAOD (Data & MC) to
produce flat-ntuple
A DODAS
generated CMS-
Site
 Bringing a huge amount of resources with almost zero effort

10. 1011/30/2018
Featured use cases:
AMS Batch System as a Service
IAM
AuthN
DODAS
Schedd
Collector
Negotiato
r
Batch System
Remote Storage
CERN
INFN-CNAF
Data
Cache/
Local
Storage
startd
 A spot batch system to analyze AMS satellite data

11. 11
User documentation:
- https://dodas-ts.github.io/dodas-doc/
Tutorial:
- https://dodas-ts.github.io/SOSC-2018/
11/30/2018
Documentations

12. eosc-hub.eu @EOSC_eu