1. Mitglied der Helmholtz-Gemeinschaft
UNICORE Server Components -
Detailed View
07/07/2009 Bastian Demuth
b.demuth@fz-juelich.de

2. Job Submission: Software Layers
Client
https “web service firewall”,
message authentication
and forwarding
Gateway
Web Services
https
Service: coherent chunk
of functionality exposed
(WSRF)
through a web-service
interface
Security
Atomic Additional
Services Services
Service Container
Execution Management (XNJS)
Target System Interface (TSI)
non WS
(batch) execution
systems, file systems,
databases, ...
Target systems
07/07/2009 Slide 2

3. Deployment Scenario: Workflow Services
Client
lookup Gateway
Global
Registry
lookup,
create TSS, Service
submit job, Container
transfer file
Gateway Gateway
UAS Local UAS Local UAS Local
Registry Registry Registry
Service Service Service
Container Container Container
lookup
XUUDB user XUUDB
07/07/2009 Slide 3

4. WSRF
 Web Services Resource Framework
 WS Resource
■ Stateful web service
■ Represented by an XML document
■ Resource properties
■ Standard methods: getter, setter, queries
■ Lifetime
 Service Group
■ List of WS addresses
■ Used for Registry
 WS-BaseFaults
07/07/2009 Slide 4

5. Configuration
 Service Container
■ Web Services to be deployed
■ Address of the shared Registry
■ XUUDB address, “Grid Component ID“
■ Gateway address
 Gateway
Everybody:
■ Connection list Security settings
(Keystore, certificate, ...)
 Registry
■ Lifetime for entries
 Client
■ Registry Address
07/07/2009 Slide 5

6. UNICORE Atomic Services (UAS)
Gateway
map grid
users
Target System Service Container to local
Factory (TSF) users
UNICORE Site
Target System Target System
Service (TSS) Service (TSS)
XUUDB
Security
Job Mgmnt Job Mgmnt Storage Mgmnt
Service (JMS) Service (JMS) Service (SMS)
Target System Interface (TSI) Key:
Storage Mgmnt Storage Mgmnt File Transfer reference
Service (SMS) Service (SMS) Service (FTS) file transfer
lookup
07/07/2009 Slide 6

7. UAS: Target System Factory Service
Target
1. createTSS System
Factory
3. return TSS
address
2. create
Target
Client 4. use TSS System
Service
07/07/2009 Slide 7

8. UAS: Target System Service
 Abstract web service interface to target system
■ List of applications
■ Links to jobs and storages (e.g. user home)
 Security
■ User authentication through XUUDB
■ Authorization: Users' target system instances and jobs are
protected by configurable XACML policy
■ Secure job submission through message signing
 Extensibility
■ Virtualization
■ Exclusive resource reservation
07/07/2009 Slide 8

9. UAS: Job Management Service
 Abstract web service interface to submitted jobs
■ Jobs can be accessed and controlled from anywhere
 Job status (queued, running, finished, failed, ...)
 Link to storage that represents the working directory (uspace)
■ Used to securely access output files
 Detailed execution log, exit code of the application
 Applications are abstracted: path of executable invisible
 Provide a copy of the job description
■ Can be used for resubmission
 Have a lifetime (like all WS-Resources)
■ Used for automatic clean-up
07/07/2009 Slide 9

10. UAS: Job Management and Storage Services
Client Target
1. submit System
Service
1.1.1 return job
address
1.1 create
3. start
Job
2. import data 2. stage-in data
Local
Filespace 4. export data USpace 4. stage-out data Remote
Storage
Spaces
07/07/2009 Slide 10

11. UAS: Storage and File Transfer Services
Storage
Management
1. importFile() /exportFile() Service
3. return FTS address
2. create
File
Transfer
Client 4. write/read data, Service
monitor
07/07/2009 Slide 11

12. UAS: File Transfer Protocols
 Pluggable mechanisms
■ Both for client-server and server-server transfers
 Default mechanism: Simple OGSA ByteIO
■ Sends data as SOAP messages through the full stack
■ Needs no additional ports
■ No installation effort (pure Java)
■ Performance of ~400kB/sec
 Plain http: ~ 3MB/sec
 GridFTP: Speed depends on line & number of parallel TCP ports
■ Drawbacks: Lots of open ports, installation effort
 UDT: ~ 100MB/sec on 1Gbit/sec line, C++ Implementation
07/07/2009 Slide 12

13. Deployment Scenario: Workflow Services
trace Client
lookup
workflow Global
submit Registry
workflow Service
Container
Workflow Location
Tracer Engine Mapper
Service Container publish
Service
Container
store submit jobs
messages callback
Service Orchestrator query Information
Service Container Service
submit jobs, Service
check job status Container
UAS UAS UAS
Service Service Service collect
Container Container Container data
07/07/2009 Slide 13

14. Workflow Engine
1. submit workflow
3. return workflow
address
2. create
Workflow
Client 4. monitor Instance
execution
07/07/2009 Slide 14

15. Configurable Security Handlers
User U
Security handler chain
SSL
U = SSL partner?
Did U sign R1?
Request R1 login, group,
User: U & role of U?
Service: S Is U allowed
XUUDB to use S?
U
XACML
Policy File read
Service S
07/07/2009 Slide 15

16. Trust Delegation
User U Request R2
Consignor: W SSL Security handler chain
SSL Service: S2 W = SSL partner?
Request R1 Did W sign R2?
Request R1 Workflow
User: U User: U
Trusts: W
Engine W Trusts: W Does U trust W?
Service: S1 (offers S1) => SAML
U
U W Is U allowed
to use S2?
XUUDB
read
XACML Service S2
Policy File
07/07/2009 Slide 16

17. UNICORE as a Web Service Hosting Environment
 Security
 Platform independence
 Lightweight and performing: Jetty, XFire
 High level programming APIs => Minimal effort
 Hot deployment of web services
 Transparent persistence layer using relational databases
07/07/2009 Slide 17

18. Ongoing Development (Incomplete List!)
 European Projects
■ Smart LM: License management
■ Phosphorus: Meta-scheduling, network reservation
■ Etics: Tool for distributed builds on different platforms
 German Projects
■ D-Mon: Monitoring in the D-Grid
■ BIS-Grid: Business workflows using BPEL
■ WisNetGrid: Data Management
 Other Activities at the JSC
■ Information service (GLUE 2.0)
■ Purely Java based UDT implementation
07/07/2009 ■ Improved MPI support Slide 18

19. Online Documentation
http://www.unicore.eu
07/07/2009 Slide 19