In this video from the Docker Workshop at ISC 2015, Christian Kniep from QNIB Solutions shows how he uses Docker in his efforts to provide a HPC software stack in a box, encapsulating each layer in the HPC stack within a Linux Container.
Watch the video presentation: http://wp.me/p3RLHQ-eos
Learn more: http://qnib.org/about/
2. • HPC in a Box
• Cluster Use-Case
QNIBInventory
Dashboards
• Development Workflow w/ Containers
• Microservices
2
Agenda
This Workshop was recorded:
https://youtu.be/L9SyY9TZyY4
5. 5
Mock-Up a Cluster Stack
• ‘Hello World’ everything
ibsim, SLURM cluster, monitoring
• First attempt using VirtualBox
Predefined resource use
Gave up after starting a handful of VMs
• Stumbled upon Docker in 2013
Applied it to my problem
39. MacBook
elasticsearch
39
bounded context
srv backend consul
Log/Events
elasticsearch
logger logstash
kibana kiabana
kopf es-kopf
• Iterate within the context
• Rely on stable version of (loosely coupled)
dependencies
Thinkpad
elasticsearch
srv backend consul
Log/Events
elasticsearch
neo4j neo4j
Inventory
inventory QINBInv
40. 40
µServices: Break down Silos
Super
User
Prod
Mgr
Sys
Arch
Dev QA
Sys
Adm
Net
Adm
HPC
Snow-
flakes
Product Team Using Monolithic Delivery
Product Team Using Monolithic Delivery
Product Team Using Microservices A
P
I
Srv Team
Srv TeamProduct Team Using Microservices
41. • Linux Containers do not add much overhead
• Lighting fast development iteration,
since boot-up is plain ‘fork()’
• build, distribute and run extremely powerful
private/public registries
• image hierarchy to add new service in minutes
41
Conclusion