Openstack cloud infrastructure for scientific community.

1. Private cloud
as a code

2. Introduction
● Andrei Kirushchanka
● Cloud infrastructure tech lead
● CESNET, Brno, Czech Republic

3. ● Introduction
● Exposition
○ Use-cases
○ Private vs Public
○ History and migration to OpenStack
Table of Contents
3
● Automated deployment
○ 2 years ago
○ Now
● Infrastructure and statistics
● Bottlenecks and issues
● Insights and conclusion
○ Partners/sponsors

4. Exposition
00
Private cloud world

5. Project aims to improve cancer drug development
success rates through the enablement of collaborative
PDX(Patient derived tumour xenografts) research.
https://www.europdx.eu/
Melanoma model

6. ELIXIR coordinates and develops life science resources across Europe so that researchers can
more easily find, analyse and share data, exchange expertise, and implement best practices. This
makes it possible for them to gain greater insights into how living organisms work.
6
https://elixir-europe.org

7. Earth observation data hub and platform
http://www.nextgeoss.eu/

8. The goal of VESPA (Virtual European Solar and Planetary
Access) is to build a Virtual Observatory (VO) for Solar
System Sciences, by adapting standard VO techniques
and developing new tools and standards to handle the
specificities of Planetary Science data.
http://europlanet-vespa.eu/
VESPA

9. Effective Management of Pests and Harmful Alien Species:
Integrated Solutions
http://www.emphasisproject.eu/

10. Predicting weather and climate and its
impacts on the environment
http://www.drihm.eu/

11. 11
?

12. 12
Migration to OpenStack

13. Automated deployment
01
Two years ago and now

14. 14
Flow
HW
Provisioning Configuration
OpenStack
deployment

15. 15
HW provisioning 2 years ago
Controller
Compute nodes
Foreman

16. 16
Configuration 2 years ago
Controller
Compute nodes
master

17. 17
OpenStack deployment 2 years ago

18. 18
OpenStack deployment 2 years ago
+ =

19. 19
OpenStack deployment 2 years ago
Controller

20. 20
Automated deployment 2 years ago
Controller
Compute nodes
Foreman

21. OpenStack components 2 years ago
21
keystone
placement OpenStack
version: Rocky
Nova
EC2
Glance
EC2 img.
builder
heat
Cloud
Formation
Cinder
EBS
Ceph_rgw
(oject)
S3
Horizon
Manage
ment
console
neutron ovn

23. 23
HW provisioning Now
Compute
nodes
...
Compute
nodes
Controller
Controller
worker1
Controller
worker2
Controller
worker3
Foreman

24. 24
Configuration Now
master
Compute
nodes
...
Compute
nodes
Controller
Controller
worker1
Controller
worker2
Controller
worker3

25. 25
OpenStack deployment Now
Controller
Controller
worker1
Controller
worker2
Controller
worker3

26. 26
Automated deployment Now
Controller
worker1
Controller
worker2
Controller
worker3
Compute
nodes
...
Compute
nodes
master

27. OpenStack components Now
27
keystone
placement
OpenStack
version: Train
Nova
EC2
Glance
EC2 img.
builder
heat
Cloud
Formation
Cinder
EBS
Ceph_rgw
(oject)
S3
Horizon
Manage
ment
console
neutron ovn
Monasca
Octavia
ELB
Magnum
(stage)
EKS
Barbican
KMS

28. Infrastructure and
Statistics
02

29. 29
Monitoring
Monasca Monasca
Now Plans
+
+

30. 30
Storage backend
Bond
10Gb
10Gb
ceph MON
10Gb
10Gb
Bond

31. 31
Statistics
Users Projects Hypervisors
April 2020 395 345 164
April 2021 763 650 227

32. 32
Statistics
VMs VMs/day
April 2020 53897 148
April 2021 131242 360

33. GPU cards
NVIDIA 2080
NVIDIA 1080
NVIDIA T4
62
CEPH Storage amount
Block storage + Object storage
1.5
PB
Statistics
GPU and storage
3
3

34. Bottlenecks and issues
03

35. Bottlenecks and
issues
3
5
● Code dependency on the infrastructure
● Hardware
● Networking
● Dev & Operations

36. Bottlenecks and
issues
3
6
● Code dependency on the infrastructure

37. Bottlenecks and
issues
3
7
● Hardware

38. Bottlenecks and
issues
3
8
● Networking

39. Bottlenecks and
issues
3
9
● Dev & Operations

40. Insights and
Conclusion
04

41. Insights and
Conclusion
4
1
● HPC CLoud
● Promote cloud usage best practices

42. Colin Powel
42

43. Andrei Kirushchanka
andy@cesnet.cz