SlideShare una empresa de Scribd logo
1 de 68
Descargar para leer sin conexión
Running Databases
on k8s
within the Dutch Government
Subjects of today
● Short introduction
● Break the glass
● Database performance on CEPH
● Some other ideas
● General recommendations
● Conclusion and takeaways
Introduction
Who am I?
Introduction
● Database space since 2000, CloudNative
enthusiast since 2015
● Worked for DJI, bol.com, EDB, RIVM,
MannemSolutions
● Contributor
○ Since ever
○ pg_hba, Stolon, wal-g, bitbucket-cli, and more
○ pgfga, pgroute66, pgquartz, and more
● Dreamer
Mission (Mannem Solutions)
Enable organizations to be successful
with modern Open Source Data
Solutions.
Gov and Private 100% Open Source!!!
Early adopter
Specialized in Data
Challenges
Temporary access
Authentication
CPU, Memory, storage, connectivity
(Kubernetes)
Nodes
Nodes
Nodes
Storage
Storage
Storage
Networking
Networking
Networking
Projects Projects
Projects
DB1
App
Federated
Authentication
Authentication - LDAP is not very CloudNative
Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic
environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable
infrastructure, and declarative APIs exemplify this approach.
These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with
robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.
The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem
of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for
everyone.
https://github.com/cncf/toc/blob/main/DEFINITION.md
Authentication
CPU, Memory, storage, connectivity
(Kubernetes)
Nodes
Nodes
Nodes
Storage
Storage
Storage
Networking
Networking
Networking
Projects Projects
Projects
DB1
App
Federated
Authentication
scalable? modern, dynamic?
declarative APIs?
loosely coupled? resilient?
manageable?
observable?
robust automation?
allow engineers to make
high-impact changes
frequently and predictably
with minimal toil?
Authentication - Natural drift from CN
CPU, Memory, storage, connectivity
(Kubernetes)
Projects
DB1
App
Federated
Authentication
1: Normal operation:
Only app connects
3:Fix Issue:
New release of App!!!
2: Investigating issues:
Humans connect
Humans can always
connect
=
remove the need to fix
with App
Seems like too much
hassle for exception
resolution
Authentication
Lets do it all with client certificates:
● Specific for project
● Not relying on external components
● Manageable with operators (Very CN)
● Short lived (Ideal for Break The Glass)
● Decoupled
○ Private key not required for verifying the cert
○ Handing out cert does not require Postgres access
Authentication - Client certs
Project
DB1
App
CNPG operator
CPU, Memory, storage, connectivity
(Kubernetes)
cert-manager BreakTheGlass
cluster.postgresql.cnpg.io/v1
certrequest.cert-manager.io
accessrequest.btg.io
Authentication - Break the glass is very CloudNative
Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic
environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable
infrastructure, and declarative APIs exemplify this approach.
These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with
robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.
The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem
of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for
everyone.
https://github.com/cncf/toc/blob/main/DEFINITION.md
Break the glass - Let’s build it
Challenges
Database performance
Lets test - CloudNativePG on Premise/OpenShift/Ceph
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
Lets test - CloudNativePG on CEPH
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1286
CloudNative PG
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
Lets test - What’s going on?
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
Let’s do some more tests:
● Separate disk for WAL (which you should always do)
● Add resources
● Improve storage performance (the wrong way)
Lets test - separate wal
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1286
CloudNative PG
1/2 vCores
512MiB/1GiB RAM
10/10 GiB storage
init: 77s
latency: 61ms
tps: 1312
CloudNative PG
(Separate wal)
Hmmm…
Separating WAL
does not help
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
Lets test - Increase resources
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1286
CloudNative PG
2/3 vCores
1/1.5 GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1337
CloudNative PG
Hmmm…
Increasing
CPU/Memory
does not help
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
Lets test - Would faster storage help?
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1286
CloudNative PG
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: 34s
latency: 33ms
tps: 2424
CloudNative PG
(fsync=off)
Yes,
faster storage
would help
(obviously)
scale: 100
clients: 80
threads: 10
Database performance
Ok, let’s compare
Lets test - separate wal
1/2 vCores
512MiB/1GiB RAM
10/10 GiB storage
init: 77s
latency: 61ms
tps: 1312
CloudNative PG
1/2 vCores
512MiB/1GiB RAM
100 GiB storage
init: 37s
latency: 147ms
tps: 1202
CloudNative PG
(AKS)
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
Standard_D4s_v3
4 vCores
16GiB RAM
Lets test - Increase resources
2/3 vCores
1/1.5 GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1337
CloudNative PG
2/3 vCores
1/1.5 GiB RAM
100 GiB storage
init: 32s
latency: 41ms
tps: 1929
CloudNative PG
cpu: 2
memory: 1GiB
scale: 100
clients: 80
threads: 10
(AKS)
Standard_D4s_v3
4 vCores
16GiB RAM
Lets test - Would faster storage help?
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1286
CloudNative PG
1/2 vCores
512M/1GiB RAM
100 GiB storage
init: s
latency: ms
tps:
CloudNative PG
(fsync=off)
scale: 100
clients: 80
threads: 10
(AKS, fsync=off)
Compare to other storage
Let’s compare - PgVillage on Azure VM
PgVillage on Azure
init: 1m38s
latency: 126ms
tps: 630
Standard_DS1_v2
Standard_LRS
1vCPU, 4G mem
PgVillage on Azure
init: 57s
latency: 69ms
tps: 1160
(Separate wal)
Standard_DS2_v2
Standard_LRS
2vCPU, 8G mem
PgVillage on Azure
init: 47s
latency: 31ms
tps: 2530
(Separate wal)
scale: 100
clients: 80
threads: 10
Conclusion - PgVillage
● Separating WAL helps (doubles performance)
● Increasing resources helps too
Why separate WAL
and Data matters
Why larger
machine matters
Let’s compare - Azure VM, Azure Postgres, CloudNativePG
Azure Postgres*
init: 2m15s
latency: 37ms
tps: 2127
Burstable, B2s,
2 vCores, 4 GiB RAM
128 GiB storage
Standard_DS2_v2
Standard_LRS
2vCPU, 8G mem
PgVillage on Azure
init: 47s
latency: 31ms
tps: 2530
(Separate wal)
2/3 vCores
1/1.5 GiB RAM
100 GiB storage
init: 69s
latency: 62ms
tps: 1337
CNPG on Azure
*also scales with machine type
scale: 100
clients: 80
threads: 10
Database performance
Why is On Premise not ‘over 9000’?
Dutch Gov
(On Premise)
Why? - Some background
Classic
VMWare / VMDK SAN storage
CloudNative
OpenShift / CEPH
DB Performance
(TPS)
Storage Performance
(fsync latency)
Filesystem
● stream based
● FS: mostly write
● write: transaction inline
● block based and memory cached
● FS: more read
● write: direct in memory
checkpoint in the background
Why? Is it data or WAL?
Database
data log
Storage
sync
ΔT: latency
Data Apply log
Memory
● Mostly read
○ and ingest write
● In memory > checkpoint
(background process)
● Block based
● Mostly write
○ ingest write also in log
● Update is inline for TPS
● stream based
Data
Apply log
Storage performance (On premise)
Storage performance
distribution centre
Assembly line
Filesystem
● stream based
● FS: mostly write
● write: commit
(transaction
inline)
● block based and
memory cached
● FS: more read
● write: direct in
memory,
checkpoint in the
background
Why? Ceph vs DAS
data log
Storage
sync
ΔT: latency
Data Apply log
Filesystem
SSD
SCSI-BUS
Filesystem
SSD
kernel/driver
kernel/driver
tcp network
node software
SCSI-BUS
driver/kernel
SSD
node software
SCSI-BUS
driver/kernel
Classic - DAS:
CloudNative - CEPH:
Memory
Lower latency?
Higher latency?
Conclusion - CloudNativePG on CEPH
● Separating WAL does not help
Probably due to all sync IO on WAL volume
● Increasing CPU and Memory has no affect.
Storage IOPS is the limiting factor?
● Disabling fsync increases performance
Yes, Storage IOPS is the limiting factor!!!
APP
Why is 1300 enough?
FS
DB
data log
SAN
Memory
FS
DB1
data log
CEPH
Memory
APP
microservice
FS
DB2
data log
CEPH
Memory
microservice
FS
DB3
data log
CEPH
Memory
microservice
Classic:
MicroService:
Database performance
Can we fix it?
How can we fix it?
Introduce store with faster fsync performance
● We could still run on VM
But wait…
● Our OpenShift clusters run on VMWare.
Can’t we use VMDK (Tanzu/vSphere CNS)?
So, I talked to the architect, and
1300 TPS is enough
So, for now
Current approach:
● At least we know :)
● Keep testing
● Tell everyone, and if anyone needs to
go faster, let’s fix it.
Database performance
But wait, there is more…
Another observation
As I tested, I noticed how
TPS and latency seem to
interact
So let’s build a tool
● How many CPU presentation
○ rust (and go), multithreaded
○ only tps, #clients was fixed, no ssl
○ just getting started to program in rust
● pg_tps_optimizer
○ rust, multithreaded
○ tps and latency
○ #clients increase (fibonacci)
○ ssl and client certs support
○ still just getting started to program in
rust, but neovim / primeagen
○ Work in progress, but not there yet :(
https://www.postgresql.eu/events/pgconfeu2019/sessions/session/2797/slides/199/2019%20How%20much%20CPU%2060.000%20TPS%20(PGCONF).pdf
https://github.com/MannemSolutions/pg_tps_optimizer
Some other issues and
solutions
Issues we ran into and how we fix them
Other examples / ideas
● bitbucket-cli
● Project Quota
○ Image with oc, tkn, etc.
238MB, 210 vulnerabilities (2 critical, 16 High)
○ BYO: 13.69MB, 0 vulnerabilities
● Break the glass functionality
○ Temporary access, with proper auditing
● Pipeline runner
○ read pipeline definition > set params from environment > start it
○ More flexible, smaller image, 0 vulnerabilities
● Image puller
○ Pull image with all tags, image LCM
○ Easy to configure
I wanna invite everyone
to join our adventure
Great, how?
Think CloudNative
Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic
environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable
infrastructure, and declarative APIs exemplify this approach.
These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with
robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil.
The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem
of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for
everyone.
Suggestion: Open Source vs Tender
● POC with Open-Source
● Acquire support
○ If you need it
○ But only if you built it
● Only acquire support for Open
Source
● GPL is fine
○ Unless you want to change, not
upstream your changes, but do want to
redistribute (you naughty boy)
DYI vs Support
1. Make it work
○ Use Open Source
○ Investigate multiple options
2. Make it right
○ Do you need support?
○ Request support for your solution,
instead of a solution with support
3. Make it perform
○ Don’t overdo
■ aim for similar performance
■ Only as required (production=yes,
cicd=no, be smart)
○ Microservice architecture helps big time
○ Use the same storage as on original
architecture
Takeaways
The one slide you wanna take home
Conclusions
Dutch government is embracing
CloudNative
Running CloudNative database on
OpenShift with Ceph
● is doable
● up to 1300TPS (maybe higher)
● In a CN environment that might be
enough
● If it isn’t, there are options
Key takeaways
● Think CloudNative
● Don’t fear to build your own
○ You will learn what you really need,
then acquire it, not before you know
● Open your Source
● Decide on
○ Expectations you have
○ Investments you are willing to put in:
■ Effort
■ Money if needed
FIN
FIN
Using Open Source helps
● POC with Open-Source
● Acquire support IF YOU NEED IT
● Get support on the Open Source
solution
● GPL is only an issue if you want to
change and redistribute without
contributing your changes
CMDB
Performance
● Performance, latency, fsync
● Performance, microservices distribution
● Need support
● Air gapped
● The power of CICD
● Pets vs Cattle
● Disaster Recovery
The idea is to talk about Dutch Gov, Databases on K8s, and their challenges.
The them would be to think CloudNative, which is especially difficult in this combi:
● Databases are usually pets, but CN thought is Cattle
● Dutch gov usually have a more classical approach where CN is a more modern approach (I will leave this out, but it def is there)
So I wanted to touch the following subjects:
● storage performance, Pets approach: Can I create a huge database which requires gzillion TPS
○ Test what performance is achievable and you will be surprised
○ Baby steps, start with same storage as database VM's
○ CN/Cattle approach would be a divide and conquer approach
which def helps bringing down requirements and increases perf of total system
● Authentication
○ Don't think classical DBA that access all databases for manual tuning, etc.
App is SPOC for DB and as such dev on app is how to apply DB changes, but that requires access.
○ general approach would be to use federated auth like LDAP, but LDAP is not really CN approach
○ Introducing an alternate approach:
Break the glass option, Client certs as short lived auth tokens for short period DBA access where you need is
● Backups
○ Classic approach is 'backup everything'
That dogma is severely limiting. Think out of the box
■ Do you want DB's in CI/CD pipelines? Do they need backups?
■ Do you want short lived databases that run workload
(e.a. generate reports from raw data)? Do they need backups?
○ Make sure you can restore your data
■ emphasis on can: make sure it is an option, not a rule of thumb
■ emphasis on restore: 'dump and load',
but also 'rebuild from other datasources' or even generate new testset can be valid options too
● CMDB?
○ Option for short lived databases, so use existing inventories instead
Ideas
● Performance test
○ Azure VM, Azure K8s
○ In-House, openShift, CEPH
● Questionaire
Title: Implementing data and databases on K8s within the Dutch government.
Description: A small walkthrough of projects within the dutch government running Data(bases) on OpenShift. This talk
shares success stories, provides a proven recipe to `get it done` and debunks some of the FUD.
Introduction: I have always been a weird DBA, trying to combine Databases with out-of-the-box thinking and a
DevOps mindset. Around 2016 I fell in love with both Postgres and Kubernetes, and I then committed my life to
enabling Dutch organizations with running their Database workloads CloudNative. Over the last few years I worked as
a private contractor for 2 large government agencies doing exactly that, and I want to share my and others (success
stories) hoping to enable and inspire Data on Kubernetes adoption.
● Why am I trying to get DoK adoption in Dutch Gov?
● Dutch government is adorable
○ Rogers: Laggards trying to become late majority
○ You are too late to have us approve your submission
○ Have we bought support?
○ Computable
● Data on Kubernetes Paradox
○ Data (stateful by nature)
○ On Kubernetes (Stateless by nature)
● Challenges
○ Multicluster
Who am I
● Masochistic by nature
○ I will shuffle the beehive if necessary
○ `Challenge accepted` mentality
● Out of the box thinking
○ In the box is BBBOOORRRIIINNNGGG
● `Just do it` mentality
○ I’m okay doing the heavy lifting if necessary
○ I’d rather do it then think about why we shouldn’t
● Ideology
○ Government is there for all of us, so let’s enable them
memes create with: https://imgflip.com/memegenerator
Who am I - Masochistic by nature
Shuffle the beehive if necessary
`Challenge accepted` mentality
memes create with: https://imgflip.com/memegenerator
Who am I - Out of the box thinker
Who am I - `Just do it` mentality
memes create with: https://imgflip.com/memegenerator
Data on Kubernetes - Why not?
●
Data On Kubernetes Paradox
Image downloaded from: https://forgottenbytheworld.blogspot.com/2012/02/abandoned-leaning-house.html
● Data is stateful by nature
● Kubernetes is Stateless by nature
How can you build on a non-solid
foundation???
Rethink the problem
You only need to DoK.io every now and then
Dutch government is adorable
● You are too late to have us approve
your submission
● Avoid computable at all costs
● Have we bought support?
● Average age of 58
Gov
Storage - do’s and don’ts
● Databases love fsync’able storage
○ fsync should be fast (short roundtrip)
○ fsync should be trustworthy (when fsync says it is ok, it must be ok)
● Do’s
○ Use block storage
○ Use same storage as on VM
■ VMWare Tanzu to have vmdk on k8s
■ Or also use CEPH on VM(if you also use it on k8s)
● Dont’s
○ Don’t use NFS!!!
○ Don’t use
Disaster recovery
K8s
Cluster 1
K8s
Cluster 2
Global Load Balancer
Primary cluster Replica cluster (DR)
Options:
● Run database outside of k8s
● Use native RDBMS capabilities
○ cloudnative-pg.io: replica-clusters
○ crunchy pgo: streaming standby

Más contenido relacionado

Similar a Implementing data and databases on K8s within the Dutch government

Benchmarking your cloud performance with top 4 global public clouds
Benchmarking your cloud performance with top 4 global public cloudsBenchmarking your cloud performance with top 4 global public clouds
Benchmarking your cloud performance with top 4 global public cloudsdata://disrupted®
 
Webinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to Know
Webinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to KnowWebinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to Know
Webinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to KnowStorage Switzerland
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructurexKinAnx
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructuresolarisyourep
 
OS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of MLOS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of MLNordic APIs
 
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Community
 
Introduction to HPC & Supercomputing in AI
Introduction to HPC & Supercomputing in AIIntroduction to HPC & Supercomputing in AI
Introduction to HPC & Supercomputing in AITyrone Systems
 
Containerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the CloudContainerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the CloudSubbu Rama
 
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...Cisco DevNet
 
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka Mario Ishara Fernando
 
SUSE - performance analysis-with_ceph
SUSE - performance analysis-with_cephSUSE - performance analysis-with_ceph
SUSE - performance analysis-with_cephinwin stack
 
OpenEBS Technical Workshop - KubeCon San Diego 2019
OpenEBS Technical Workshop - KubeCon San Diego 2019OpenEBS Technical Workshop - KubeCon San Diego 2019
OpenEBS Technical Workshop - KubeCon San Diego 2019MayaData Inc
 
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageWebinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageMayaData Inc
 
GPU cloud with Job scheduler and Container
GPU cloud with Job scheduler and ContainerGPU cloud with Job scheduler and Container
GPU cloud with Job scheduler and ContainerAndrew Yongjoon Kong
 
Design & Secure Your Cloud Infrastructure
Design & Secure Your Cloud Infrastructure Design & Secure Your Cloud Infrastructure
Design & Secure Your Cloud Infrastructure Anoop Nair
 
K8s best practices from the field!
K8s best practices from the field!K8s best practices from the field!
K8s best practices from the field!DoiT International
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...NETWAYS
 
Choose the Right Container Storage for Kubernetes
Choose the Right Container Storage for KubernetesChoose the Right Container Storage for Kubernetes
Choose the Right Container Storage for KubernetesYusuf Hadiwinata Sutandar
 
[BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes]
[BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes][BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes]
[BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes]Wong Hoi Sing Edison
 

Similar a Implementing data and databases on K8s within the Dutch government (20)

Benchmarking your cloud performance with top 4 global public clouds
Benchmarking your cloud performance with top 4 global public cloudsBenchmarking your cloud performance with top 4 global public clouds
Benchmarking your cloud performance with top 4 global public clouds
 
Where should I run my code? Serverless, Containers, Virtual Machines and more
Where should I run my code? Serverless, Containers, Virtual Machines and moreWhere should I run my code? Serverless, Containers, Virtual Machines and more
Where should I run my code? Serverless, Containers, Virtual Machines and more
 
Webinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to Know
Webinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to KnowWebinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to Know
Webinar: NVMe, NVMe over Fabrics and Beyond - Everything You Need to Know
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructure
 
Presentation architecting a cloud infrastructure
Presentation   architecting a cloud infrastructurePresentation   architecting a cloud infrastructure
Presentation architecting a cloud infrastructure
 
OS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of MLOS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of ML
 
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
Ceph Day Seoul - AFCeph: SKT Scale Out Storage Ceph
 
Introduction to HPC & Supercomputing in AI
Introduction to HPC & Supercomputing in AIIntroduction to HPC & Supercomputing in AI
Introduction to HPC & Supercomputing in AI
 
Containerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the CloudContainerizing GPU Applications with Docker for Scaling to the Cloud
Containerizing GPU Applications with Docker for Scaling to the Cloud
 
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
 
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
Microservices , Docker , CI/CD , Kubernetes Seminar - Sri Lanka
 
SUSE - performance analysis-with_ceph
SUSE - performance analysis-with_cephSUSE - performance analysis-with_ceph
SUSE - performance analysis-with_ceph
 
OpenEBS Technical Workshop - KubeCon San Diego 2019
OpenEBS Technical Workshop - KubeCon San Diego 2019OpenEBS Technical Workshop - KubeCon San Diego 2019
OpenEBS Technical Workshop - KubeCon San Diego 2019
 
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageWebinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
 
GPU cloud with Job scheduler and Container
GPU cloud with Job scheduler and ContainerGPU cloud with Job scheduler and Container
GPU cloud with Job scheduler and Container
 
Design & Secure Your Cloud Infrastructure
Design & Secure Your Cloud Infrastructure Design & Secure Your Cloud Infrastructure
Design & Secure Your Cloud Infrastructure
 
K8s best practices from the field!
K8s best practices from the field!K8s best practices from the field!
K8s best practices from the field!
 
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...OSDC 2018 | Three years running containers with Kubernetes in Production by T...
OSDC 2018 | Three years running containers with Kubernetes in Production by T...
 
Choose the Right Container Storage for Kubernetes
Choose the Right Container Storage for KubernetesChoose the Right Container Storage for Kubernetes
Choose the Right Container Storage for Kubernetes
 
[BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes]
[BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes][BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes]
[BarCamp2018][20180915][Tips for Virtual Hosting on Kubernetes]
 

Más de DoKC

Distributed Vector Databases - What, Why, and How
Distributed Vector Databases - What, Why, and HowDistributed Vector Databases - What, Why, and How
Distributed Vector Databases - What, Why, and HowDoKC
 
Is It Safe? Security Hardening for Databases Using Kubernetes Operators
Is It Safe? Security Hardening for Databases Using Kubernetes OperatorsIs It Safe? Security Hardening for Databases Using Kubernetes Operators
Is It Safe? Security Hardening for Databases Using Kubernetes OperatorsDoKC
 
Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery
Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster RecoveryStop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery
Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster RecoveryDoKC
 
Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...
Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...
Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...DoKC
 
The State of Stateful on Kubernetes
The State of Stateful on KubernetesThe State of Stateful on Kubernetes
The State of Stateful on KubernetesDoKC
 
Make Your Kafka Cluster Production-Ready
Make Your Kafka Cluster Production-ReadyMake Your Kafka Cluster Production-Ready
Make Your Kafka Cluster Production-ReadyDoKC
 
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...DoKC
 
Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
Run PostgreSQL in Warp Speed Using NVMe/TCP in the CloudRun PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
Run PostgreSQL in Warp Speed Using NVMe/TCP in the CloudDoKC
 
ING Data Services hosted on ICHP DoK Amsterdam 2023
ING Data Services hosted on ICHP DoK Amsterdam 2023ING Data Services hosted on ICHP DoK Amsterdam 2023
ING Data Services hosted on ICHP DoK Amsterdam 2023DoKC
 
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...DoKC
 
Analytics with Apache Superset and ClickHouse - DoK Talks #151
Analytics with Apache Superset and ClickHouse - DoK Talks #151Analytics with Apache Superset and ClickHouse - DoK Talks #151
Analytics with Apache Superset and ClickHouse - DoK Talks #151DoKC
 
Mastering MongoDB on Kubernetes, the power of operators
Mastering MongoDB on Kubernetes, the power of operators Mastering MongoDB on Kubernetes, the power of operators
Mastering MongoDB on Kubernetes, the power of operators DoKC
 
Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?DoKC
 
Using Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” serviceUsing Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” serviceDoKC
 
The many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent dataThe many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent dataDoKC
 
Testing the Mettle: Evaluating data solutions for large-scale production to c...
Testing the Mettle: Evaluating data solutions for large-scale production to c...Testing the Mettle: Evaluating data solutions for large-scale production to c...
Testing the Mettle: Evaluating data solutions for large-scale production to c...DoKC
 
Tech with project RapGOD
Tech with project RapGODTech with project RapGOD
Tech with project RapGODDoKC
 
Running a database on local NVMes on Kubernetes
Running a database on local NVMes on KubernetesRunning a database on local NVMes on Kubernetes
Running a database on local NVMes on KubernetesDoKC
 
Operator Lifecycle Management
Operator Lifecycle ManagementOperator Lifecycle Management
Operator Lifecycle ManagementDoKC
 
How to protect your data
How to protect your dataHow to protect your data
How to protect your dataDoKC
 

Más de DoKC (20)

Distributed Vector Databases - What, Why, and How
Distributed Vector Databases - What, Why, and HowDistributed Vector Databases - What, Why, and How
Distributed Vector Databases - What, Why, and How
 
Is It Safe? Security Hardening for Databases Using Kubernetes Operators
Is It Safe? Security Hardening for Databases Using Kubernetes OperatorsIs It Safe? Security Hardening for Databases Using Kubernetes Operators
Is It Safe? Security Hardening for Databases Using Kubernetes Operators
 
Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery
Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster RecoveryStop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery
Stop Worrying and Keep Querying, Using Automated Multi-Region Disaster Recovery
 
Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...
Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...
Transforming Data Processing with Kubernetes: Journey Towards a Self-Serve Da...
 
The State of Stateful on Kubernetes
The State of Stateful on KubernetesThe State of Stateful on Kubernetes
The State of Stateful on Kubernetes
 
Make Your Kafka Cluster Production-Ready
Make Your Kafka Cluster Production-ReadyMake Your Kafka Cluster Production-Ready
Make Your Kafka Cluster Production-Ready
 
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...
Dynamic Large Scale Spark on Kubernetes: Empowering the Community with Argo W...
 
Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
Run PostgreSQL in Warp Speed Using NVMe/TCP in the CloudRun PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
Run PostgreSQL in Warp Speed Using NVMe/TCP in the Cloud
 
ING Data Services hosted on ICHP DoK Amsterdam 2023
ING Data Services hosted on ICHP DoK Amsterdam 2023ING Data Services hosted on ICHP DoK Amsterdam 2023
ING Data Services hosted on ICHP DoK Amsterdam 2023
 
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
Running PostgreSQL in Kubernetes: from day 0 to day 2 with CloudNativePG - Do...
 
Analytics with Apache Superset and ClickHouse - DoK Talks #151
Analytics with Apache Superset and ClickHouse - DoK Talks #151Analytics with Apache Superset and ClickHouse - DoK Talks #151
Analytics with Apache Superset and ClickHouse - DoK Talks #151
 
Mastering MongoDB on Kubernetes, the power of operators
Mastering MongoDB on Kubernetes, the power of operators Mastering MongoDB on Kubernetes, the power of operators
Mastering MongoDB on Kubernetes, the power of operators
 
Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?Why run Postgres in Kubernetes?
Why run Postgres in Kubernetes?
 
Using Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” serviceUsing Kubernetes to deliver a “serverless” service
Using Kubernetes to deliver a “serverless” service
 
The many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent dataThe many uses of Kubernetes cross cluster migration of persistent data
The many uses of Kubernetes cross cluster migration of persistent data
 
Testing the Mettle: Evaluating data solutions for large-scale production to c...
Testing the Mettle: Evaluating data solutions for large-scale production to c...Testing the Mettle: Evaluating data solutions for large-scale production to c...
Testing the Mettle: Evaluating data solutions for large-scale production to c...
 
Tech with project RapGOD
Tech with project RapGODTech with project RapGOD
Tech with project RapGOD
 
Running a database on local NVMes on Kubernetes
Running a database on local NVMes on KubernetesRunning a database on local NVMes on Kubernetes
Running a database on local NVMes on Kubernetes
 
Operator Lifecycle Management
Operator Lifecycle ManagementOperator Lifecycle Management
Operator Lifecycle Management
 
How to protect your data
How to protect your dataHow to protect your data
How to protect your data
 

Último

VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxGDSC PJATK
 

Último (20)

VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Cybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptxCybersecurity Workshop #1.pptx
Cybersecurity Workshop #1.pptx
 

Implementing data and databases on K8s within the Dutch government

  • 1. Running Databases on k8s within the Dutch Government
  • 2. Subjects of today ● Short introduction ● Break the glass ● Database performance on CEPH ● Some other ideas ● General recommendations ● Conclusion and takeaways
  • 4. Introduction ● Database space since 2000, CloudNative enthusiast since 2015 ● Worked for DJI, bol.com, EDB, RIVM, MannemSolutions ● Contributor ○ Since ever ○ pg_hba, Stolon, wal-g, bitbucket-cli, and more ○ pgfga, pgroute66, pgquartz, and more ● Dreamer
  • 5. Mission (Mannem Solutions) Enable organizations to be successful with modern Open Source Data Solutions. Gov and Private 100% Open Source!!! Early adopter Specialized in Data
  • 7. Authentication CPU, Memory, storage, connectivity (Kubernetes) Nodes Nodes Nodes Storage Storage Storage Networking Networking Networking Projects Projects Projects DB1 App Federated Authentication
  • 8. Authentication - LDAP is not very CloudNative Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for everyone. https://github.com/cncf/toc/blob/main/DEFINITION.md
  • 9. Authentication CPU, Memory, storage, connectivity (Kubernetes) Nodes Nodes Nodes Storage Storage Storage Networking Networking Networking Projects Projects Projects DB1 App Federated Authentication scalable? modern, dynamic? declarative APIs? loosely coupled? resilient? manageable? observable? robust automation? allow engineers to make high-impact changes frequently and predictably with minimal toil?
  • 10. Authentication - Natural drift from CN CPU, Memory, storage, connectivity (Kubernetes) Projects DB1 App Federated Authentication 1: Normal operation: Only app connects 3:Fix Issue: New release of App!!! 2: Investigating issues: Humans connect Humans can always connect = remove the need to fix with App Seems like too much hassle for exception resolution
  • 11. Authentication Lets do it all with client certificates: ● Specific for project ● Not relying on external components ● Manageable with operators (Very CN) ● Short lived (Ideal for Break The Glass) ● Decoupled ○ Private key not required for verifying the cert ○ Handing out cert does not require Postgres access
  • 12. Authentication - Client certs Project DB1 App CNPG operator CPU, Memory, storage, connectivity (Kubernetes) cert-manager BreakTheGlass cluster.postgresql.cnpg.io/v1 certrequest.cert-manager.io accessrequest.btg.io
  • 13. Authentication - Break the glass is very CloudNative Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for everyone. https://github.com/cncf/toc/blob/main/DEFINITION.md
  • 14. Break the glass - Let’s build it
  • 16. Lets test - CloudNativePG on Premise/OpenShift/Ceph cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10
  • 17. Lets test - CloudNativePG on CEPH 1/2 vCores 512M/1GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1286 CloudNative PG cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10
  • 18. Lets test - What’s going on? cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10 Let’s do some more tests: ● Separate disk for WAL (which you should always do) ● Add resources ● Improve storage performance (the wrong way)
  • 19. Lets test - separate wal 1/2 vCores 512M/1GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1286 CloudNative PG 1/2 vCores 512MiB/1GiB RAM 10/10 GiB storage init: 77s latency: 61ms tps: 1312 CloudNative PG (Separate wal) Hmmm… Separating WAL does not help cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10
  • 20. Lets test - Increase resources 1/2 vCores 512M/1GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1286 CloudNative PG 2/3 vCores 1/1.5 GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1337 CloudNative PG Hmmm… Increasing CPU/Memory does not help cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10
  • 21. Lets test - Would faster storage help? 1/2 vCores 512M/1GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1286 CloudNative PG 1/2 vCores 512M/1GiB RAM 100 GiB storage init: 34s latency: 33ms tps: 2424 CloudNative PG (fsync=off) Yes, faster storage would help (obviously) scale: 100 clients: 80 threads: 10
  • 23. Lets test - separate wal 1/2 vCores 512MiB/1GiB RAM 10/10 GiB storage init: 77s latency: 61ms tps: 1312 CloudNative PG 1/2 vCores 512MiB/1GiB RAM 100 GiB storage init: 37s latency: 147ms tps: 1202 CloudNative PG (AKS) cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10 Standard_D4s_v3 4 vCores 16GiB RAM
  • 24. Lets test - Increase resources 2/3 vCores 1/1.5 GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1337 CloudNative PG 2/3 vCores 1/1.5 GiB RAM 100 GiB storage init: 32s latency: 41ms tps: 1929 CloudNative PG cpu: 2 memory: 1GiB scale: 100 clients: 80 threads: 10 (AKS) Standard_D4s_v3 4 vCores 16GiB RAM
  • 25. Lets test - Would faster storage help? 1/2 vCores 512M/1GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1286 CloudNative PG 1/2 vCores 512M/1GiB RAM 100 GiB storage init: s latency: ms tps: CloudNative PG (fsync=off) scale: 100 clients: 80 threads: 10 (AKS, fsync=off)
  • 26. Compare to other storage
  • 27. Let’s compare - PgVillage on Azure VM PgVillage on Azure init: 1m38s latency: 126ms tps: 630 Standard_DS1_v2 Standard_LRS 1vCPU, 4G mem PgVillage on Azure init: 57s latency: 69ms tps: 1160 (Separate wal) Standard_DS2_v2 Standard_LRS 2vCPU, 8G mem PgVillage on Azure init: 47s latency: 31ms tps: 2530 (Separate wal) scale: 100 clients: 80 threads: 10
  • 28. Conclusion - PgVillage ● Separating WAL helps (doubles performance) ● Increasing resources helps too Why separate WAL and Data matters Why larger machine matters
  • 29. Let’s compare - Azure VM, Azure Postgres, CloudNativePG Azure Postgres* init: 2m15s latency: 37ms tps: 2127 Burstable, B2s, 2 vCores, 4 GiB RAM 128 GiB storage Standard_DS2_v2 Standard_LRS 2vCPU, 8G mem PgVillage on Azure init: 47s latency: 31ms tps: 2530 (Separate wal) 2/3 vCores 1/1.5 GiB RAM 100 GiB storage init: 69s latency: 62ms tps: 1337 CNPG on Azure *also scales with machine type scale: 100 clients: 80 threads: 10
  • 30. Database performance Why is On Premise not ‘over 9000’?
  • 31. Dutch Gov (On Premise) Why? - Some background Classic VMWare / VMDK SAN storage CloudNative OpenShift / CEPH DB Performance (TPS) Storage Performance (fsync latency)
  • 32. Filesystem ● stream based ● FS: mostly write ● write: transaction inline ● block based and memory cached ● FS: more read ● write: direct in memory checkpoint in the background Why? Is it data or WAL? Database data log Storage sync ΔT: latency Data Apply log Memory
  • 33. ● Mostly read ○ and ingest write ● In memory > checkpoint (background process) ● Block based ● Mostly write ○ ingest write also in log ● Update is inline for TPS ● stream based Data Apply log Storage performance (On premise) Storage performance distribution centre Assembly line
  • 34. Filesystem ● stream based ● FS: mostly write ● write: commit (transaction inline) ● block based and memory cached ● FS: more read ● write: direct in memory, checkpoint in the background Why? Ceph vs DAS data log Storage sync ΔT: latency Data Apply log Filesystem SSD SCSI-BUS Filesystem SSD kernel/driver kernel/driver tcp network node software SCSI-BUS driver/kernel SSD node software SCSI-BUS driver/kernel Classic - DAS: CloudNative - CEPH: Memory Lower latency? Higher latency?
  • 35. Conclusion - CloudNativePG on CEPH ● Separating WAL does not help Probably due to all sync IO on WAL volume ● Increasing CPU and Memory has no affect. Storage IOPS is the limiting factor? ● Disabling fsync increases performance Yes, Storage IOPS is the limiting factor!!!
  • 36. APP Why is 1300 enough? FS DB data log SAN Memory FS DB1 data log CEPH Memory APP microservice FS DB2 data log CEPH Memory microservice FS DB3 data log CEPH Memory microservice Classic: MicroService:
  • 38. How can we fix it? Introduce store with faster fsync performance ● We could still run on VM But wait… ● Our OpenShift clusters run on VMWare. Can’t we use VMDK (Tanzu/vSphere CNS)? So, I talked to the architect, and
  • 39. 1300 TPS is enough
  • 40. So, for now Current approach: ● At least we know :) ● Keep testing ● Tell everyone, and if anyone needs to go faster, let’s fix it.
  • 41. Database performance But wait, there is more…
  • 42. Another observation As I tested, I noticed how TPS and latency seem to interact
  • 43. So let’s build a tool ● How many CPU presentation ○ rust (and go), multithreaded ○ only tps, #clients was fixed, no ssl ○ just getting started to program in rust ● pg_tps_optimizer ○ rust, multithreaded ○ tps and latency ○ #clients increase (fibonacci) ○ ssl and client certs support ○ still just getting started to program in rust, but neovim / primeagen ○ Work in progress, but not there yet :( https://www.postgresql.eu/events/pgconfeu2019/sessions/session/2797/slides/199/2019%20How%20much%20CPU%2060.000%20TPS%20(PGCONF).pdf https://github.com/MannemSolutions/pg_tps_optimizer
  • 44. Some other issues and solutions Issues we ran into and how we fix them
  • 45. Other examples / ideas ● bitbucket-cli ● Project Quota ○ Image with oc, tkn, etc. 238MB, 210 vulnerabilities (2 critical, 16 High) ○ BYO: 13.69MB, 0 vulnerabilities ● Break the glass functionality ○ Temporary access, with proper auditing ● Pipeline runner ○ read pipeline definition > set params from environment > start it ○ More flexible, smaller image, 0 vulnerabilities ● Image puller ○ Pull image with all tags, image LCM ○ Easy to configure
  • 46. I wanna invite everyone to join our adventure Great, how?
  • 47. Think CloudNative Cloud native technologies empower organizations to build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds. Containers, service meshes, microservices, immutable infrastructure, and declarative APIs exemplify this approach. These techniques enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil. The Cloud Native Computing Foundation seeks to drive adoption of this paradigm by fostering and sustaining an ecosystem of open source, vendor-neutral projects. We democratize state-of-the-art patterns to make these innovations accessible for everyone.
  • 48. Suggestion: Open Source vs Tender ● POC with Open-Source ● Acquire support ○ If you need it ○ But only if you built it ● Only acquire support for Open Source ● GPL is fine ○ Unless you want to change, not upstream your changes, but do want to redistribute (you naughty boy)
  • 49. DYI vs Support 1. Make it work ○ Use Open Source ○ Investigate multiple options 2. Make it right ○ Do you need support? ○ Request support for your solution, instead of a solution with support 3. Make it perform ○ Don’t overdo ■ aim for similar performance ■ Only as required (production=yes, cicd=no, be smart) ○ Microservice architecture helps big time ○ Use the same storage as on original architecture
  • 50. Takeaways The one slide you wanna take home
  • 51. Conclusions Dutch government is embracing CloudNative Running CloudNative database on OpenShift with Ceph ● is doable ● up to 1300TPS (maybe higher) ● In a CN environment that might be enough ● If it isn’t, there are options Key takeaways ● Think CloudNative ● Don’t fear to build your own ○ You will learn what you really need, then acquire it, not before you know ● Open your Source ● Decide on ○ Expectations you have ○ Investments you are willing to put in: ■ Effort ■ Money if needed
  • 53. Using Open Source helps ● POC with Open-Source ● Acquire support IF YOU NEED IT ● Get support on the Open Source solution ● GPL is only an issue if you want to change and redistribute without contributing your changes
  • 54. CMDB
  • 55. Performance ● Performance, latency, fsync ● Performance, microservices distribution ● Need support ● Air gapped ● The power of CICD ● Pets vs Cattle ● Disaster Recovery
  • 56. The idea is to talk about Dutch Gov, Databases on K8s, and their challenges. The them would be to think CloudNative, which is especially difficult in this combi: ● Databases are usually pets, but CN thought is Cattle ● Dutch gov usually have a more classical approach where CN is a more modern approach (I will leave this out, but it def is there) So I wanted to touch the following subjects: ● storage performance, Pets approach: Can I create a huge database which requires gzillion TPS ○ Test what performance is achievable and you will be surprised ○ Baby steps, start with same storage as database VM's ○ CN/Cattle approach would be a divide and conquer approach which def helps bringing down requirements and increases perf of total system ● Authentication ○ Don't think classical DBA that access all databases for manual tuning, etc. App is SPOC for DB and as such dev on app is how to apply DB changes, but that requires access. ○ general approach would be to use federated auth like LDAP, but LDAP is not really CN approach ○ Introducing an alternate approach: Break the glass option, Client certs as short lived auth tokens for short period DBA access where you need is ● Backups ○ Classic approach is 'backup everything' That dogma is severely limiting. Think out of the box ■ Do you want DB's in CI/CD pipelines? Do they need backups? ■ Do you want short lived databases that run workload (e.a. generate reports from raw data)? Do they need backups? ○ Make sure you can restore your data ■ emphasis on can: make sure it is an option, not a rule of thumb ■ emphasis on restore: 'dump and load', but also 'rebuild from other datasources' or even generate new testset can be valid options too ● CMDB? ○ Option for short lived databases, so use existing inventories instead
  • 57. Ideas ● Performance test ○ Azure VM, Azure K8s ○ In-House, openShift, CEPH ● Questionaire Title: Implementing data and databases on K8s within the Dutch government. Description: A small walkthrough of projects within the dutch government running Data(bases) on OpenShift. This talk shares success stories, provides a proven recipe to `get it done` and debunks some of the FUD. Introduction: I have always been a weird DBA, trying to combine Databases with out-of-the-box thinking and a DevOps mindset. Around 2016 I fell in love with both Postgres and Kubernetes, and I then committed my life to enabling Dutch organizations with running their Database workloads CloudNative. Over the last few years I worked as a private contractor for 2 large government agencies doing exactly that, and I want to share my and others (success stories) hoping to enable and inspire Data on Kubernetes adoption.
  • 58. ● Why am I trying to get DoK adoption in Dutch Gov? ● Dutch government is adorable ○ Rogers: Laggards trying to become late majority ○ You are too late to have us approve your submission ○ Have we bought support? ○ Computable ● Data on Kubernetes Paradox ○ Data (stateful by nature) ○ On Kubernetes (Stateless by nature) ● Challenges ○ Multicluster
  • 59. Who am I ● Masochistic by nature ○ I will shuffle the beehive if necessary ○ `Challenge accepted` mentality ● Out of the box thinking ○ In the box is BBBOOORRRIIINNNGGG ● `Just do it` mentality ○ I’m okay doing the heavy lifting if necessary ○ I’d rather do it then think about why we shouldn’t ● Ideology ○ Government is there for all of us, so let’s enable them memes create with: https://imgflip.com/memegenerator
  • 60. Who am I - Masochistic by nature Shuffle the beehive if necessary `Challenge accepted` mentality memes create with: https://imgflip.com/memegenerator
  • 61. Who am I - Out of the box thinker
  • 62. Who am I - `Just do it` mentality memes create with: https://imgflip.com/memegenerator
  • 63. Data on Kubernetes - Why not? ●
  • 64. Data On Kubernetes Paradox Image downloaded from: https://forgottenbytheworld.blogspot.com/2012/02/abandoned-leaning-house.html ● Data is stateful by nature ● Kubernetes is Stateless by nature How can you build on a non-solid foundation???
  • 65. Rethink the problem You only need to DoK.io every now and then
  • 66. Dutch government is adorable ● You are too late to have us approve your submission ● Avoid computable at all costs ● Have we bought support? ● Average age of 58 Gov
  • 67. Storage - do’s and don’ts ● Databases love fsync’able storage ○ fsync should be fast (short roundtrip) ○ fsync should be trustworthy (when fsync says it is ok, it must be ok) ● Do’s ○ Use block storage ○ Use same storage as on VM ■ VMWare Tanzu to have vmdk on k8s ■ Or also use CEPH on VM(if you also use it on k8s) ● Dont’s ○ Don’t use NFS!!! ○ Don’t use
  • 68. Disaster recovery K8s Cluster 1 K8s Cluster 2 Global Load Balancer Primary cluster Replica cluster (DR) Options: ● Run database outside of k8s ● Use native RDBMS capabilities ○ cloudnative-pg.io: replica-clusters ○ crunchy pgo: streaming standby