SlideShare una empresa de Scribd logo
1 de 50
Descargar para leer sin conexión
HOW CONTAINERS HELPED A
SAAS STARTUP BE DEVELOPED
INTRANETUM
intranetum
Ramon Navarro Bosch
CTO iskra.cat
CTO intranetum.com
FWT plone.org
WHO AM I?
Machine Learning - Python - Angular 2
Agile Test Driven Development
Always a sysadmin in my heart
In love with docker / k8s
intranetumWHAT IS INTRANETUM?
BRAIN TO CLASSIFY
▸ Knowledge management
▸ Files, Notes, Links
▸ Deep learning by scope/user
▸ Auto classify information
▸ Search information
▸ Less time to find what you are
looking for + discover knowledge
inside the company
intranetumTHE PROBLEM
FAST, AGILE, TEST, COMPLEX
▸ We needed to develop a SaaS solution in 3 months (proof)
▸ 1 Desember - 23 Febrary (4YFN)
▸ A team of 2 developers
▸ Needs to be modular for evolution
▸ Needs to scale fast
▸ Initial architecture design showed 10 different
components ( 40 by the end of February )
intranetum
DEMO
INTRANETUM…
intranetum
DEMO
DEPLOY I …
intranetum
CHOSEN OPS
COMPONENTS
intranetumCOMPONENTS
DOCKER
▸ General Adopted Container Solution
▸ Native support Mac OS X + Linux +
Windows
▸ Standardization of build process of
images
▸ Service is not an application, is a
container
▸ Ports, volumes and build layers
management
▸ Definition of repository of images
intranetumCOMPONENTS
KUBERNETTES (K8S)
▸ Same deployment infrastructure for
production, testing, development and
stage
▸ Same deployment infrastructure for cloud
and in-house
▸ Load balancer integrated
▸ Management of HDD
▸ Secret configuration management
▸ Internal network discovery (DNS)
▸ Jobs
intranetumCOMPONENTS
GOOGLE CLOUD PLATFORM
▸ Clusters for stage / production
▸ HTTPS load balancer
▸ 100% integrated with k8s
▸ Full system monitoring
▸ Scaling fast
▸ Private Docker Registry
intranetumCOMPONENTS
JENKINS + GIT
▸ Continuous Integration
▸ Continuous Deployment
▸ Orchestration of versioning and
stability
▸ Workflow of testing & deployment
▸ quality, Quality, QUALITY!
▸ testing, Testing, TESTING!
intranetumCOMPONENTS
METRICS (ELK)
▸ mesure, Mesure, MESURE
▸ Continuous Metrics
▸ Centralized Log Service
▸ Nice diagrams to show VCs
intranetum
WE
STARTED…
intranetumGROUPING
CLUSTER | GROUPS | CONTAINERS
STAGE
GROUP2 GROUP3 GROUP4GROUP1
RC RC RC
C1 C2 C1 C2 C3 C1
intranetumDEVELOPMENT
MAIN OPEN SOURCE COMPONENTS
▸ PLONE (STORAGE/CONTENT TYPES/SECURITY)
▸ ELASTIC SEARCH (SEARCH / LOGGING)
▸ ZODB (DATABASE)
▸ REDIS (CACHE)
▸ LDAP (USER/PERMISSION DB)
▸ RABBITMQ (QUEUE)
intranetumDEVELOPMENT
MAIN OPEN SOURCE FRAMEWORKS
▸ ANGULAR2
▸ TENSORFLOW
▸ NLTP
▸ PROTOBUFFERS
▸ PYRAMID + ASYNCIO
CMS
GROUP1
intranetumDEVELOPMENT
SEMANTIC GROUP1 CMS - DEV
TEST CLUSTER CLOUD
ELAS
TIC
ZOPE
GROUP2 GROUP3
GROUP4 GROUP1
ZEO
NFS
intranetumDEVELOPMENT
ELASTICSEARCH - DOCKERFILE
FROM elasticsearch:1.7
MAINTAINER Iskra
# Expose
EXPOSE 9200
EXPOSE 9300
# Cors enabled for testing
CMD ["elasticsearch", "-Des.cluster.name=intranetum", "-Des.http.cors.enabled=true"]
intranetumDEVELOPMENT
ELASTICSEARCH - LOCAL DEV/TEST DOCKER COMPOSE
elasticsearch:
image: elasticsearch:1.7
ports:
- "9200:9200"
- "9300:9300"
volumes:
- ./esconfig:/usr/share/elasticsearch/config
- ./esdata:/usr/share/elasticsearch/data
command: ["elasticsearch", "-Des.network.publish_host=localhost", "-Dhttp.cors.enabled=true", "-Dhttp.cors.allow-
origin='*'"]
intranetumDEVELOPMENT
ELASTICSEARCH
- STAGE GCP
kind: ReplicationController
spec:
replicas: 1
template:
spec:
containers:
- name: masterelastic
resources:
limits:
cpu: 0.250
image: eu.gcr.io/XXXXXXXX/elasticsearch
imagePullPolicy: Always
ports:
- containerPort: 9200
name: masterelastic
volumeMounts:
- mountPath: /usr/share/elasticsearch/data
name: elastic-data
volumes:
- name: elastic-data
gcePersistentDisk:
pdName: YYYYYYYYYYYYYY
fsType: ext4
apiVersion: v1
kind: Service
metadata:
name: serviceelastic
labels:
name: serviceelastic
spec:
type: NodePort
ports:
- port: 9200
selector:
name: masterelastic
intranetumDEVELOPMENT
ELASTICSEARCH
- STAGE GCP - SNAPSHOT (INCREMENTAL)
STOP PRODUCTION ELASTIC (unmounted volume - optimal)
gcloud compute disks snapshot prod-elastic-plone-disk --snapshot-names elastic-plone-snapshot
START PRODUCTION ELASTIC
STOP STAGE ELASTIC
gcloud compute disks delete stage-elastic-plone-disk
gcloud compute disks create stage-elastic-plone-disk —source-snapshot=elastic-plone-snapshot
START STAGE ELASTIC
intranetumDEVELOPMENT
ELASTICSEARCH
- PROD GCP
kind: ReplicationController
spec:
replicas: 1
template:
spec:
containers:
- name: masterelastic
resources:
limits:
cpu: 1
image: eu.gcr.io/XXXXXXXX/elasticsearch
imagePullPolicy: Always
ports:
- containerPort: 9200
name: masterelastic
volumeMounts:
- mountPath: /usr/share/elasticsearch/data
name: elastic-data
volumes:
- name: elastic-data
gcePersistentDisk:
pdName: prod-elastic-plone-disk
fsType: ext4
apiVersion: v1
kind: Service
metadata:
name: serviceelastic
labels:
name: serviceelastic
spec:
type: NodePort
ports:
- port: 9200
selector:
name: masterelastic
intranetumDEVELOPMENT
ELASTICSEARCH
- PROD GCP - CLUSTER
apiVersion: v1
kind: Service
metadata:
name: elasticsearch-discovery
labels:
component: elasticsearch
role: master
spec:
selector:
component: elasticsearch
role: master
ports:
- name: transport
port: 9300
protocol: TCP
apiVersion: v1
kind: Service
metadata:
name: elasticsearch
labels:
component: elasticsearch
role: client
spec:
type: LoadBalancer
selector:
component: elasticsearch
role: client
ports:
- name: http
port: 9200
protocol: TCP
apiVersion: v1
kind: ServiceAccount
metadata:
name: elasticsearch
intranetumDEVELOPMENT
ELASTICSEARCH
- PROD GCP - CLUSTER
apiVersion: v1
kind: ReplicationController
metadata:
name: es-master
labels:
component: elasticsearch
role: master
spec:
replicas: 1
template:
metadata:
labels:
component: elasticsearch
role: master
spec:
serviceAccount: elasticsearch
containers:
- name: es-master
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: NODE_MASTER
value: "true"
- name: NODE_DATA
value: "false"
- name: HTTP_ENABLE
value: "false"
ports:
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}
intranetumDEVELOPMENT
ELASTICSEARCH
- PROD GCP - CLUSTER
apiVersion: v1
kind: ReplicationController
metadata:
name: es-data
labels:
component: elasticsearch
role: data
spec:
replicas: 1
template:
metadata:
labels:
component: elasticsearch
role: data
spec:
serviceAccount: elasticsearch
containers:
- name: es-data
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: NODE_MASTER
value: "false"
- name: HTTP_ENABLE
value: "false"
ports:
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}
intranetumDEVELOPMENT
ELASTICSEARCH
- PROD GCP - CLUSTER
apiVersion: v1
kind: ReplicationController
metadata:
name: es-client
labels:
component: elasticsearch
role: client
spec:
replicas: 1
template:
metadata:
labels:
component: elasticsearch
role: client
spec:
serviceAccount: elasticsearch
containers:
- name: es-client
securityContext:
capabilities:
add:
- IPC_LOCK
image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4
env:
- name: KUBERNETES_CA_CERTIFICATE_FILE
value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: "CLUSTER_NAME"
value: "myesdb"
- name: NODE_MASTER
value: "false"
- name: NODE_DATA
value: "false"
- name: HTTP_ENABLE
value: "true"
ports:
- containerPort: 9200
name: http
protocol: TCP
- containerPort: 9300
name: transport
protocol: TCP
volumeMounts:
- mountPath: /data
name: storage
volumes:
- name: storage
source:
emptyDir: {}
intranetumDEVELOPMENT
ELASTICSEARCH
- PROD GCP - CLUSTER
kubectl create -f service-account.yaml
kubectl create -f es-discovery-svc.yaml
kubectl create -f es-svc.yaml
kubectl create -f es-master-rc.yaml
kubectl create -f es-client-rc.yaml
kubectl create -f es-data-rc.yaml
kubectl scale --replicas=3 rc es-master
kubectl scale --replicas=2 rc es-client
kubectl scale --replicas=2 rc es-data
intranetumDEVELOPMENT
ZOPE/PLONE - DOCKERFILE PLONEBASE
# plonebase/Dockerfile
FROM python:2.7-slim
MAINTAINER Intranetum
ADD . /plone
WORKDIR /plone
ADD clau/key /root/.ssh/id_rsa
RUN set -x 
&& chmod 600 /root/.ssh/id_rsa 
&& ploneDeps=' 
openssh-client 
git-core 
libxml2 
libxslt1.1 
libjpeg62 
curl 
gcc 
vim 
libbz2-dev 
libc6-dev 
libncurses-dev 
libreadline-dev 
libsqlite3-dev 
libssl-dev 
libxslt-dev 
libxml2-dev 
libjpeg-dev 
make 
xz-utils 
zlib1g-dev 
build-essential 
' 
&& apt-get update && apt-get install -y $ploneDeps --no-install-
recommends
RUN ssh-keyscan -H XXXXXXXXXXX >> /root/.ssh/known_hosts 
&& python bootstrap.py && ./bin/buildout -v 
&& find /plone 
( -type d -a -name test -o -name tests ) 
-o ( -type f -a -name '*.pyc' -o -name '*.pyo' ) 
-exec rm -rf '{}' + 
&& chown -R www-data:www-data /plone/* 
&& chown www-data:www-data /plone/.installed.cfg 
&& chown www-data:www-data /plone/.mr.developer.cfg
USER www-data
# Configure and Run
ENTRYPOINT ["/plone/docker-entrypoint.sh"]
CMD ["/plone/bin/instance", "fg"]
intranetumDEVELOPMENT
ZOPE/PLONE - DOCKERFILE ZEOCLIENT
FROM eu.gcr.io/XXXXXXXXXX/plonebase
MAINTAINER Intranetum
RUN rm -rf /plone/src/*
# Bundle app source
ADD . /plone
WORKDIR /plone
RUN set -x 
&& rm -rf /plone/libsrc/* 
&& chown www-data /plone/.mr.developer.cfg 
&& ./bin/buildout -v -c intranetum.cfg 
&& chown -R www-data /plone
# Configure and Run
ENTRYPOINT ["/plone/docker-entrypoint.sh"]
CMD ["/plone/bin/instance", "fg"]
intranetumDEVELOPMENT
ZOPE/PLONE - ENTRYPOINT.SH
#!/bin/bash
set -e
if [ -n "$ZEOSERVER_PORT_8100_TCP_ADDR" ]; then
if [ -z "$ZEO_HOST" ]; then
ZEO_HOST=$ZEOSERVER_PORT_8100_TCP_ADDR
else
echo >&2 'warning: both ZEO_HOST and ZEOSERVER_PORT_8100_TCP_ADDR found'
echo >&2 " Connecting to ZEO_HOST ($ZEO_HOST)"
echo >&2 ' instead of the linked zeo container'
fi
fi
if [ -z "$ZEO_HOST" ]; then
echo >&2 'error: missing ZEO_HOST'
echo >&2 ' Did you forget to --link some_zeo_container:zeoserver or set an external db'
echo >&2 ' with -e ZEO_HOST=hostname'
exit 1
fi
if [ -n "$ELASTICSEARCH_PORT_9200_TCP_ADDR" ]; then
if [ -z "$ELASTIC_HOSTS" ]; then
ELASTIC_HOSTS=$ELASTICSEARCH_PORT_9200_TCP_ADDR
else
echo >&2 'warning: both ELASTIC_HOSTS and ELASTICSEARCH_PORT_9200_TCP_ADDR
found'
echo >&2 " Connecting to ELASTIC_HOSTS ($ELASTIC_HOSTS)"
echo >&2 ' instead of the linked elastic container'
fi
fi
….
if [ -z "$SHAREDBLOB" ]; then
echo >&2 'warning: no SHAREDBLOB setted: default off'
fi
set_config() {
echo "SET VAR"
echo $1
echo $2
key="$1"
value="$2"
sed -i -re 's!('"${key}"'s+)[^=]*$!1'"${value}"'!' /plone/parts/instance/etc/zope.conf
}
set_config_section() {
echo "SET VAR"
echo $1
echo $2
echo $3
section="$1"
key="$2"
value="$3"
sed -i -re '/<'"${section}"'>/ ,/</'"${section}"'>/ s!('"${key}"'s+)[^=]*$!1'"${value}"'!' /
plone/parts/instance/etc/zope.conf
}
echo "SET CONFIG"
set_config_section zeoclient server $ZEO_HOST:$ZEO_PORT
set_config_section zeoclient shared-blob-dir $SHAREDBLOB
set_config elasticsearch-hosts $ELASTIC_HOSTS
echo "START ZOPE"
exec "$@"
intranetumDEVELOPMENT
ZOPE/PLONE - INTRANETUM.CFG
[buildout]
extends =
buildout.cfg
versions = versions
always-checkout = true
parts =
instance
eggs +=
XXXX
auto-checkout +=
plone.rest
collective.elasticsearch
…
[instance]
recipe = plone.recipe.zope2instance
effective-user = www-data
http-address = 8080
zeo-address = $ZEO_ADDR:$ZEO_PORT
blob-storage = ${buildout:directory}/var/blobstorage/beta
zeo-client = on
shared-blob = $SHAREDBLOB
eggs = ${buildout:eggs}
environment-vars =
zope_i18n_compile_mo_files true
zope-conf-additional =
<product-config intranetum>
elasticsearch-hosts $ELASTIC_HOSTS
</product-config>
intranetumDEVELOPMENT
ZOPE/PLONE - DOCKERFILE ZEO
FROM eu.gcr.io/XXXXXXXXXX/zeoserver
MAINTAINER Intranetum
ADD . /plone
WORKDIR /plone
RUN set -x 
&& chown www-data /plone/.mr.developer.cfg 
&& rm -rf /plone/libsrc/* 
&& ./bin/buildout -v -c intranetum.cfg 
&& chown -R www-data /plone
# Expose
EXPOSE 8100
VOLUME /plone/var/filestorage
VOLUME /plone/var/blobstorage
# Configure and Run
ENTRYPOINT ["/plone/docker-entrypoint.sh"]
CMD ["/plone/bin/zeo", “fg"]
[buildout]
parts = zeo
always-checkout = true
extends = buildout.cfg
auto-checkout +=
plone.rest
collective.elasticsearch
…
[zeo]
recipe = plone.recipe.zeoserver
zeo-address = 8100
effective-user = www-data
eggs = ${buildout:eggs}
file-storage = filestorage/beta/Data.fs
blob-storage = blobstorage/beta
intranetumDEVELOPMENT
ZOPE/PLONE - STAGE YAML
apiVersion: v1
kind: ReplicationController
metadata:
name: masterzeoclient
labels:
name: masterzeoclient
spec:
replicas: 2
selector:
name: masterzeoclient
template:
metadata:
labels:
name: masterzeoclient
spec:
containers:
- image: eu.gcr.io/XXXXXXXXXXXX/zeoclient
imagePullPolicy: Always
resources:
limits:
cpu: 0.25
name: masterzeoclient
env:
- name: ELASTIC_HOSTS
value: "serviceelastic"
- name: ZEO_HOST
value: "servicezeo"
…
- name: SHAREDBLOB
value: "off"
ports:
- containerPort: 8080
name: masterzeoclient
apiVersion: v1
kind: ReplicationController
…
spec:
replicas: 1
selector:
name: masterzeo
template:
metadata:
labels:
name: masterzeo
spec:
containers:
- name: masterzeo
resources:
limits:
cpu: 0.4
image: eu.gcr.io/XXXXXXXXXXXX/zeoserver
imagePullPolicy: Always
ports:
- containerPort: 8100
name: masterzeo
volumeMounts:
- mountPath: /plone/var/filestorage
name: zeo-filestorage
- mountPath: /plone/var/blobstorage
name: zeo-blobstorage
volumes:
- name: zeo-filestorage
gcePersistentDisk:
pdName: XXXXXXXXXXXX
fsType: ext4
- name: zeo-blobstorage
gcePersistentDisk:
pdName: zeo-blobstorage-disk
fsType: ext4
intranetumDEVELOPMENT
ZOPE/PLONE - STAGE YAML
apiVersion: v1
kind: Service
metadata:
labels:
name: servicezeoclient
name: servicezeoclient
spec:
type: LoadBalancer
ports:
# The port that this service should serve on.
- port: 80
targetPort: 8080
protocol: TCP
# Label keys and values that must match in order to receive traffic for this service.
selector:
name: masterzeoclient
apiVersion: v1
kind: Service
metadata:
name: servicezeo
labels:
name: servicezeo
spec:
type: NodePort
ports:
- port: 8100
selector:
name: masterzeo
intranetumDEVELOPMENT
ZOPE/PLONE - PRODUCTION YAML
apiVersion: v1
kind: ReplicationController
…
spec:
replicas: 3
selector:
name: masterzeoclient
template:
metadata:
labels:
name: masterzeoclient
spec:
containers:
- image: eu.gcr.io/XXXXXXXXXXX/zeoclient
imagePullPolicy: Always
resources:
limits:
cpu: 0.8
name: masterzeoclient
env:
- name: ELASTIC_HOSTS
value: "serviceelastic"
- name: ZEO_HOST
value: "servicezeo"
- name: SHAREDBLOB
value: "on"
ports:
- containerPort: 8080
name: masterzeoclient
volumeMounts:
- mountPath: /plone/var/blobstorage
name: nfs-zeo
volumes:
- name: nfs-zeo
persistentVolumeClaim:
claimName: nfs-zeo
apiVersion: v1
kind: ReplicationController
…
spec:
replicas: 1
selector:
name: masterzeo
template:
metadata:
labels:
name: masterzeo
spec:
containers:
- name: masterzeo
resources:
limits:
cpu: 0.5
image: eu.gcr.io/XXXXXXXXXXXXXXX/zeoserver
imagePullPolicy: Always
ports:
- containerPort: 8100
name: masterzeo
volumeMounts:
- mountPath: /plone/var/filestorage
name: zeo-filestorage
- mountPath: /plone/var/blobstorage
name: nfs-zeo
volumes:
- name: zeo-filestorage
gcePersistentDisk:
pdName: zeo-filestorage-disk-prod
fsType: ext4
- name: nfs-zeo
persistentVolumeClaim:
claimName: nfs-zeo
intranetumDEVELOPMENT
ZOPE/PLONE
- PROD GCP - UPDATE
kubectl rolling-update -f zeo.yaml
kubectl rolling-update -f zeoclient.yaml
kubectl scale --replicas=3 rc zeoclient
▸ SNAPSHOTS
▸ NFS
▸ https://goo.gl/Zmub08
▸ HEALTH CHECKS
intranetumDEVELOPMENT
TESTING - GROUP 1
▸ LOCAL TESTING
▸ docker-compose Elasticsearch
▸ GROUP CONFIG ENV vars to TESTING ENV
▸ COMMIT ON CODE
▸ JENKINS testing INTEGRATION on NATIVE Buildout agains TESTING ENV with
SETTING JOBS
▸ JENKINS BUILD Docker Images and PUSH
▸ JENKINS DEPLOY TESTING - ACCEPTANCE TESTS
▸ JENKINS DEPLOY STAGE
▸ MANUAL TRIGGER DEPLOY PRODUCTION
intranetumDEVELOPMENT
JOBS
▸ UPGRADE STEPS ON STAGE / PRODUCTION
▸ MAINTENENCE JOBS
# upgrade-b16.py
def upgrade(site):
site.doNothing()
if __name__ == "__main__":
output = upgrade(app.Plone)
logger.info(output)
# upgrade-b16.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: upgrade-b16
spec:
template:
metadata:
name: upgrade-b16
spec:
containers:
- name: plone
image: eu.gcr.io/XXXXXXXX/upgrade-plone
command: [“/plone/bin/instance“, “/plone/upgrade-b16.py“]
restartPolicy: Never
intranetumDEVELOPMENT
JOBS
# upgrade-plone/Dockerfile
FROM eu.gcr.io/XXXXXXXXXXXX/zeoclient
MAINTAINER Intranetum
ADD . /plone
kubectl create -f ./upgrade-b16.yaml
intranetumDEVELOPMENT
MAIN CLOSED COMPONENTS
▸ FRONT END (NODE + ANGULAR 2)
▸ FRONT END MIDDLEWARE
▸ HORUS (AUTHZ AUTHN)
▸ 3 components PY 3.5 ASYNCIO
▸ BEATS (MACHINE LEARNING ENGINE)
▸ >15 HIPOS (ASYNC components PY3.5/GO/ERLANG)
intranetumDEVELOPMENT
DOCKER FROM 0 - HIPO
FROM python:3.5.1
RUN apt-get update -y 
&& apt-get install -y netcat 
&& apt-get autoremove -y 
&& rm -rf /var/lib/apt/lists/*
RUN pip3.5 install aioamqp python-logstash
RUN pip3.5 install https://pypi.python.org/packages/source/p/protobuf/protobuf-3.0.0b2.tar.gz
COPY docker-entrypoint.sh /
COPY run.py /
COPY api_pb2.py /
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["python3", "/run.py"]
intranetum
DEMO
DEPLOY II …
intranetumSO…
CONCLUSIONS
▸ Split arch in groups of components to test, deploy and develop
▸ K8s much powerful than docker-compose / swarm (rolling-
updates / secrets / scaling / jobs)
▸ Optimization Service on K8S vs SaaS (not everything in containers)
▸ Docker from moment 0 (easier than later)
▸ Tests in Docker, Dev outside Docker (find a way to isolate
components and connect to the testing/stage cluster)
▸ Proxy NPM / Debian packages / Pypi / … CI / CD is hard
intranetumI START TO BE HUNGRY…
FUTURE
▸ Ansible on docker ? - We use buildout
▸ Jenkins jobs to rollback to version X
▸ K8S jobs to manage backups/testing data set
▸ Docker to build Jenkins to build Docker
▸ Elastic search cluster deployment
▸ Remove keys from docker
▸ Local k8s hardware (Rancher)
▸ Upgrade jobs testing Stage
▸ TESTS on k8s cluster (snapshot)
intranetum
WE ARE
HIRING!jobs@iskra.cat
intranetum…REALLY HUNGRY
OPEN QUESTIONS
▸ Container data manager VS SaaS data manager
▸ Volumes ZFS/NFS snapshoots
▸ Performance on computation container
▸ Persistent Disks vs local SSD vs Buckets vs RAM disk
▸ Monitoring ?
▸ More questions ?
intranetum
THANK YOU!
r.navarro@iskra.cat

Más contenido relacionado

La actualidad más candente

Forget the Web
Forget the WebForget the Web
Forget the WebRemy Sharp
 
DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술
DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술
DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술John Kim
 
ApacheConNA 2015: What's new in Apache httpd 2.4
ApacheConNA 2015: What's new in Apache httpd 2.4ApacheConNA 2015: What's new in Apache httpd 2.4
ApacheConNA 2015: What's new in Apache httpd 2.4Jim Jagielski
 
Nuts and Bolts of WebSocket Devoxx 2014
Nuts and Bolts of WebSocket Devoxx 2014Nuts and Bolts of WebSocket Devoxx 2014
Nuts and Bolts of WebSocket Devoxx 2014Arun Gupta
 
실시간 서비스 플랫폼 개발 사례
실시간 서비스 플랫폼 개발 사례실시간 서비스 플랫폼 개발 사례
실시간 서비스 플랫폼 개발 사례John Kim
 
A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices Nebulaworks
 
Windows Attacks AT is the new black
Windows Attacks   AT is the new blackWindows Attacks   AT is the new black
Windows Attacks AT is the new blackRob Fuller
 
cache concepts and varnish-cache
cache concepts and varnish-cachecache concepts and varnish-cache
cache concepts and varnish-cacheMarc Cortinas Val
 
Facebook的缓存系统
Facebook的缓存系统Facebook的缓存系统
Facebook的缓存系统yiditushe
 
Common Pitfalls for your Drupal Site, and How to Avoid Them
Common Pitfalls for your Drupal Site, and How to Avoid ThemCommon Pitfalls for your Drupal Site, and How to Avoid Them
Common Pitfalls for your Drupal Site, and How to Avoid ThemAcquia
 
AWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp VaultAWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp VaultGrzegorz Adamowicz
 
A Case Study in Attacking KeePass
A Case Study in Attacking KeePassA Case Study in Attacking KeePass
A Case Study in Attacking KeePassWill Schroeder
 
Die .htaccess richtig nutzen
Die .htaccess richtig nutzenDie .htaccess richtig nutzen
Die .htaccess richtig nutzenWalter Ebert
 
Meetup Performance
Meetup PerformanceMeetup Performance
Meetup PerformanceGreg Whalin
 
Keybase Vault Auto-Unseal HashiTalks2020
Keybase Vault Auto-Unseal HashiTalks2020Keybase Vault Auto-Unseal HashiTalks2020
Keybase Vault Auto-Unseal HashiTalks2020Bas Meijer
 
How to build a High Performance PSGI/Plack Server
How to build a High Performance PSGI/Plack Server How to build a High Performance PSGI/Plack Server
How to build a High Performance PSGI/Plack Server Masahiro Nagano
 
HTTPS + Let's Encrypt
HTTPS + Let's EncryptHTTPS + Let's Encrypt
HTTPS + Let's EncryptWalter Ebert
 
Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013
Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013
Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013Puppet
 

La actualidad más candente (20)

Forget the Web
Forget the WebForget the Web
Forget the Web
 
DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술
DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술
DEVIEW - 오픈소스를 활용한 분산아키텍처 구현기술
 
ApacheConNA 2015: What's new in Apache httpd 2.4
ApacheConNA 2015: What's new in Apache httpd 2.4ApacheConNA 2015: What's new in Apache httpd 2.4
ApacheConNA 2015: What's new in Apache httpd 2.4
 
Nuts and Bolts of WebSocket Devoxx 2014
Nuts and Bolts of WebSocket Devoxx 2014Nuts and Bolts of WebSocket Devoxx 2014
Nuts and Bolts of WebSocket Devoxx 2014
 
실시간 서비스 플랫폼 개발 사례
실시간 서비스 플랫폼 개발 사례실시간 서비스 플랫폼 개발 사례
실시간 서비스 플랫폼 개발 사례
 
A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices A Hands-on Introduction on Terraform Best Concepts and Best Practices
A Hands-on Introduction on Terraform Best Concepts and Best Practices
 
Windows Attacks AT is the new black
Windows Attacks   AT is the new blackWindows Attacks   AT is the new black
Windows Attacks AT is the new black
 
cache concepts and varnish-cache
cache concepts and varnish-cachecache concepts and varnish-cache
cache concepts and varnish-cache
 
Facebook的缓存系统
Facebook的缓存系统Facebook的缓存系统
Facebook的缓存系统
 
Common Pitfalls for your Drupal Site, and How to Avoid Them
Common Pitfalls for your Drupal Site, and How to Avoid ThemCommon Pitfalls for your Drupal Site, and How to Avoid Them
Common Pitfalls for your Drupal Site, and How to Avoid Them
 
AWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp VaultAWS DevOps - Terraform, Docker, HashiCorp Vault
AWS DevOps - Terraform, Docker, HashiCorp Vault
 
A Case Study in Attacking KeePass
A Case Study in Attacking KeePassA Case Study in Attacking KeePass
A Case Study in Attacking KeePass
 
Die .htaccess richtig nutzen
Die .htaccess richtig nutzenDie .htaccess richtig nutzen
Die .htaccess richtig nutzen
 
Meetup Performance
Meetup PerformanceMeetup Performance
Meetup Performance
 
Keybase Vault Auto-Unseal HashiTalks2020
Keybase Vault Auto-Unseal HashiTalks2020Keybase Vault Auto-Unseal HashiTalks2020
Keybase Vault Auto-Unseal HashiTalks2020
 
Memcached Study
Memcached StudyMemcached Study
Memcached Study
 
How to build a High Performance PSGI/Plack Server
How to build a High Performance PSGI/Plack Server How to build a High Performance PSGI/Plack Server
How to build a High Performance PSGI/Plack Server
 
Unity Makes Strength
Unity Makes StrengthUnity Makes Strength
Unity Makes Strength
 
HTTPS + Let's Encrypt
HTTPS + Let's EncryptHTTPS + Let's Encrypt
HTTPS + Let's Encrypt
 
Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013
Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013
Deploying VMware vCloud Hybrid Service with Puppet - PuppetConf 2013
 

Destacado

Think innovation issue 4 share - scamper
Think innovation issue 4   share - scamperThink innovation issue 4   share - scamper
Think innovation issue 4 share - scamperThinkInnovation
 
Entrepreneurship In Corporations
Entrepreneurship In CorporationsEntrepreneurship In Corporations
Entrepreneurship In CorporationsSonny Navaratnam
 
Tot Community Development
Tot Community DevelopmentTot Community Development
Tot Community DevelopmentAri Raharjo
 
Get Out Of Your Cube Evo Terra
Get Out Of Your Cube Evo TerraGet Out Of Your Cube Evo Terra
Get Out Of Your Cube Evo TerraEvo Terra
 
Latihan survey kepuasan konsumen bank
Latihan survey kepuasan konsumen bankLatihan survey kepuasan konsumen bank
Latihan survey kepuasan konsumen bankAri Raharjo
 
Love Doesn't Always Have a Happy Ending
Love Doesn't Always Have a Happy EndingLove Doesn't Always Have a Happy Ending
Love Doesn't Always Have a Happy EndingMrs. McCabe
 
Mobiles by Julie Frith
Mobiles by Julie FrithMobiles by Julie Frith
Mobiles by Julie Frithfrithmobiles
 
Nous reptes a la web 20
Nous reptes a la web 20Nous reptes a la web 20
Nous reptes a la web 20Nuria Alart
 
Bacon Cheeseburgers
Bacon CheeseburgersBacon Cheeseburgers
Bacon CheeseburgersMrs. McCabe
 
Luminous Landscapes
Luminous LandscapesLuminous Landscapes
Luminous LandscapesJan Blencowe
 
Greenland Prpoerty Mgt
Greenland Prpoerty MgtGreenland Prpoerty Mgt
Greenland Prpoerty MgtJulie Kim
 
Taking notes on_research_sources
Taking notes on_research_sourcesTaking notes on_research_sources
Taking notes on_research_sourceschrissienehrenberg
 

Destacado (20)

Think innovation issue 4 share - scamper
Think innovation issue 4   share - scamperThink innovation issue 4   share - scamper
Think innovation issue 4 share - scamper
 
Entrepreneurship In Corporations
Entrepreneurship In CorporationsEntrepreneurship In Corporations
Entrepreneurship In Corporations
 
Aula 2 good manners obedience
Aula 2 good manners   obedienceAula 2 good manners   obedience
Aula 2 good manners obedience
 
Vfinal V1
Vfinal V1Vfinal V1
Vfinal V1
 
Birds
BirdsBirds
Birds
 
Tot Community Development
Tot Community DevelopmentTot Community Development
Tot Community Development
 
Get Out Of Your Cube Evo Terra
Get Out Of Your Cube Evo TerraGet Out Of Your Cube Evo Terra
Get Out Of Your Cube Evo Terra
 
Latihan survey kepuasan konsumen bank
Latihan survey kepuasan konsumen bankLatihan survey kepuasan konsumen bank
Latihan survey kepuasan konsumen bank
 
La Comunicació
La ComunicacióLa Comunicació
La Comunicació
 
Love Doesn't Always Have a Happy Ending
Love Doesn't Always Have a Happy EndingLove Doesn't Always Have a Happy Ending
Love Doesn't Always Have a Happy Ending
 
Academic summary
Academic summaryAcademic summary
Academic summary
 
Mobiles by Julie Frith
Mobiles by Julie FrithMobiles by Julie Frith
Mobiles by Julie Frith
 
Nous reptes a la web 20
Nous reptes a la web 20Nous reptes a la web 20
Nous reptes a la web 20
 
Bacon Cheeseburgers
Bacon CheeseburgersBacon Cheeseburgers
Bacon Cheeseburgers
 
Luminous Landscapes
Luminous LandscapesLuminous Landscapes
Luminous Landscapes
 
Greenland Prpoerty Mgt
Greenland Prpoerty MgtGreenland Prpoerty Mgt
Greenland Prpoerty Mgt
 
Multitasquesweb
MultitasqueswebMultitasquesweb
Multitasquesweb
 
Tropical Fish
Tropical FishTropical Fish
Tropical Fish
 
Taking notes on_research_sources
Taking notes on_research_sourcesTaking notes on_research_sources
Taking notes on_research_sources
 
Musicalsby144
Musicalsby144Musicalsby144
Musicalsby144
 

Similar a How containers helped a SaaS startup be developed and go live

Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016Zabbix
 
Intro to containerization
Intro to containerizationIntro to containerization
Intro to containerizationBalint Pato
 
HPE Storage KubeCon US 2018 Workshop
HPE Storage KubeCon US 2018 WorkshopHPE Storage KubeCon US 2018 Workshop
HPE Storage KubeCon US 2018 WorkshopMichael Mattsson
 
Drupal Efficiency - Coding, Deployment, Scaling
Drupal Efficiency - Coding, Deployment, ScalingDrupal Efficiency - Coding, Deployment, Scaling
Drupal Efficiency - Coding, Deployment, Scalingsmattoon
 
Drupal Efficiency using open source technologies from Sun
Drupal Efficiency using open source technologies from SunDrupal Efficiency using open source technologies from Sun
Drupal Efficiency using open source technologies from Sunsmattoon
 
Scaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesScaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesRobert Lemke
 
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps dayAprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps dayPlain Concepts
 
Containers - Portable, repeatable user-oriented application delivery. Build, ...
Containers - Portable, repeatable user-oriented application delivery. Build, ...Containers - Portable, repeatable user-oriented application delivery. Build, ...
Containers - Portable, repeatable user-oriented application delivery. Build, ...Walid Shaari
 
Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...
Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...
Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...Patrick Chanezon
 
High Performance Computing (HPC) in cloud
High Performance Computing (HPC) in cloudHigh Performance Computing (HPC) in cloud
High Performance Computing (HPC) in cloudAccubits Technologies
 
Practical Chaos Engineering
Practical Chaos EngineeringPractical Chaos Engineering
Practical Chaos EngineeringSIGHUP
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesYevgeniy Brikman
 
Ekoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's MethodologyEkoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's Methodologybugcrowd
 
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:Invent
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:InventHow Zalando runs Kubernetes clusters at scale on AWS - AWS re:Invent
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:InventHenning Jacobs
 
Docker Security workshop slides
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slidesDocker, Inc.
 
Incrementalism: An Industrial Strategy For Adopting Modern Automation
Incrementalism: An Industrial Strategy For Adopting Modern AutomationIncrementalism: An Industrial Strategy For Adopting Modern Automation
Incrementalism: An Industrial Strategy For Adopting Modern AutomationSean Chittenden
 
Weave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapWeave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapPatrick Chanezon
 
30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as CodeGuido Schmutz
 
Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...
Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...
Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...OpenCredo
 

Similar a How containers helped a SaaS startup be developed and go live (20)

Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
Erik Skytthe - Monitoring Mesos, Docker, Containers with Zabbix | ZabConf2016
 
Intro to containerization
Intro to containerizationIntro to containerization
Intro to containerization
 
HPE Storage KubeCon US 2018 Workshop
HPE Storage KubeCon US 2018 WorkshopHPE Storage KubeCon US 2018 Workshop
HPE Storage KubeCon US 2018 Workshop
 
Drupal Efficiency - Coding, Deployment, Scaling
Drupal Efficiency - Coding, Deployment, ScalingDrupal Efficiency - Coding, Deployment, Scaling
Drupal Efficiency - Coding, Deployment, Scaling
 
Drupal Efficiency using open source technologies from Sun
Drupal Efficiency using open source technologies from SunDrupal Efficiency using open source technologies from Sun
Drupal Efficiency using open source technologies from Sun
 
Scaleable PHP Applications in Kubernetes
Scaleable PHP Applications in KubernetesScaleable PHP Applications in Kubernetes
Scaleable PHP Applications in Kubernetes
 
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps dayAprovisionamiento multi-proveedor con Terraform  - Plain Concepts DevOps day
Aprovisionamiento multi-proveedor con Terraform - Plain Concepts DevOps day
 
Containers - Portable, repeatable user-oriented application delivery. Build, ...
Containers - Portable, repeatable user-oriented application delivery. Build, ...Containers - Portable, repeatable user-oriented application delivery. Build, ...
Containers - Portable, repeatable user-oriented application delivery. Build, ...
 
Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...
Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...
Docker Azure Friday OSS March 2017 - Developing and deploying Java & Linux on...
 
High Performance Computing (HPC) in cloud
High Performance Computing (HPC) in cloudHigh Performance Computing (HPC) in cloud
High Performance Computing (HPC) in cloud
 
Practical Chaos Engineering
Practical Chaos EngineeringPractical Chaos Engineering
Practical Chaos Engineering
 
Reusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modulesReusable, composable, battle-tested Terraform modules
Reusable, composable, battle-tested Terraform modules
 
Ekoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's MethodologyEkoparty 2017 - The Bug Hunter's Methodology
Ekoparty 2017 - The Bug Hunter's Methodology
 
Docker 101 Checonf 2016
Docker 101 Checonf 2016Docker 101 Checonf 2016
Docker 101 Checonf 2016
 
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:Invent
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:InventHow Zalando runs Kubernetes clusters at scale on AWS - AWS re:Invent
How Zalando runs Kubernetes clusters at scale on AWS - AWS re:Invent
 
Docker Security workshop slides
Docker Security workshop slidesDocker Security workshop slides
Docker Security workshop slides
 
Incrementalism: An Industrial Strategy For Adopting Modern Automation
Incrementalism: An Industrial Strategy For Adopting Modern AutomationIncrementalism: An Industrial Strategy For Adopting Modern Automation
Incrementalism: An Industrial Strategy For Adopting Modern Automation
 
Weave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 RecapWeave User Group Talk - DockerCon 2017 Recap
Weave User Group Talk - DockerCon 2017 Recap
 
30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code30 Minutes to the Analytics Platform with Infrastructure as Code
30 Minutes to the Analytics Platform with Infrastructure as Code
 
Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...
Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...
Hashidays London 2017 - Evolving your Infrastructure with Terraform By Nicki ...
 

Más de Ramon Navarro

Pipelines for model deployment
Pipelines for model deploymentPipelines for model deployment
Pipelines for model deploymentRamon Navarro
 
CI on large open source software : Plone & Plone 5 is here!
CI on large open source software : Plone & Plone 5 is here!CI on large open source software : Plone & Plone 5 is here!
CI on large open source software : Plone & Plone 5 is here!Ramon Navarro
 
Multilingual sites in plone
Multilingual sites in ploneMultilingual sites in plone
Multilingual sites in ploneRamon Navarro
 
Presentacio meetup Python BCN
Presentacio meetup Python BCNPresentacio meetup Python BCN
Presentacio meetup Python BCNRamon Navarro
 
plone.app.multilingual
plone.app.multilingual plone.app.multilingual
plone.app.multilingual Ramon Navarro
 
WPD Barcelona 2008 Què és Plone ?
WPD Barcelona 2008 Què és Plone ?WPD Barcelona 2008 Què és Plone ?
WPD Barcelona 2008 Què és Plone ?Ramon Navarro
 

Más de Ramon Navarro (10)

Guillotina
GuillotinaGuillotina
Guillotina
 
Pipelines for model deployment
Pipelines for model deploymentPipelines for model deployment
Pipelines for model deployment
 
Plone server
Plone serverPlone server
Plone server
 
CI on large open source software : Plone & Plone 5 is here!
CI on large open source software : Plone & Plone 5 is here!CI on large open source software : Plone & Plone 5 is here!
CI on large open source software : Plone & Plone 5 is here!
 
Pyramid
PyramidPyramid
Pyramid
 
Multilingual sites in plone
Multilingual sites in ploneMultilingual sites in plone
Multilingual sites in plone
 
Cafè amb web
Cafè amb webCafè amb web
Cafè amb web
 
Presentacio meetup Python BCN
Presentacio meetup Python BCNPresentacio meetup Python BCN
Presentacio meetup Python BCN
 
plone.app.multilingual
plone.app.multilingual plone.app.multilingual
plone.app.multilingual
 
WPD Barcelona 2008 Què és Plone ?
WPD Barcelona 2008 Què és Plone ?WPD Barcelona 2008 Què és Plone ?
WPD Barcelona 2008 Què és Plone ?
 

Último

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024The Digital Insurer
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Bhuvaneswari Subramani
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...Zilliz
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Orbitshub
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxRustici Software
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusZilliz
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 

Último (20)

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 

How containers helped a SaaS startup be developed and go live

  • 1. HOW CONTAINERS HELPED A SAAS STARTUP BE DEVELOPED INTRANETUM
  • 2. intranetum Ramon Navarro Bosch CTO iskra.cat CTO intranetum.com FWT plone.org WHO AM I? Machine Learning - Python - Angular 2 Agile Test Driven Development Always a sysadmin in my heart In love with docker / k8s
  • 3. intranetumWHAT IS INTRANETUM? BRAIN TO CLASSIFY ▸ Knowledge management ▸ Files, Notes, Links ▸ Deep learning by scope/user ▸ Auto classify information ▸ Search information ▸ Less time to find what you are looking for + discover knowledge inside the company
  • 4. intranetumTHE PROBLEM FAST, AGILE, TEST, COMPLEX ▸ We needed to develop a SaaS solution in 3 months (proof) ▸ 1 Desember - 23 Febrary (4YFN) ▸ A team of 2 developers ▸ Needs to be modular for evolution ▸ Needs to scale fast ▸ Initial architecture design showed 10 different components ( 40 by the end of February )
  • 7.
  • 8.
  • 10. intranetumCOMPONENTS DOCKER ▸ General Adopted Container Solution ▸ Native support Mac OS X + Linux + Windows ▸ Standardization of build process of images ▸ Service is not an application, is a container ▸ Ports, volumes and build layers management ▸ Definition of repository of images
  • 11. intranetumCOMPONENTS KUBERNETTES (K8S) ▸ Same deployment infrastructure for production, testing, development and stage ▸ Same deployment infrastructure for cloud and in-house ▸ Load balancer integrated ▸ Management of HDD ▸ Secret configuration management ▸ Internal network discovery (DNS) ▸ Jobs
  • 12. intranetumCOMPONENTS GOOGLE CLOUD PLATFORM ▸ Clusters for stage / production ▸ HTTPS load balancer ▸ 100% integrated with k8s ▸ Full system monitoring ▸ Scaling fast ▸ Private Docker Registry
  • 13. intranetumCOMPONENTS JENKINS + GIT ▸ Continuous Integration ▸ Continuous Deployment ▸ Orchestration of versioning and stability ▸ Workflow of testing & deployment ▸ quality, Quality, QUALITY! ▸ testing, Testing, TESTING!
  • 14. intranetumCOMPONENTS METRICS (ELK) ▸ mesure, Mesure, MESURE ▸ Continuous Metrics ▸ Centralized Log Service ▸ Nice diagrams to show VCs
  • 16. intranetumGROUPING CLUSTER | GROUPS | CONTAINERS STAGE GROUP2 GROUP3 GROUP4GROUP1 RC RC RC C1 C2 C1 C2 C3 C1
  • 17. intranetumDEVELOPMENT MAIN OPEN SOURCE COMPONENTS ▸ PLONE (STORAGE/CONTENT TYPES/SECURITY) ▸ ELASTIC SEARCH (SEARCH / LOGGING) ▸ ZODB (DATABASE) ▸ REDIS (CACHE) ▸ LDAP (USER/PERMISSION DB) ▸ RABBITMQ (QUEUE)
  • 18. intranetumDEVELOPMENT MAIN OPEN SOURCE FRAMEWORKS ▸ ANGULAR2 ▸ TENSORFLOW ▸ NLTP ▸ PROTOBUFFERS ▸ PYRAMID + ASYNCIO
  • 20. intranetumDEVELOPMENT SEMANTIC GROUP1 CMS - DEV TEST CLUSTER CLOUD ELAS TIC ZOPE GROUP2 GROUP3 GROUP4 GROUP1 ZEO NFS
  • 21. intranetumDEVELOPMENT ELASTICSEARCH - DOCKERFILE FROM elasticsearch:1.7 MAINTAINER Iskra # Expose EXPOSE 9200 EXPOSE 9300 # Cors enabled for testing CMD ["elasticsearch", "-Des.cluster.name=intranetum", "-Des.http.cors.enabled=true"]
  • 22. intranetumDEVELOPMENT ELASTICSEARCH - LOCAL DEV/TEST DOCKER COMPOSE elasticsearch: image: elasticsearch:1.7 ports: - "9200:9200" - "9300:9300" volumes: - ./esconfig:/usr/share/elasticsearch/config - ./esdata:/usr/share/elasticsearch/data command: ["elasticsearch", "-Des.network.publish_host=localhost", "-Dhttp.cors.enabled=true", "-Dhttp.cors.allow- origin='*'"]
  • 23. intranetumDEVELOPMENT ELASTICSEARCH - STAGE GCP kind: ReplicationController spec: replicas: 1 template: spec: containers: - name: masterelastic resources: limits: cpu: 0.250 image: eu.gcr.io/XXXXXXXX/elasticsearch imagePullPolicy: Always ports: - containerPort: 9200 name: masterelastic volumeMounts: - mountPath: /usr/share/elasticsearch/data name: elastic-data volumes: - name: elastic-data gcePersistentDisk: pdName: YYYYYYYYYYYYYY fsType: ext4 apiVersion: v1 kind: Service metadata: name: serviceelastic labels: name: serviceelastic spec: type: NodePort ports: - port: 9200 selector: name: masterelastic
  • 24. intranetumDEVELOPMENT ELASTICSEARCH - STAGE GCP - SNAPSHOT (INCREMENTAL) STOP PRODUCTION ELASTIC (unmounted volume - optimal) gcloud compute disks snapshot prod-elastic-plone-disk --snapshot-names elastic-plone-snapshot START PRODUCTION ELASTIC STOP STAGE ELASTIC gcloud compute disks delete stage-elastic-plone-disk gcloud compute disks create stage-elastic-plone-disk —source-snapshot=elastic-plone-snapshot START STAGE ELASTIC
  • 25. intranetumDEVELOPMENT ELASTICSEARCH - PROD GCP kind: ReplicationController spec: replicas: 1 template: spec: containers: - name: masterelastic resources: limits: cpu: 1 image: eu.gcr.io/XXXXXXXX/elasticsearch imagePullPolicy: Always ports: - containerPort: 9200 name: masterelastic volumeMounts: - mountPath: /usr/share/elasticsearch/data name: elastic-data volumes: - name: elastic-data gcePersistentDisk: pdName: prod-elastic-plone-disk fsType: ext4 apiVersion: v1 kind: Service metadata: name: serviceelastic labels: name: serviceelastic spec: type: NodePort ports: - port: 9200 selector: name: masterelastic
  • 26. intranetumDEVELOPMENT ELASTICSEARCH - PROD GCP - CLUSTER apiVersion: v1 kind: Service metadata: name: elasticsearch-discovery labels: component: elasticsearch role: master spec: selector: component: elasticsearch role: master ports: - name: transport port: 9300 protocol: TCP apiVersion: v1 kind: Service metadata: name: elasticsearch labels: component: elasticsearch role: client spec: type: LoadBalancer selector: component: elasticsearch role: client ports: - name: http port: 9200 protocol: TCP apiVersion: v1 kind: ServiceAccount metadata: name: elasticsearch
  • 27. intranetumDEVELOPMENT ELASTICSEARCH - PROD GCP - CLUSTER apiVersion: v1 kind: ReplicationController metadata: name: es-master labels: component: elasticsearch role: master spec: replicas: 1 template: metadata: labels: component: elasticsearch role: master spec: serviceAccount: elasticsearch containers: - name: es-master securityContext: capabilities: add: - IPC_LOCK image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4 env: - name: KUBERNETES_CA_CERTIFICATE_FILE value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: "CLUSTER_NAME" value: "myesdb" - name: NODE_MASTER value: "true" - name: NODE_DATA value: "false" - name: HTTP_ENABLE value: "false" ports: - containerPort: 9300 name: transport protocol: TCP volumeMounts: - mountPath: /data name: storage volumes: - name: storage source: emptyDir: {}
  • 28. intranetumDEVELOPMENT ELASTICSEARCH - PROD GCP - CLUSTER apiVersion: v1 kind: ReplicationController metadata: name: es-data labels: component: elasticsearch role: data spec: replicas: 1 template: metadata: labels: component: elasticsearch role: data spec: serviceAccount: elasticsearch containers: - name: es-data securityContext: capabilities: add: - IPC_LOCK image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4 env: - name: KUBERNETES_CA_CERTIFICATE_FILE value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: "CLUSTER_NAME" value: "myesdb" - name: NODE_MASTER value: "false" - name: HTTP_ENABLE value: "false" ports: - containerPort: 9300 name: transport protocol: TCP volumeMounts: - mountPath: /data name: storage volumes: - name: storage source: emptyDir: {}
  • 29. intranetumDEVELOPMENT ELASTICSEARCH - PROD GCP - CLUSTER apiVersion: v1 kind: ReplicationController metadata: name: es-client labels: component: elasticsearch role: client spec: replicas: 1 template: metadata: labels: component: elasticsearch role: client spec: serviceAccount: elasticsearch containers: - name: es-client securityContext: capabilities: add: - IPC_LOCK image: quay.io/pires/docker-elasticsearch-kubernetes:1.7.1-4 env: - name: KUBERNETES_CA_CERTIFICATE_FILE value: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace - name: "CLUSTER_NAME" value: "myesdb" - name: NODE_MASTER value: "false" - name: NODE_DATA value: "false" - name: HTTP_ENABLE value: "true" ports: - containerPort: 9200 name: http protocol: TCP - containerPort: 9300 name: transport protocol: TCP volumeMounts: - mountPath: /data name: storage volumes: - name: storage source: emptyDir: {}
  • 30. intranetumDEVELOPMENT ELASTICSEARCH - PROD GCP - CLUSTER kubectl create -f service-account.yaml kubectl create -f es-discovery-svc.yaml kubectl create -f es-svc.yaml kubectl create -f es-master-rc.yaml kubectl create -f es-client-rc.yaml kubectl create -f es-data-rc.yaml kubectl scale --replicas=3 rc es-master kubectl scale --replicas=2 rc es-client kubectl scale --replicas=2 rc es-data
  • 31. intranetumDEVELOPMENT ZOPE/PLONE - DOCKERFILE PLONEBASE # plonebase/Dockerfile FROM python:2.7-slim MAINTAINER Intranetum ADD . /plone WORKDIR /plone ADD clau/key /root/.ssh/id_rsa RUN set -x && chmod 600 /root/.ssh/id_rsa && ploneDeps=' openssh-client git-core libxml2 libxslt1.1 libjpeg62 curl gcc vim libbz2-dev libc6-dev libncurses-dev libreadline-dev libsqlite3-dev libssl-dev libxslt-dev libxml2-dev libjpeg-dev make xz-utils zlib1g-dev build-essential ' && apt-get update && apt-get install -y $ploneDeps --no-install- recommends RUN ssh-keyscan -H XXXXXXXXXXX >> /root/.ssh/known_hosts && python bootstrap.py && ./bin/buildout -v && find /plone ( -type d -a -name test -o -name tests ) -o ( -type f -a -name '*.pyc' -o -name '*.pyo' ) -exec rm -rf '{}' + && chown -R www-data:www-data /plone/* && chown www-data:www-data /plone/.installed.cfg && chown www-data:www-data /plone/.mr.developer.cfg USER www-data # Configure and Run ENTRYPOINT ["/plone/docker-entrypoint.sh"] CMD ["/plone/bin/instance", "fg"]
  • 32. intranetumDEVELOPMENT ZOPE/PLONE - DOCKERFILE ZEOCLIENT FROM eu.gcr.io/XXXXXXXXXX/plonebase MAINTAINER Intranetum RUN rm -rf /plone/src/* # Bundle app source ADD . /plone WORKDIR /plone RUN set -x && rm -rf /plone/libsrc/* && chown www-data /plone/.mr.developer.cfg && ./bin/buildout -v -c intranetum.cfg && chown -R www-data /plone # Configure and Run ENTRYPOINT ["/plone/docker-entrypoint.sh"] CMD ["/plone/bin/instance", "fg"]
  • 33. intranetumDEVELOPMENT ZOPE/PLONE - ENTRYPOINT.SH #!/bin/bash set -e if [ -n "$ZEOSERVER_PORT_8100_TCP_ADDR" ]; then if [ -z "$ZEO_HOST" ]; then ZEO_HOST=$ZEOSERVER_PORT_8100_TCP_ADDR else echo >&2 'warning: both ZEO_HOST and ZEOSERVER_PORT_8100_TCP_ADDR found' echo >&2 " Connecting to ZEO_HOST ($ZEO_HOST)" echo >&2 ' instead of the linked zeo container' fi fi if [ -z "$ZEO_HOST" ]; then echo >&2 'error: missing ZEO_HOST' echo >&2 ' Did you forget to --link some_zeo_container:zeoserver or set an external db' echo >&2 ' with -e ZEO_HOST=hostname' exit 1 fi if [ -n "$ELASTICSEARCH_PORT_9200_TCP_ADDR" ]; then if [ -z "$ELASTIC_HOSTS" ]; then ELASTIC_HOSTS=$ELASTICSEARCH_PORT_9200_TCP_ADDR else echo >&2 'warning: both ELASTIC_HOSTS and ELASTICSEARCH_PORT_9200_TCP_ADDR found' echo >&2 " Connecting to ELASTIC_HOSTS ($ELASTIC_HOSTS)" echo >&2 ' instead of the linked elastic container' fi fi …. if [ -z "$SHAREDBLOB" ]; then echo >&2 'warning: no SHAREDBLOB setted: default off' fi set_config() { echo "SET VAR" echo $1 echo $2 key="$1" value="$2" sed -i -re 's!('"${key}"'s+)[^=]*$!1'"${value}"'!' /plone/parts/instance/etc/zope.conf } set_config_section() { echo "SET VAR" echo $1 echo $2 echo $3 section="$1" key="$2" value="$3" sed -i -re '/<'"${section}"'>/ ,/</'"${section}"'>/ s!('"${key}"'s+)[^=]*$!1'"${value}"'!' / plone/parts/instance/etc/zope.conf } echo "SET CONFIG" set_config_section zeoclient server $ZEO_HOST:$ZEO_PORT set_config_section zeoclient shared-blob-dir $SHAREDBLOB set_config elasticsearch-hosts $ELASTIC_HOSTS echo "START ZOPE" exec "$@"
  • 34. intranetumDEVELOPMENT ZOPE/PLONE - INTRANETUM.CFG [buildout] extends = buildout.cfg versions = versions always-checkout = true parts = instance eggs += XXXX auto-checkout += plone.rest collective.elasticsearch … [instance] recipe = plone.recipe.zope2instance effective-user = www-data http-address = 8080 zeo-address = $ZEO_ADDR:$ZEO_PORT blob-storage = ${buildout:directory}/var/blobstorage/beta zeo-client = on shared-blob = $SHAREDBLOB eggs = ${buildout:eggs} environment-vars = zope_i18n_compile_mo_files true zope-conf-additional = <product-config intranetum> elasticsearch-hosts $ELASTIC_HOSTS </product-config>
  • 35. intranetumDEVELOPMENT ZOPE/PLONE - DOCKERFILE ZEO FROM eu.gcr.io/XXXXXXXXXX/zeoserver MAINTAINER Intranetum ADD . /plone WORKDIR /plone RUN set -x && chown www-data /plone/.mr.developer.cfg && rm -rf /plone/libsrc/* && ./bin/buildout -v -c intranetum.cfg && chown -R www-data /plone # Expose EXPOSE 8100 VOLUME /plone/var/filestorage VOLUME /plone/var/blobstorage # Configure and Run ENTRYPOINT ["/plone/docker-entrypoint.sh"] CMD ["/plone/bin/zeo", “fg"] [buildout] parts = zeo always-checkout = true extends = buildout.cfg auto-checkout += plone.rest collective.elasticsearch … [zeo] recipe = plone.recipe.zeoserver zeo-address = 8100 effective-user = www-data eggs = ${buildout:eggs} file-storage = filestorage/beta/Data.fs blob-storage = blobstorage/beta
  • 36. intranetumDEVELOPMENT ZOPE/PLONE - STAGE YAML apiVersion: v1 kind: ReplicationController metadata: name: masterzeoclient labels: name: masterzeoclient spec: replicas: 2 selector: name: masterzeoclient template: metadata: labels: name: masterzeoclient spec: containers: - image: eu.gcr.io/XXXXXXXXXXXX/zeoclient imagePullPolicy: Always resources: limits: cpu: 0.25 name: masterzeoclient env: - name: ELASTIC_HOSTS value: "serviceelastic" - name: ZEO_HOST value: "servicezeo" … - name: SHAREDBLOB value: "off" ports: - containerPort: 8080 name: masterzeoclient apiVersion: v1 kind: ReplicationController … spec: replicas: 1 selector: name: masterzeo template: metadata: labels: name: masterzeo spec: containers: - name: masterzeo resources: limits: cpu: 0.4 image: eu.gcr.io/XXXXXXXXXXXX/zeoserver imagePullPolicy: Always ports: - containerPort: 8100 name: masterzeo volumeMounts: - mountPath: /plone/var/filestorage name: zeo-filestorage - mountPath: /plone/var/blobstorage name: zeo-blobstorage volumes: - name: zeo-filestorage gcePersistentDisk: pdName: XXXXXXXXXXXX fsType: ext4 - name: zeo-blobstorage gcePersistentDisk: pdName: zeo-blobstorage-disk fsType: ext4
  • 37. intranetumDEVELOPMENT ZOPE/PLONE - STAGE YAML apiVersion: v1 kind: Service metadata: labels: name: servicezeoclient name: servicezeoclient spec: type: LoadBalancer ports: # The port that this service should serve on. - port: 80 targetPort: 8080 protocol: TCP # Label keys and values that must match in order to receive traffic for this service. selector: name: masterzeoclient apiVersion: v1 kind: Service metadata: name: servicezeo labels: name: servicezeo spec: type: NodePort ports: - port: 8100 selector: name: masterzeo
  • 38. intranetumDEVELOPMENT ZOPE/PLONE - PRODUCTION YAML apiVersion: v1 kind: ReplicationController … spec: replicas: 3 selector: name: masterzeoclient template: metadata: labels: name: masterzeoclient spec: containers: - image: eu.gcr.io/XXXXXXXXXXX/zeoclient imagePullPolicy: Always resources: limits: cpu: 0.8 name: masterzeoclient env: - name: ELASTIC_HOSTS value: "serviceelastic" - name: ZEO_HOST value: "servicezeo" - name: SHAREDBLOB value: "on" ports: - containerPort: 8080 name: masterzeoclient volumeMounts: - mountPath: /plone/var/blobstorage name: nfs-zeo volumes: - name: nfs-zeo persistentVolumeClaim: claimName: nfs-zeo apiVersion: v1 kind: ReplicationController … spec: replicas: 1 selector: name: masterzeo template: metadata: labels: name: masterzeo spec: containers: - name: masterzeo resources: limits: cpu: 0.5 image: eu.gcr.io/XXXXXXXXXXXXXXX/zeoserver imagePullPolicy: Always ports: - containerPort: 8100 name: masterzeo volumeMounts: - mountPath: /plone/var/filestorage name: zeo-filestorage - mountPath: /plone/var/blobstorage name: nfs-zeo volumes: - name: zeo-filestorage gcePersistentDisk: pdName: zeo-filestorage-disk-prod fsType: ext4 - name: nfs-zeo persistentVolumeClaim: claimName: nfs-zeo
  • 39. intranetumDEVELOPMENT ZOPE/PLONE - PROD GCP - UPDATE kubectl rolling-update -f zeo.yaml kubectl rolling-update -f zeoclient.yaml kubectl scale --replicas=3 rc zeoclient ▸ SNAPSHOTS ▸ NFS ▸ https://goo.gl/Zmub08 ▸ HEALTH CHECKS
  • 40. intranetumDEVELOPMENT TESTING - GROUP 1 ▸ LOCAL TESTING ▸ docker-compose Elasticsearch ▸ GROUP CONFIG ENV vars to TESTING ENV ▸ COMMIT ON CODE ▸ JENKINS testing INTEGRATION on NATIVE Buildout agains TESTING ENV with SETTING JOBS ▸ JENKINS BUILD Docker Images and PUSH ▸ JENKINS DEPLOY TESTING - ACCEPTANCE TESTS ▸ JENKINS DEPLOY STAGE ▸ MANUAL TRIGGER DEPLOY PRODUCTION
  • 41. intranetumDEVELOPMENT JOBS ▸ UPGRADE STEPS ON STAGE / PRODUCTION ▸ MAINTENENCE JOBS # upgrade-b16.py def upgrade(site): site.doNothing() if __name__ == "__main__": output = upgrade(app.Plone) logger.info(output) # upgrade-b16.yaml apiVersion: batch/v1 kind: Job metadata: name: upgrade-b16 spec: template: metadata: name: upgrade-b16 spec: containers: - name: plone image: eu.gcr.io/XXXXXXXX/upgrade-plone command: [“/plone/bin/instance“, “/plone/upgrade-b16.py“] restartPolicy: Never
  • 43. intranetumDEVELOPMENT MAIN CLOSED COMPONENTS ▸ FRONT END (NODE + ANGULAR 2) ▸ FRONT END MIDDLEWARE ▸ HORUS (AUTHZ AUTHN) ▸ 3 components PY 3.5 ASYNCIO ▸ BEATS (MACHINE LEARNING ENGINE) ▸ >15 HIPOS (ASYNC components PY3.5/GO/ERLANG)
  • 44. intranetumDEVELOPMENT DOCKER FROM 0 - HIPO FROM python:3.5.1 RUN apt-get update -y && apt-get install -y netcat && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* RUN pip3.5 install aioamqp python-logstash RUN pip3.5 install https://pypi.python.org/packages/source/p/protobuf/protobuf-3.0.0b2.tar.gz COPY docker-entrypoint.sh / COPY run.py / COPY api_pb2.py / ENTRYPOINT ["/docker-entrypoint.sh"] CMD ["python3", "/run.py"]
  • 46. intranetumSO… CONCLUSIONS ▸ Split arch in groups of components to test, deploy and develop ▸ K8s much powerful than docker-compose / swarm (rolling- updates / secrets / scaling / jobs) ▸ Optimization Service on K8S vs SaaS (not everything in containers) ▸ Docker from moment 0 (easier than later) ▸ Tests in Docker, Dev outside Docker (find a way to isolate components and connect to the testing/stage cluster) ▸ Proxy NPM / Debian packages / Pypi / … CI / CD is hard
  • 47. intranetumI START TO BE HUNGRY… FUTURE ▸ Ansible on docker ? - We use buildout ▸ Jenkins jobs to rollback to version X ▸ K8S jobs to manage backups/testing data set ▸ Docker to build Jenkins to build Docker ▸ Elastic search cluster deployment ▸ Remove keys from docker ▸ Local k8s hardware (Rancher) ▸ Upgrade jobs testing Stage ▸ TESTS on k8s cluster (snapshot)
  • 49. intranetum…REALLY HUNGRY OPEN QUESTIONS ▸ Container data manager VS SaaS data manager ▸ Volumes ZFS/NFS snapshoots ▸ Performance on computation container ▸ Persistent Disks vs local SSD vs Buckets vs RAM disk ▸ Monitoring ? ▸ More questions ?