SlideShare una empresa de Scribd logo
1 de 40
Salamander:
Using Docker with NoSQL
Manuel E. de Paz
Software Architect at BEEVA (a BBVA Company)
use #SalamanderAtDockerCon during the track to Q&A,
Salamander’s Team it’s awaiting to answer you.
Some Ideas
● Salamander, an adventure in AWS.
● How does Docker helps us?
● Our next Challenges
What is Salamander?
Frontend, Backend & API Data ETL
Batch Import
native-drivers
Java
What is Salamander?
Data Processing
ETL & Data Typologies
Relationships
1 day ≅ 250.000 nodes
ETL & Data Typologies
Data
Properties
1 day ≅ 300.000 executions
Workflows and pipelines
Deploy &
Continuous
Integration
VPC EC2
DevOps Flow
Data Pipeline
Process
Java
AWS S3
Cloud Architecture
Proof of Concept
Elastic
Beanstalk
EMR
EC2
S3
MongoDB
Data’s Growth
DB on
Instance ReplicaSet ShardingReplicaSet
High
Availability
Horizontal
Scaling
The data requirements force us to grown...
+
MicroData
DB on
Instance
Vertical
Scaling
DB on
Instance
...
DATA
Concept of MicroData = Isolation + Scaling
Neo4j
Production Situation
EC2 Container
Service
Identity & Access
Management
Glacier
ElastiCache
EC2
S3
VPC
Docker Challenge #1
Docker Challenge #1
● 1 Machine & 3x Container
● AWS S3 Persistence, Virtually infinite
● Docker Compose
● Docker Registry
S3
Docker Challenge #1
EC2 instance contents
Amazon S3
bucket
cluster #2
cluster #3
Docker Challenge #2
● Cluster used for support Business Logic
● High Availability
● Fail Redundancy
● AWS EC Services: Containers as a Service
Docker Challenge #2
+ EC2 Container
Service
Cluster #2
EC2 Container
Service
EC2 instance contents
DATA
Elastic Load
Balancing
EC2 instance contents
DATA
EC2 instance
web
app
server
AWS ECS Services
Docker Challenge #2
Docker Challenge #3
● Security & Stability Vs Flexibility
● Neo4j WebGUI Browser
● Stress Isolation
● AWS EC2 Container: Container as a Service
Docker Challenge #3
EC2 Container
Service
Docker Challenge #3
Cluster #3
EC2 Container
Service
EC2 instance contents
EC2 instance
web
app
server
EC2 instance
REST
server
DATA
???????????? Frontend:
Web Proxy
Backend:
Reverse Proxy
user
Next Challenges
Docker Challenge #4
AWS ECS Container: tasks
DB as a Service
Containers
Auto organized proxy nodes
Seaport: Cluster Discover & Management
Node.js
EC2 Container
Service
New Challenges
Private
Registry
Amazon S3
bucket
cluster
EC2 instance
web
app
server
EC2 instance
REST
server
Frontend:
WebApp
Backend:
REST Server
user
EC2 instance
REST
server
Orchestrator:
Proxy & Mgmt
DATA
DATA
DATA
...
DATA
Node.js seaport
Next Challenges
Reduce the size of containers
AWS Private Registry Service
Replace RunDeck for AWS Lambda
Lambda
Conclusions
Conclusions
Docker help us to avoid a high complex database
infrastructure
Data Segmentation
Isolation
Fine Tuning
Performance
Conclusions
AWS EC2 Container Services API help us from a
more complex architecture.
High Availability
Fail Redundancy
Management
Service Discovery
Reducing Costs
Requirements
Some criteria that help us:
Immutable Data
Segmentable Info
NoSQL with Restful Driver
© Albert Uderzo
DEMO
Link to video
Our Team
Name Technology Version
REDIS Database NoSQL N/A / 2:3.0.5-3
Neo4j Database NoSQL 2.1.6 / 2.2
MongoDB Database NoSQL 2.6.11
Docker Registry Service 1.6
NodeJS App Development v4.0.0 / v0.12.2 / v0.10.29
AWS SDK App Development 2.0.31
Express App Development 4.0.0
AngularJS App Development 1.4.7
References & Versions
Image @ slide Link
7 https://www.flickr.com/photos/niznoz/3116766661
10 https://www.flickr.com/photos/fordeu/5709826282
13 https://unsplash.com/photos/ni9mKm62QnA
18 http://mrg.bz/xBONJ8
21 https://www.flickr.com/photos/cote/14692025435/
24 https://flic.kr/p/imxgqh
27 https://unsplash.com/photos/ZXVk-NMWtgg
31 https://unsplash.com/photos/_r19nfvS3wY
Image Credits & Attributions
All product names, logos, brands, and other trademarks featured or referred
in this presentation are the property of their respective trademark holders.
www.beeva.com
hablemos@beeva.com
Feel free to contact us:
manuel.depaz@beeva.com
salamandra@beeva.com
hablemos@beeva.com
Thank you!
Manuel E. de Paz
@manueldepaz
manuel.depaz@beeva.com

Más contenido relacionado

Destacado

Dockerfile Basics Workshop #1
Dockerfile Basics Workshop #1Dockerfile Basics Workshop #1
Dockerfile Basics Workshop #1
Docker, Inc.
 
DockerCon 14 Keynote Day 2
DockerCon 14 Keynote Day 2DockerCon 14 Keynote Day 2
DockerCon 14 Keynote Day 2
Docker, Inc.
 
DockerCon Recap - Online Meetup by Ben Firshman
DockerCon Recap - Online Meetup by Ben FirshmanDockerCon Recap - Online Meetup by Ben Firshman
DockerCon Recap - Online Meetup by Ben Firshman
Docker, Inc.
 
DockerCon14 John Engates
DockerCon14 John EngatesDockerCon14 John Engates
DockerCon14 John Engates
Docker, Inc.
 
Dockerizing Stashboard
Dockerizing StashboardDockerizing Stashboard
Dockerizing Stashboard
Docker, Inc.
 
Trust and Image Provenance by Derek McGowan
Trust and Image Provenance by Derek McGowanTrust and Image Provenance by Derek McGowan
Trust and Image Provenance by Derek McGowan
Docker, Inc.
 
Building Images from dockerfiles
Building Images from dockerfilesBuilding Images from dockerfiles
Building Images from dockerfiles
Docker, Inc.
 
DockerCon14 Automated Chef Cookbook Testing
DockerCon14 Automated Chef Cookbook TestingDockerCon14 Automated Chef Cookbook Testing
DockerCon14 Automated Chef Cookbook Testing
Docker, Inc.
 
Deploying Containers and Managing Them
Deploying Containers and Managing ThemDeploying Containers and Managing Them
Deploying Containers and Managing Them
Docker, Inc.
 
Docker at DevTable
Docker at DevTableDocker at DevTable
Docker at DevTable
Docker, Inc.
 

Destacado (19)

DockerCon SF 2015: Docker After Launching 1 Billion Containers
DockerCon SF 2015: Docker After Launching 1 Billion ContainersDockerCon SF 2015: Docker After Launching 1 Billion Containers
DockerCon SF 2015: Docker After Launching 1 Billion Containers
 
Dockerfile Basics Workshop #1
Dockerfile Basics Workshop #1Dockerfile Basics Workshop #1
Dockerfile Basics Workshop #1
 
DockerCon 14 Keynote Day 2
DockerCon 14 Keynote Day 2DockerCon 14 Keynote Day 2
DockerCon 14 Keynote Day 2
 
DockerCon Recap - Online Meetup by Ben Firshman
DockerCon Recap - Online Meetup by Ben FirshmanDockerCon Recap - Online Meetup by Ben Firshman
DockerCon Recap - Online Meetup by Ben Firshman
 
DockerCon14 John Engates
DockerCon14 John EngatesDockerCon14 John Engates
DockerCon14 John Engates
 
DockerCon14 eBay
DockerCon14 eBayDockerCon14 eBay
DockerCon14 eBay
 
Dockerizing Stashboard
Dockerizing StashboardDockerizing Stashboard
Dockerizing Stashboard
 
DockerCon SF 2015: From Months to Minutes
DockerCon SF 2015: From Months to MinutesDockerCon SF 2015: From Months to Minutes
DockerCon SF 2015: From Months to Minutes
 
Understanding Containers through Gaming by Brendan Fosberry
Understanding Containers through Gaming by Brendan Fosberry Understanding Containers through Gaming by Brendan Fosberry
Understanding Containers through Gaming by Brendan Fosberry
 
Trust and Image Provenance by Derek McGowan
Trust and Image Provenance by Derek McGowanTrust and Image Provenance by Derek McGowan
Trust and Image Provenance by Derek McGowan
 
DockerCon 2015: Docker Engine Breakout Session
DockerCon 2015: Docker Engine Breakout SessionDockerCon 2015: Docker Engine Breakout Session
DockerCon 2015: Docker Engine Breakout Session
 
Building Images from dockerfiles
Building Images from dockerfilesBuilding Images from dockerfiles
Building Images from dockerfiles
 
DockerCon14 Automated Chef Cookbook Testing
DockerCon14 Automated Chef Cookbook TestingDockerCon14 Automated Chef Cookbook Testing
DockerCon14 Automated Chef Cookbook Testing
 
Deploying Containers and Managing Them
Deploying Containers and Managing ThemDeploying Containers and Managing Them
Deploying Containers and Managing Them
 
Docker at DevTable
Docker at DevTableDocker at DevTable
Docker at DevTable
 
Containerize All the (Multi-Platform) Things! by Phil Estes
Containerize All the (Multi-Platform) Things! by Phil EstesContainerize All the (Multi-Platform) Things! by Phil Estes
Containerize All the (Multi-Platform) Things! by Phil Estes
 
DockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
DockerCon SF 2015: Beyond CI to Production Scale PaaS with DockerDockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
DockerCon SF 2015: Beyond CI to Production Scale PaaS with Docker
 
Docker 1.11 @ Docker SF Meetup
Docker 1.11 @ Docker SF MeetupDocker 1.11 @ Docker SF Meetup
Docker 1.11 @ Docker SF Meetup
 
The Future of Hardware-Backed Keys
The Future of Hardware-Backed KeysThe Future of Hardware-Backed Keys
The Future of Hardware-Backed Keys
 

Más de Docker, Inc.

Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
Docker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
Docker, Inc.
 

Más de Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
 
Hands-on Helm
Hands-on Helm Hands-on Helm
Hands-on Helm
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices World
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with Docker
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog Scale
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm Architecture
 

Último

Último (20)

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
 
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live StreamsTop 5 Benefits OF Using Muvi Live Paywall For Live Streams
Top 5 Benefits OF Using Muvi Live Paywall For Live Streams
 

DockerCon EU 2015: Using Docker with NoSQL