Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Cargando en…3
×
1 de 54

Powering Microservices with Docker, Kubernetes, Kafka, and MongoDB

8

Compartir

Speaker: Andrew Morgan

Organizations are building their applications around microservice architectures because of the flexibility, speed of delivery, and maintainability they deliver. Want to try out MongoDB on your laptop? Execute a single command and you have a lightweight, self-contained sandbox; another command removes all trace when you're done. Replicate your complete application for your development, test, operations, and support teams. This session introduces you to technologies such as Docker, Kubernetes, and Kafka, which are driving the microservices revolution. Learn about containers and orchestration, and most importantly, how to exploit them for stateful services such as MongoDB.

Powering Microservices with Docker, Kubernetes, Kafka, and MongoDB

  1. 1. # M D B l o c a l Andrew Morgan Powering Microservices with Docker, Kubernetes, Kafka, & MongoDB MongoDB London @andrewmorgan
  2. 2. # M D B l o c a l Agenda Microservic es What, Why, When MongoDB Why Containers Docker Messaging Kafka Orchestrati on Kubernetes Deploying MongoDB Options, How 1 2 3 4 5 6
  3. 3. # M D B l o c a l Microservices
  4. 4. # M D B l o c a l Why Use Microservices? (tl;dr WebScale) Speed Change Maintain Scale Empower Build MVP quickly Rapid iterations Simple components Product Team == Component React to market Isolated impact Team Committees
  5. 5. # M D B l o c a l Why Use Microservices? (tl;dr WebScale) Speed Change Maintain Scale Empower Build MVP quickly Rapid iterations Simple components Product Team == Component React to market Isolated impact Team Committees
  6. 6. # M D B l o c a l Why Use Microservices? (tl;dr WebScale) Speed Change Maintain Scale Empower Build MVP quickly Rapid iterations Simple components Product Team == Component React to market Isolated impact Team Committees
  7. 7. # M D B l o c a l Why Use Microservices? (tl;dr WebScale) Speed Change Maintain Scale Empower Build MVP quickly Rapid iterations Simple components Product Team == Component React to market Isolated impact Team Committees
  8. 8. # M D B l o c a l Why Use Microservices? (tl;dr WebScale) Speed Change Maintain Scale Empower Build MVP quickly Rapid iterations Simple components Product Team == Component React to market Isolated impact Team Committees
  9. 9. # M D B l o c a l Why Use Microservices? (tl;dr WebScale) Speed Change Maintain Scale Empower Build MVP quickly Rapid iterations Simple components Product Team == Component React to market Isolated impact Team Committees
  10. 10. Microservices Decoupled Independent Dev Isolated Impact
  11. 11. # M D B l o c a l Microservices Example Twitter IngestGoogle+ Ingest Snapchat Ingest Feed merge Facebook Ingest
  12. 12. # M D B l o c a l Microservices Example Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest
  13. 13. # M D B l o c a l Microservices Example Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest
  14. 14. # M D B l o c a l Microservices Example Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest Whatsapp Ingest
  15. 15. # M D B l o c a l Microservices Example Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest Whatsapp Ingest Snapchat Ingest Snapchat Ingest
  16. 16. # M D B l o c a l When to use Microservices
  17. 17. # M D B l o c a l When to use Microservices
  18. 18. # M D B l o c a l Why MongoDB & Microservices
  19. 19. # M D B l o c a l Why MongoDB is a Good Fit For Microservices Monitoring & Automation Flexible Data Model Redundancy Scalability Simplicity
  20. 20. # M D B l o c a l Why MongoDB is a Good Fit For Microservices Monitoring & Automation Flexible Data Model Redundancy Scalability Simplicity
  21. 21. # M D B l o c a l Why MongoDB is a Good Fit For Microservices Monitoring & Automation Flexible Data Model Redundancy Scalability Simplicity
  22. 22. # M D B l o c a l Why MongoDB is a Good Fit For Microservices Monitoring & Automation Flexible Data Model Redundancy Scalability Simplicity
  23. 23. # M D B l o c a l Why MongoDB is a Good Fit For Microservices Monitoring & Automation Flexible Data Model Redundancy Scalability Simplicity
  24. 24. # M D B l o c a l Why MongoDB is a Good Fit For Microservices Monitoring & Automation Flexible Data Model Redundancy Scalability Simplicity
  25. 25. # M D B l o c a l Containers
  26. 26. # M D B l o c a l Software containers • 1 image -> Many containers • Laptop, DC, cloud • Dev, QA, production, support • Efficient • Isolation • Constraints Containers – Powering Microservices
  27. 27. # M D B l o c a l • Simple to use • 100K+ images on Docker Hub • Build images from images • Platforms • Linux, OS X, Windows • Laptop, VM, Cloud,… • Cloud services Docker
  28. 28. # M D B l o c a l docker run -d mongo Run MongoDB
  29. 29. # M D B l o c a l FROM debian:jessie-slim RUN groupadd -r mongodb && useradd -r -g mongodb mongodb ... ARG MONGO_PACKAGE=mongodb-org ENV MONGO_VERSION 3.4.9 ... RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list” RUN set -x && apt-get update && apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION ${MONGO_PACKAGE}-server=$MONGO_VERSION ${MONGO_PACKAGE}-shell=$MONGO_VERSION ${MONGO_PACKAGE}-mongos=$MONGO_VERSION ${MONGO_PACKAGE}-tools=$MONGO_VERSION && rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/mongodb && mv /etc/mongod.conf /etc/mongod.conf.orig RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db /data/configdb VOLUME /data/db /data/configdb COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 27017 CMD ["mongod"] https://github.com/docker-library/mongo
  30. 30. # M D B l o c a l FROM debian:jessie-slim RUN groupadd -r mongodb && useradd -r -g mongodb mongodb ... ARG MONGO_PACKAGE=mongodb-org ENV MONGO_VERSION 3.4.9 ... RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list” RUN set -x && apt-get update && apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION ${MONGO_PACKAGE}-server=$MONGO_VERSION ${MONGO_PACKAGE}-shell=$MONGO_VERSION ${MONGO_PACKAGE}-mongos=$MONGO_VERSION ${MONGO_PACKAGE}-tools=$MONGO_VERSION && rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/mongodb && mv /etc/mongod.conf /etc/mongod.conf.orig RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db /data/configdb VOLUME /data/db /data/configdb COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 27017 CMD ["mongod"] https://github.com/docker-library/mongo
  31. 31. # M D B l o c a l FROM debian:jessie-slim RUN groupadd -r mongodb && useradd -r -g mongodb mongodb ... ARG MONGO_PACKAGE=mongodb-org ENV MONGO_VERSION 3.4.9 ... RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list” RUN set -x && apt-get update && apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION ${MONGO_PACKAGE}-server=$MONGO_VERSION ${MONGO_PACKAGE}-shell=$MONGO_VERSION ${MONGO_PACKAGE}-mongos=$MONGO_VERSION ${MONGO_PACKAGE}-tools=$MONGO_VERSION && rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/mongodb && mv /etc/mongod.conf /etc/mongod.conf.orig RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db /data/configdb VOLUME /data/db /data/configdb COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 27017 CMD ["mongod"] https://github.com/docker-library/mongo
  32. 32. # M D B l o c a l FROM debian:jessie-slim RUN groupadd -r mongodb && useradd -r -g mongodb mongodb ... ARG MONGO_PACKAGE=mongodb-org ENV MONGO_VERSION 3.4.9 ... RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list” RUN set -x && apt-get update && apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION ${MONGO_PACKAGE}-server=$MONGO_VERSION ${MONGO_PACKAGE}-shell=$MONGO_VERSION ${MONGO_PACKAGE}-mongos=$MONGO_VERSION ${MONGO_PACKAGE}-tools=$MONGO_VERSION && rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/mongodb && mv /etc/mongod.conf /etc/mongod.conf.orig RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db /data/configdb VOLUME /data/db /data/configdb COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 27017 CMD ["mongod"] https://github.com/docker-library/mongo
  33. 33. # M D B l o c a l FROM debian:jessie-slim RUN groupadd -r mongodb && useradd -r -g mongodb mongodb ... ARG MONGO_PACKAGE=mongodb-org ENV MONGO_VERSION 3.4.9 ... RUN echo "deb http://$MONGO_REPO/apt/debian jessie/${MONGO_PACKAGE%-unstable}/$MONGO_MAJOR main" | tee "/etc/apt/sources.list.d/${MONGO_PACKAGE%-unstable}.list” RUN set -x && apt-get update && apt-get install -y ${MONGO_PACKAGE}=$MONGO_VERSION ${MONGO_PACKAGE}-server=$MONGO_VERSION ${MONGO_PACKAGE}-shell=$MONGO_VERSION ${MONGO_PACKAGE}-mongos=$MONGO_VERSION ${MONGO_PACKAGE}-tools=$MONGO_VERSION && rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/mongodb && mv /etc/mongod.conf /etc/mongod.conf.orig RUN mkdir -p /data/db /data/configdb && chown -R mongodb:mongodb /data/db /data/configdb VOLUME /data/db /data/configdb COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["docker-entrypoint.sh"] EXPOSE 27017 CMD ["mongod"] https://github.com/docker-library/mongo
  34. 34. # M D B l o c a l Many small, focused containers -> sophisticated services • Well defined APIs • Independent languages & libraries • Modular: easy maintenance + reuse • Fault tolerant • Scalable Microservice Architectures Built on Containers
  35. 35. # M D B l o c a l Apache Kafka
  36. 36. # M D B l o c a l Connecting the Microservices – Kafka Producer 987 123... Topic A Consumer New  Old
  37. 37. # M D B l o c a l Connecting the Microservices – Kafka Producer 987 123... Topic A Consumer Producer Consumer
  38. 38. # M D B l o c a l Connecting the Microservices – Kafka Producer 987 123... Partition 0 Topic A Consumer Producer Consumer 435 123... Partition 1
  39. 39. # M D B l o c a l Connecting the Microservices – Kafka Producer LEADER Topic A / Partition 0 Broker 1 FOLLOWER Topic A / Partition 1 FOLLOWER Topic A / Partition 0 Broker 2 LEADER Topic A / Partition 1
  40. 40. # M D B l o c a l Connecting the Microservices – Kafka Producer Producer Producer 9 8 7 123 ... Partition 0 4 3 5 123 ... Partition 1 7 3 2 123 ... Partition N Topic A Topic B 7 6 5 123 ... Partition 0 Consumer Consumer
  41. 41. # M D B l o c a l Orchestration – Kubernetes
  42. 42. # M D B l o c a l Created by Google, feature-rich and widely adopted • Deployment and ‘replication’ • On-line scale out/in • Rolling upgrades • High Availability • Persistence • Ports • Load balancing • Google Compute Engine Kubernetes
  43. 43. # M D B l o c a l MongoDB & Kubernetes
  44. 44. # M D B l o c a l How to use MongoDB with Containers Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest Whatsapp Ingest Snapchat Ingest Snapchat Ingest MongoDB Atlas
  45. 45. # M D B l o c a l How to use MongoDB with Containers Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest Whatsapp Ingest Snapchat Ingest Snapchat Ingest Kubernetes Ops Mgr agent Ops Mgr agent Ops Mgr agent
  46. 46. # M D B l o c a l How to use MongoDB with Containers Twitter Ingest Snapchat Ingest Feed merge Facebook Ingest Whatsapp Ingest Snapchat Ingest Snapchat Ingest Kubernetes mongod mongod mongod
  47. 47. # M D B l o c a l Kubernetes Building Blocks POD Host Host Service
  48. 48. Volume name: mongo-persistent-storage1 pdName: mongodb-disk1 mongodb -disk1 Container name: mongo-node1 image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage1 mountPath: /data/db Docker Hub Registry Pod labels: [name: mongo-node1; instance: rod] ReplicationController name: mongo-rc1 labels: [name: mongo-rc] replicas: 1 selector: [name: mongo -node1] Service: LoadBalancer name: mongo-svc-a labels: [name: mongo-svc-a] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node, instance: rod] ExternalIP Address 104.1.1.1 App 104.1.1.1:27017
  49. 49. # M D B l o c a l ReplicationControler name: mongo-rc-europe labels: [name: mongo-europe] replicas: 1 selector: [name: mongo -node] Volume name: mongo-persistent-storage pdName: mongodb-disk-europe mongodb- disk-europe Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-europe labels: [name: mongo-svc-europe] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] ReplicationControler name: mongo-rc-asia labels: [name: mongo-asia] replicas: 1 selector: [name: mongo -node] Volume name: mongo-persistent-storage pdName: mongodb-disk-asia mongodb- disk-asia Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-asia labels: [name: mongo-svc-asia] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] ReplicationControler name: mongo-rc-us labels: [name: mongo-us] replicas: 1 selector: [name: mongo -node] Volume name: mongo-persistent-storage pdName: mongodb-disk-us mongodb- disk-us Container name: mongo-node image: mongo command: mongod –replSet my_replica_set containerPort: 27017 volumeMounts: name: mongo-persistent-storage mountPath: /data/db Pod labels: [name: mongo-node] Service: LoadBalancer name: mongo-svc-us labels: [name: mongo-svc-us] ports: [port: 27017, targetPort: 27017]] selector: [name: mongo-node] Loosely Coupled Replica Set
  50. 50. # M D B l o c a l Beta in Kubernetes 1.5+ • Stable, predictable, unique network identifiers • IP addresses may change • Stable, persistent storage • Ordered, graceful deployment and scaling (0 N-1) • Ordered, graceful deletion and termination (N-1  0) StatefulSets
  51. 51. # M D B l o c a l MongoDB Replica Set as StatefulSet rs.initiate() rs.add('mongo-1.mongo:27017') rs.add('mongo-2.mongo:27017')
  52. 52. # M D B l o c a l • Enabling Microservices: Containers & Orchestration Explained https://www.mongodb.com/collateral/microservices-containers-and-orchestration-explained • Microservices: The Evolution of Building Modern Applications https://www.mongodb.com/collateral/microservices-the-evolution-of-building-modern-applications • Data Streaming with Apache Kafka & MongoDB https://www.mongodb.com/collateral/data-streaming-with-apache-kafka-and-mongodb • Guidance and examples http://k8smongodb.net/ References
  53. 53. # M D B l o c a l MongoDB & Microservices in the Wild
  54. 54. # M D B l o c a l

Notas

  • 1 min
  • 2 mins
    A microservice architecture is one where you break an application down into multiple processes; where each one:
    Is self-contained
    Has a single, well defined role

    The microservices communicate with each other using network technologies (even if running on the same host)
  • 3 min.
    Web & Mobile Apps => Enterprise
    Micro Web Services



    Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
    Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
    Benefits of microservices:
    - Development Speed
    - Rapid Iteration
    Evolve quickly, continuous deployment
    Isolate impact of changes to existing functions or just add a new one
    Reactive development
    Maintainable
    Independent, empowered work teams
  • Web & Mobile Apps => Enterprise
    Micro Web Services



    Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
    Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
    Benefits of microservices:
    - Development Speed
    - Rapid Iteration
    Evolve quickly, continuous deployment
    Isolate impact of changes to existing functions or just add a new one
    Reactive development
    Maintainable
    Independent, empowered work teams
  • Web & Mobile Apps => Enterprise
    Micro Web Services



    Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
    Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
    Benefits of microservices:
    - Development Speed
    - Rapid Iteration
    Evolve quickly, continuous deployment
    Isolate impact of changes to existing functions or just add a new one
    Reactive development
    Maintainable
    Independent, empowered work teams
  • Web & Mobile Apps => Enterprise
    Micro Web Services



    Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
    Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
    Benefits of microservices:
    - Development Speed
    - Rapid Iteration
    Evolve quickly, continuous deployment
    Isolate impact of changes to existing functions or just add a new one
    Reactive development
    Maintainable
    Independent, empowered work teams
  • Web & Mobile Apps => Enterprise
    Micro Web Services



    Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
    Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
    Benefits of microservices:
    - Development Speed
    - Rapid Iteration
    Evolve quickly, continuous deployment
    Isolate impact of changes to existing functions or just add a new one
    Reactive development
    Maintainable
    Independent, empowered work teams
  • Web & Mobile Apps => Enterprise
    Micro Web Services



    Microservices were pioneered in the web and then mobile App worlds; at one time called micro-web-services. Now other enterprises are looking for the same benefits.
    Microservice architectures implement applications as a series of small, self-contained, loosely coupled software components. Each has a specific and well understood role.
    Benefits of microservices:
    - Development Speed
    - Rapid Iteration
    Evolve quickly, continuous deployment
    Isolate impact of changes to existing functions or just add a new one
    Reactive development
    Maintainable
    Independent, empowered work teams
  • 5 mins.
    Add a new flavor independently.
    Chef making chocolate knows that well but need not know the others
    Blueberry goes out of fashion, remove them
    Need more green cakes, add them
    Improved pink frosting, throw out the old ones and the new ones.

    Microservices are like Cupcakes
    Can add new ones with different flavors, remove ones that you no longer need, add more pink ones if there’s greater demand
    Developers can create and activate new microservices without prior coordination with others. Their adherence to MSA principles makes continuous delivery of new or modified services possible
    Greater modularity, looser coupling.
    Started in the web and mobile app world, moving to Enterprise. Big in media and startups
    Plan for flexibility rather than reuse
  • 7 mins.
    Each of the ovals represents a microservice.
    Each source of social media feeds has its own microservice which is specialised in interfacing with the relevant API.
    Each of those microservices passes messages to the ‘feed merge’ microservice which can then make them available for further microservices to work with.
    Communication between the microservices is over the network – they can be local to the same machine or distributed.
    Best practice is for each microservice to be stateless and to have its own database or schema
  • Individual microservices can be updated in isolation or even removed if their role is no longer needed
  • When a new role (or even a change to an existing one) appears, best practice is to implement a new microservice rather than extending an existing one.
  • When a new role (or even a change to an existing one) appears, best practice is to implement a new microservice rather than extending an existing one.
  • Microservices allow scale-out.
    Each type of microservice can be scaled independently – add extra instances just for the functions that are being overworked.
    Multiple instances of each service can provide High Availability
  • 9 mins
  • Fast > Elegant
    Sagrada Familia –1882-2026 (144 years).
    Frequent, localised changes
    Localised scaling
    Upgrades

    Only if:
    Scaling team
    Designing for change



    Fast is more important than elegant.
    Change in the application’s functionality and usage is frequent.
    Change occurs at different rates within the application, so functional isolation and simple integration are more important than module cohesiveness.
    Functionality is easily separated into simple, isolatable components.
    When you have the developer/DevOps skillsets.
    Where development org boundaries match service boundaries.
    Don’t forget that you’re building a distributed system -> complexity but there are precedents to read up on.
    One argument is that you shouldn’t bother with microservices unless you need either:
    - Scale your team
    - Design for change

    Sagrada Familia – designed by Gaudi; construction started on March 19, 1882. Expected to be finished in 2026.
  • 11 mins
  • 13 mins
  • Software containers
    Build an image containing the full application stack only once
    Spin up many containers from the same image in multiple environments
    Laptop, data center, cloud
    Development, QA, production, support
    Simple to use & efficient
    Contents of each container isolated from the others
    Storage, memory, namespace
    Constrain resources available to each container
    Storage, memory, CPU, IO
  • 16 mins
    The most popular container technology

    Simple to use and has a rich ecosystem
    100,000+ images available from Docker Hub
    Including mongo hub.docker.com/_/mongo/
    Syncs with GitHub projects
    Define new images built upon base images
    Define interfaces between containers
    LINUX, (and now) Windows, and OS X
    Runs on bare metal, VMs, and cloud. Cloud providers supply the Docker infrastructure (e.g. Google Container Engine)
  • Not best practice

    The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg <varname>=<value> flag.
    The ENV instruction sets the environment variable <key> to the value <value>.The environment variables set using ENV will persist when a container is run from the resulting image.
  • 20 mins
    Microservices built by combining multiple containers

    Build sophisticated services from many small, focused processes (containers)
    Well defined APIs between components
    Each component can use different libraries, middleware & programming languages
    Modular, decoupled architecture simplifies maintenance and enables reuse
    Fault tolerant
    Scalable
  • 21 mins
  • 21 mins
    3,6 Change Streams


    To do useful work, microservices need a way of communicating – Apache Kafka

    Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
    Examples of **events** (or **messages**) include:
    A periodic sensor reading such as the current temperature
    A user adding an item to the shopping cart in an online store
    A Tweet being sent with a specific hashtag
    A log entry generated for each click in a web application

    Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.

    Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
    A microservice can be upgraded and then catch up on everything it missed
    Or even apply its updated business logic to the full history of events
    A new microservice can be added and it can be brought up to speed with everything that’s gone before
    If one service is generating more work than another can keep up with then Kafka operates as a buffer
  • To do useful work, microservices need a way of communicating – Apache Kafka

    Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
    Examples of **events** (or **messages**) include:
    A periodic sensor reading such as the current temperature
    A user adding an item to the shopping cart in an online store
    A Tweet being sent with a specific hashtag
    A log entry generated for each click in a web application

    Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.

    Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
    A microservice can be upgraded and then catch up on everything it missed
    Or even apply its updated business logic to the full history of events
    A new microservice can be added and it can be brought up to speed with everything that’s gone before
    If one service is generating more work than another can keep up with then Kafka operates as a buffer
  • To do useful work, microservices need a way of communicating – Apache Kafka

    Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
    Examples of **events** (or **messages**) include:
    A periodic sensor reading such as the current temperature
    A user adding an item to the shopping cart in an online store
    A Tweet being sent with a specific hashtag
    A log entry generated for each click in a web application

    Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.

    Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
    A microservice can be upgraded and then catch up on everything it missed
    Or even apply its updated business logic to the full history of events
    A new microservice can be added and it can be brought up to speed with everything that’s gone before
    If one service is generating more work than another can keep up with then Kafka operates as a buffer
  • To do useful work, microservices need a way of communicating – Apache Kafka

    Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
    Examples of **events** (or **messages**) include:
    A periodic sensor reading such as the current temperature
    A user adding an item to the shopping cart in an online store
    A Tweet being sent with a specific hashtag
    A log entry generated for each click in a web application

    Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.

    Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
    A microservice can be upgraded and then catch up on everything it missed
    Or even apply its updated business logic to the full history of events
    A new microservice can be added and it can be brought up to speed with everything that’s gone before
    If one service is generating more work than another can keep up with then Kafka operates as a buffer
  • To do useful work, microservices need a way of communicating – Apache Kafka

    Kafka provides a flexible, scalable, and reliable method to distribute streams of event data from one or more **producers** to one or more **consumers**.
    Examples of **events** (or **messages**) include:
    A periodic sensor reading such as the current temperature
    A user adding an item to the shopping cart in an online store
    A Tweet being sent with a specific hashtag
    A log entry generated for each click in a web application

    Streams of Kafka events are organized into **topics**. A producer chooses a topic to send a given event to and consumers select which topics they pull events from. For example, a financial application could pull NYSE stock trades from one topic, and company financial announcements from another in order to look for trading opportunities.

    Kafka actually stores all of the messages that it passes around – this makes it ideal for production microservice deployments
    A microservice can be upgraded and then catch up on everything it missed
    Or even apply its updated business logic to the full history of events
    A new microservice can be added and it can be brought up to speed with everything that’s gone before
    If one service is generating more work than another can keep up with then Kafka operates as a buffer
  • 23 mins
  • 23 mins

    Cloud Native Computing Foundation

    Created by Google, feature-rich and widely adopted

    Automated container deployment and ‘replication’
    On-line scale out/in
    Rolling upgrades
    HA – automatic rescheduling of failed containers
    Exposure of network ports to external apps
    Load balancing over groups of containers providing a service
    Provided as a service by Google Compute Engine
  • 25.5 mins
  • 27.5 mins
  • 29.5 mins
    Kubernetes.
    Single Pod/comtainer/mongod in a ReplicationController.
    Use external IP addresses (other IP addresses & hostnames are local to Kubernetes cluster and they change)

    Refer to white paper for details
  • 31 mins
  • 31.5 mins
  • 33.5 mins
  • 35.5 mins
  • 36.5 mins
    Gap (flexibility): Monolith -> microservice (75 days). New types of PO took just days
    FuboTV (scalability). Single cluster for dev, QA + production. Cope with 100x bursts. Run MongoDB on Kubernetes
    Otto (arch == org). Fast, iterative delivery
    Backcountry (> distributed dev team): Schema changes were taking 20% of dev time. Flexible schema
    Compare The Market (Use Docker, Kafka, MongoDB & Ops Manager).

    GAP moved their purchase order system from a monolith architecture to microservices. Due to MongoDB’s flexible schema, it took just 75 days to build the new system. When requirements changed and they had to add new types of purchase orders, it took days instead of months.
    FuboTV is a North American soccer streaming service. Using Microservices with Kubernetes, Docker & MongoDB. Isolation means that they can use a single cluster of machines (in Google Cloud) for dev, QA & production. Very birsty application – scalability lets them handle 100x increases in traffic.
    Otto – the key was to have an architecture that fits with their organization. Microservices empower loosely couple development teams (business, project management, IT). This is all enabling Fast test & deployment + Iterative, Continuous Delivery
    Backcountry.com is an online specialty retailer that sells outdoor clothing and gear. The driver to Microservices for them was a growing, distributed development team. As more and more developers joined and made contributions to the code, the schemas became convoluted and harder to maintain; contributing to 20% of the Scrum backlog. Taking advantage of MongoDB’s flexible data model, Backcountry was able to iterate faster, reduce development time, and mitigate technical debt.
    Compare The Market: In the cloud, each microservice, or logical grouping of related microservices, is provisioned with its own MongoDB replica set running in Encrypted storage engine to further reduce our security-related surface area. Use Docker, Kafka & MongoDB.


  • ×