As we shift from monolithic software development practices to microservices, our well-designed CD pipeline will need to change. Microservices are small functions, deployed independently and linked via APIs at run-time. While these differences seem minor, they actually have a large impact on your overall CD structure. Think hundreds of workflows, small of any builds and the loss of a monolithic 'application.'
Join Tracy Ragan, CEO of DeployHub and Brendan O'Leary, Developer Evangelist at GitLab, to learn more.
It's never too early to start the conversation.
The Reality of Managing Microservices in Your CD Pipeline
1. The Reality of Managing
Microservices in Your CD Pipeline
2. Speakers – Experts in all things CD
Tracy Ragan, CEO & Co-founder
Microservices and Pipeline Evangelist,
Expert in microservice management.
Board Member CD Foundation, DevOps
Institute Ambassador.
Brendan O’Leary, DevOps Evangelist, GitLab
CI/CD and Jira expert. Passion for software
development and iterating on processes. Highly
focused on the end user’s experience.
3. Microservices are Taking Over
86% of development professionals internationally expect microservices to
become the default application architecture within the next five years.
Martin Henley, SD Times, 9/19
Its not a question of “if.” Its a question of “when.”
4. Key Takeaways:
q Build - Application configuration at
compile/link goes away.
q Deploy and Test - Microservices are
independently deployed.
q Orchestrate - The number of CD
Pipelines explode.
q Configuration Management – Tracking
and versioning microservices and their
usage will be critical and a new step in
the CD Pipeline.
The Way We Put Software Together is Broken
"What happened is we would release an update to a library, and
then one service would use the new service and now all of a
sudden all these other services were using an older version and
we had to try to keep track of which service was using what
version of the library."
Alexandra Noonan, Software Engineer, Segment
SDTimes April 2019 "Microservices: More isn't always better”
Microservice development “falls apart without coherent, disciplined
management”
Randy Heffner, VP and Research Analyst, Forrester
“3 Big Mistakes When It Comes to APIs and Microservices,” Akana Webinar
A massive disruption in how we write and deliver software
5. Container Orchestration:
An open source platform
from Google for
orchestrating containers
across clusters of servers
providing auto scaling and
fault tolerance.
Containers and Docker:
A container is a standard
unit of software that
packages up code and all
dependencies so the
application runs quickly
and reliably.
Microservices:
An architectural style that structures an
application as a collection of loosely
coupled services.
In a microservices architecture,
services are fine-grained and
independently deployable.
Let’s Talk Modern Architecture
6. Think small snippets of code.
• A service should implement a small set of strongly related functions;
• Services that change together should be packaged together;
• The Service can be changed without affecting clients;
• And, each team that owns one or more services must be autonomous. A team must
be able to develop and deploy their services with minimal collaboration with other
teams.
Chris Richardson of CloudFoundry
More on Microservices
7. • A service should implement a small set of strongly related functions;
• Services that change together should be packaged together;
• The Service can be changed without affecting clients;
Where are you in your microservice
implementation journey?
Poll Question 1
8. Increase Operational Efficiencies
One consistent view and collaboration space for Dev, Ops, and Security teams
Public cloud independent, deploy anywhere, SaaS and/or self-managed
Deliver Better Products Faster
Ability to work in parallel, get feedback and not have to wait on other teams
Automate testing, security, deployments to minimize manual intervention
Assess and resolve security, compliance, and code quality issues at the point of code change
Reduce Security and Compliance Risk
Decrease security exposure, cleaner and easier audits, reduce disruptions
What’s needed for digital transformation?
9. Today’s software delivery landscape
Toolchain tax - multiple tools, fragile integrations, hard to scale, outages
10. With Microservices your Landscape Changes
Shifting to a modern architecture will
disrupt our traditional CI/CD pipeline.
Why is the CI/CD process disrupted?
Microservices are deployed independently
and that change impacts everything.
11. Builds are Different
Smaller code means smaller builds, if at all. Python
is interpreted.
Linking is done at runtime, not at compile/link time.
Builds will focus on creating a container.
+
Reality of Builds
12. Traditional Modern
DEV DEVPROD PROD
Reality of Builds
Less Control
The compile/link step assembles the
complete ‘application’ package to pass
to Test and Prod. Library configuration
management decision making is done
here. This is the hear of CI.
Microservices are loosely
coupled and linked at runtime
via APIs. Configuration
information is buried in
Kubernetes Deployment files or
Helm files.
13. • A service should implement a small set of strongly related functions;
• Services that change together should be packaged together;
• The Service can be changed without affecting clients;
If you are writing microservices, what
language are you using?
Poll Question 2
14. Independently
Deployable
Because Microservices are independently
deployable - they should have their own
Repository and Workflow. Your application
will workflow will be replaced by many
microservice workflows.
Reality of Deployments
Loss of an Application Version and View
15. Multiple Workflows
To manage many moving parts, each
microservice will have their own repository
and CD Workflow. Orchestration of the CD
process will become increasingly critical.
Reality of Workflows
Multiple Workflows for a Single Application
16. • A service should implement a small set of strongly related functions;
• Services that change together should be packaged together;
• The Service can be changed without affecting clients;
Are you currently or planning to manage
each microservice in an independent
workflow?
Poll Question 3
17. Configuration
Management
Your application goes away, but a logical view is
of its configuration is critical. Mapping (with
versions) your service to service and application
to service dependencies replaces your
traditional software bill of material report.
Haunted Graveyards, Frankenstein
Clusters, when do we deprecate?
Reality of Configuration
Navigating the Deathstar
18. • A service should implement a small set of strongly related functions;
• Services that change together should be packaged together;
• The Service can be changed without affecting clients;
Are you tracking service to service or
application to service dependencies?
Poll Question 4
19. New Microservice Pipeline
Build
microservice
container
image
Continually track
service to service and service
to application configurations
Push a new
single
microservice
to the cluster
(Test, Prod)
Collect Feedback
Configuration reporting
identifies impact.
CD Pipeline
for each
microservice
20. Security Repo
Front End Repo
Product Catalog
Repo
Payment
Service Repo
Security Repo V1
Front End
Service V3
Product
Catalog Service
V5
Payment Service
V2
GitLab Workflow
GitLab Workflow
GitLab Workflow
GitLab Workflow
Application
Hipster Store
V2
Prod
Test
Dev
Hipster Store
microservices
Shifting with GitLab and DeployHub
21. Results
GitLab and DeployHub centralized reports
provides much needed visibility into containers
and what is running in your cluster
Container Image Build replaces
monolithic compile/link scripts
The GitLab delivery pipeline supports
independently deployable microservices
with 100s of workflows
DeployHub Configuration management is critical
to provide continuous mapping of service to
service and application to service relationships
(Your new BOM)
22. • DeployHub is a SaaS based central “Hub” for
sharing, managing and releasing microservices
• DeployHub tracks and shows the complete
application picture
• DeployHub simplifies the complexity of microservices
so organizations can achieve business agility
DeployHub – The First Microservice
Management Platform
DeployHub Team
Hosted Open Source
Ortelius.io
Open Source Project
We open sourced the mapping - DeployHub
Team is free and based on our Open source
Project – Ortelius.io
23. Microservices
Maturity
High level of re-use
and collaboration
Digital Transformation
Business Agility
Microservices
Initiatives
High Performance
Computing
Facilitates
collaboration
and reuse
Microservices
adoption
stages
DeployHub streamlines the microservices adoption journey
Reduced cost
- minimizes code redundancy
- Reduces confusion
- Reduced deployment cost
Increased Collaboration
-internal ‘marketplace’ of
microservices.
Data-Driven Decisions
- Service to Service Mapping
- Application versioning
Integrated into CD
- Automated configuration
management.
- Evolves the CD Pipeline for
microservice management.
Containerized
Applications and
traditional models
Tracking content and
microservice
relationships
Hybrid DeployDomains
Domain structure
to facilitate
microservice re-use
Independently
releases microservices
Organization
Map and
Deploy
Provides service to
service and
application to
service
relationships
mapping and
release.
Central Catalog of
microservices and
reusable components.
DeployHub facilitates microservice
collaboration and reuse as you mature
Typical starting points
Versioning
24. DeployHub – Benefits
Visibility gives Site
Reliability
Engineers the
knowledge of
what’s going on
and the ability to
make data-driven
decisions quickly
Automated
relationship mapping
saves 1-2 hours of
manual work per
week for every
development team
Microservice
sharing reduces
redundant coding
by up to 50% and
creates a flexible
and dynamic
ecosystem
The delivery
pipeline is
evolved to
support
microservices
without a costly
re-design
25. GITLAB IS REPLACING POINT TOOLS…
Project management
SCM
CI/CD
Issue tracking
Container registries
Logging
Dependency scanning
License management
AND ENABLING ENTIRELY NEW CAPABILITIES…
Concurrent development
Moving security forward (SecDevOps)
Seamless collaboration
Full accountability
Cycle time measurement
Transparency
Real-time feedback
...
A single, intuitive user experience, data model and integrations
Say Hello to GitLab - an entirely new thing
27. DevOps Maturity
Faster releases, fewer
errors, lower costs
Digital Transformation
Faster time-to-market,
increased revenue
DevOps Initiatives
Better collaboration,
higher IT productivity
Holistic value
increases
with GitLab
adoption
DevOps
adoption
stages
GitLab streamlines the DevOps adoption journey
Reduced cost
- Single license
- Reduced maintenance
- Reduced integration costs
Auditability
- Single data store
Improved toolchain security
Speed of innovation
Improved compliance
Continuous improvement
visibility and analytics
- Cycle time analytics
- Single pane of glass visibility
Seamless collaboration
Developer experience
- Single interface
- Developer flexibility (movement
between projects/new toolsets)
Code reviews,
collaboration,
developer experience,
productivity
Security Dashboard,
Shift left Security
SecuritySCM CDCI
Automated
build/test pipeline,
quality and speed
Multi cloud,
flexible infrastructure,
infrastructure as code
Portfolio
Project
Backlog, sprint,
burndown, tracking
deliverable
Epics and roadmaps
enable visualizing and
prioritizing future work
Holistic value increases with GitLab
adoption
Typical starting points
28.
29. Pricing
Team Pro Enterprise
Free $333
Per Project/Month
$160k
• Unlimited Projects
• Unlimited Users
• Unlimited Updates
• Unlimited Endpoints
• Restricted Domains
• Restricted Security
• No User Groups
• Single Project
• Unlimited Users
• Unlimited Updates
• Unlimited Endpoints
• Full Domain Hierarchy
• Secured Updates
• User and Group
Access Privileges
• Unlimited Projects
• Unlimited Users
• Unlimited Endpoints
• Unlimited Updates
• Full Domain Hierarchy
• Secured Updates
• User and Group
Access Privileges
30. API Developers publish
their microservices into
the Domain Catalog
allowing other developers
to easily find and reuse
the service. All meta data
is collected and
versioned.
How it Works
Application Developers define
their application by
associating which
microservices are used. They
also publish application
specific services. All
application level meta data is
collected and versioned.
DeployHub’s versioning engine
maps microservice versions to
the application versions that
consume them, with all
deployment meta data.
(Change Request, SHA, Helm
Chart, DB updates, env.
settings, etc. )
DeployHub feeds deployment data and
logic to deployment engines to update
independent microservices to the
cluster, based on application version
and service version.
DeployHub locks the deployment to
its backend engine, providing a
‘Single Source of Truth’ for what is
running in the cluster.