Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings

12/12/2018 1Demetris Trihinas
trihinas.d@unic.ac.cy
1Tutorial | CloudCom 2018
Designing Scalable and Secure
Microservices by Embracing
DevOps-as-a-Service Offerings
Demetris Trihinas+*, Athanasios Tryfonos*,
Marios D. Dikaiakos*
* Department of Computer Science
University of Cyprus
+ Department of Computer Science
University of Nicosia
IEEE International Conference on Cloud Computing Technology and Science (IEEE CloudCom 2018)

Tutorial Overview
• Software architecture paradigms
• Containerization and Microservices
• Microservices design and development
• Microservices challenges at scale
• Unicorn DevOps-as-a-Service Platform
Monolithic
SOA
Microservices

The Good Old Days…
• You had an idea for an application…
Books
UI
Ratings
Shopping Cart
Reviews
Book Store
(WAR/EAR)
Application Server

Hello… Monolith
• A monolith is a software application whose architectural
modules cannot be executed independently.
• The term monolith first appeared in
“The Art of Unix Programming” to
describe systems that get too big.

Time Passes By...
• As you add more features it becomes difficult to track
bugs and maintain a healthy code base.
• You enter “dependency hell” because adding or
updating libraries in shared projects results in
inconsistencies.
Monoliths are difficult to maintain and evolve due to
their complexity
Michael Jang, “Linux Annoyances for Geeks”, 2006

Your User-Base Grows…
• You hire more developers to cope with adding new
features, improvements and bugs.
• Embrace cloud computing to handle increased demand
with pay-as-you-use model.usage

Cloud Computing
Books
UI
Ratings
Shopping Cart
Reviews
Book Store
(WAR/EAR)
Application Server
Pay as you use model

Cloud Computing - Vertical Scaling
Books
UI
Ratings
Shopping Cart
Reviews
Book Store
(WAR/EAR)
Application Server

Cloud Computing - Vertical Scaling
Books
UI
Ratings
Shopping Cart
Reviews
Book Store
(WAR/EAR)
Application Server
Pay as you use

Challenges of a Monolith
• To cope with increasing load, a larger server/VM is
allocated.
• But… VM pricing does not scale linearly.
• Any change in one module of a monolith requires
rebooting the whole application.
• For large-sized monoliths rebooting means downtime.

Cloud Computing - Horizontal Scaling
Books
UI
Ratings
Shopping Cart
Reviews
Book Store
(WAR/EAR)
Application Server

Challenges of a Monolith
• But… sub-optimal deployment because of conflicting
requirements.
• For example one module is memory-bound, others cpu-
bound, while another may be sparsely accessed.
• To satisfy every module’s requirements, you end up
with LARGE MONOLITHS.

Lack of Innovation
Books
UI
Ratings
Shopping Cart
Reviews
Book Store
(WAR/EAR)
Application Server
Build/Test/Release
Long build cycles and releases take months.
who broke
the built?

Software Architecture Paradigms
time

Hello… Service-Oriented Architecture
• The monolith is decomposed into services.
• Services are based on separation of concerns.
• Services expose their offerings to other services over a
communication mechanism –universally agreed
among all services– part of the application.
• Enterprise Service Bus (EBS)
Georgakopoulos, D., Papazoglou, M.P.: Service-Oriented Computing. The MIT Press, Cambridge (2008)

Benefits of SOA
• Dynamism – launch only instances of the services in
demand and use only the needed resources.
• Services can be reused in other systems.
Book
Service
Rating
Service
Cart
Service
• New services easily integrated
by embracing comm. protocol*.
• Services can be independently
developed*.
N. Dragoni “Microservices: yesterday, today, and tomorrow”, Software Engineering, Springer, 2017

The Enterprise Service Bus
Book
Service
Rating
Service
Cart
Service
A. Barker, C. D. Walton, D. Robertson, “Choreographing Web Services”. IEEE Transactions on Services Computing, 2009.
• Sophisticated communication facilities via choreography:
• Message-passing and business rules application to establish
service collaboration.

Drawbacks of SOA
• The ESB does not scale and is ungovernable as you add
more services.
Book
Service
Rating
Service
Cart
Service
• To change anything in the
ESB or the communication
protocol ALL services must
be changed as well
• So… builds still take forever.
Jim Webber and Martin Fowler, “Does My Bus Look Big In This”, 2008

Software Architecture Paradigms
time

Hello… Microservices
• The Monolith is decomposed into even smaller
services.
• Each discrete business functionality is a service.
• Microservices run independently and inter-
communicate using lightweight communication
mechanisms.
• Platform-agnostic API’s over the internet (bye-bye EBS).

Decentralized Governance and Data
• Microservices are to be as decoupled and as cohesive as
possible.
• Each service can be independently developed in different prog.
Languages and frameworks, and even have its own database.
• Each (micro-)service owns its own domain logic and acts
more as a filter in the classical Unix sense.
• A service receives a request, it applies logic as appropriate and
produces a response that is served back to the client.
J. Thones. Microservices. IEEE Software, Jan 2015.

Microservices Architecture
Book
Service
Rating
Service
Cart
Service
Review
Service
RESTTCP SOAP REST

Microservices == SOA Revamped?
• The term “microservices” was first introduced in 2011
at a SOA workshop (Venice, Italy).
• Netflix initially used the name “Fine-grained SOA” to
describe its microservices architecture.
• Martin Fowler - “In contrast to SOA, microservices are
about smart endpoints and dumb pipes”
James Lewis, “Micro services - Java, the Unix Way ”, 2011
Netflix, “Announcing Ribbon: : Tying the Netflix Mid-Tier Services Together”, 2013

Microservices and DevOps
Embrace agility and autonomicity to increase innovation!
Build/Test/Release UI
Build/Test/Release Books
Build/Test/Release Ratings
Build/Test/Release Reviews
Build/Test/Release Carts
For each service one can understand, alter and write new code without
knowing anything about the internals of its peers.

Amazon e-Commerce Platform
Peter Dalbhanjan, Solutions Architect AWS, 2016

Does Size Matter?
• How Big Should a “Micro” Service Be?
• It’s about the team… Amazon’s “Two pizza rule”
• J. Bezos: If you cant feed a team with two pizzas its too large.
• It’s about discretizing functionality.
• “Micro” has nothing to do with size – 5MB or 5GB is
irrelevant.

Conway’s Law - DevOps in 1967
…any organization that designs a system will inevitably
produce a design whose structure is a copy of the
organization’s communication structure…
Melvin E. Conway, 1967

Service Deployments
• Decoupling services means builds and releases need
less coordination (from 1 app -> N services).
• You build it, you run it.
• Deployments are also more often.
• This means… deployments must be:
• Fast, Reliable and Automated.
Jim Gray, “A Conversation with Werner Vogels: Learning from the Amazon technology platform”, ACM Queue, 2006.

Containerization
• Containerization allows applications to share a single
host OS in a portable manner.
• Perfect for microservices as complex applications are
split into discrete and modular units.
• Reduced management complexity because problem or
change related to one service does not require an
overhaul of the overall application.
R. Morabito, V. Cozzolino, A. Y. Ding, N. Beijar and J. Ott, "Consolidate IoT Edge Computing with Lightweight
Virtualization,” IEEE Networks, 2018.

Containers are an OS Technology
GENERAL DISTRIBUTION
CONTAINERS ARE AN OS TECHNOLOGY
TRADITIONAL OS CONTAINERS
Hardware
LIB A
Host OS
LIB B LIB C
Hardware
LIB A
OS Deps
Container
Host OS
Container Container
App A App B App C
App A
LIB B
OS Deps
App B
LIB C
OS Deps
App C
Remember
dependency hell?
Container Isolation

But… Containers are NOT a New Technology
• Part of the Linux ecosystem for more than a decade.
• Resource isolation at the OS level through kernel
namespaces and control groups.
• Namespaces deal with resource isolation for a single
process.
• cgroups manage access for groups of processes.
S. Hogg, "Software Containers: Used More Frequently than Most Realize”, Network Magazine, 2015.

VMs vs Containers
VIRTUAL MACHINES AND CONTAINERS
VIRTUAL MACHINES CONTAINERS
virtual machines are isolated
apps are not
containers are isolated
so are the apps
VM
OSDependencies
Kernel
Hypervisor
Hardware
App App App App
Hardware
Container Host (Kernel)
Container
App
OSdeps
Container
App
OSdeps
Container
App
OSdeps
Container
App
OSdeps

At the end… What are Containers?
• It depends who you ask…
WHAT ARE CONTAINERS?
It Depends Who You Ask
● Sandboxed application processes on a
shared Linux OS kernel
● Simpler, lighter, and denser than virtual
machines
● Portable across different environments
● Package my application and all of its
dependencies
● Deploy to any environment in seconds
and enable CI/CD
● Easily access and share containerized
components
INFRASTRUCTURE APPLICATIONS
Amir, Zipory (Red Hat Openshift), “DevOps with Containers: From Hype to Reality”, 2018.

Docker – Container Engine
• To run containers, specialized software must streamline
and coordinate the process on top of the OS.
• Golden standard for containerized technologies.

Docker Mantra
• Build, ship, run any application, anywhere.
• Build – package app in a container.
• Ship – move container from one machine to another.
• Run – execute the container runtime.
• Any application – literally anything.
• Anywhere – Cloud, laptop, bare metal, microcontroller,
sensor (?).
Dirk Merkel, “Docker: lightweight Linux containers for consistent development and deployment”, Linux J., 2014(239),March 2014.

Designing and Developing
Microservices

Tutorial Code is Publically Available
https://github.com/UCY-LINC-LAB/CloudCom2018-Tutorial

Microservices Development
“Decomposing” the monolith

Decomposing the Monolith into Services
• Books – product information (e.g.
title, author, summary, ISBN)
• Ratings – star-based score
• Reviews – user feedback
• Cart – items to be purchased
• Each Service has its own data store
Books
Ratings
Carts
Reviews
Database
UI

The Book Store

“Behind” the Book Store
Books
Ratings
Reviews
Cart
Separation of Logic!

• Java framework providing complimentary to the business
logic features for app development:
• Embedded servers to deploy within JVM containers.
• Application components are auto-configured externally upon
instantiation and at runtime.
• Inherently distributed with tools for service discovery, security, API
gateways, circuit breaking, message-passing, database integration.
• Open-source ecosystem with tons of smaller frameworks created
(e.g., major contributors include Netflix, AWS, Spotify).
http://spring.io/projects/spring-boot

• Spring Boot is based on Spring Framework:
• Aspect-Oriented Programming (AOP).
• Model-View-Controller (MVC) approach for web service
development.
• Annotations for including Spring –complimentary– features.
• Authentication, Authorization, Logging, Testing…
• Limited to running on a single JVM container.
https://spring.io/

Annotation-Based Programming
• Annotations were initailly a form of metadata providing
instructions that are not part of the actual application.
• So they do not “directly” affect the program semantics.
• They provide compilers and build engines with:
• Useful information and
hints

Annotation-Based Programming
• They provide compilers and build engines with:
• Avoids bloating of business logic!
Code injection and
interpretation at
compile or
deployment time.

Other Microservices Frameworks
• Eclipse Vert.X
• Lightweight framework (use only the libs you need).
• Polyglot (Java, JavaScript, Groovy, Ruby, Scala and Kotlin).
• Reactive programming (event-driven and non-blocking).
• Framework functionality mixes with business logic.
• Relatively new with need of more documentation and guides.
https://vertx.io/

Other Microservices Frameworks
• Thorntail (previously Wildfly Swarm)
• RedHat’s open-source Java application server micro-framework.
• Lightweight framework (use only the libs you need).
• Framework functionality mixes with business logic.
• New with need of more documentation and guides.
• Oracle Helidon (another Java framework, introduced 2018).
• Nameko (python, RPC over AMQP)
• Java seems the language of trade. https://thorntail.io/
https://helidon.io/
https://www.nameko.io/

The “Books” (Micro-) Service
The Books microservice entry point

The “Books” (Micro-) Service
• Spring Boot does all the configuration (e.g., dependency
management, parameter passing).
• Bundles app with web server (Apache Tomcat by default).
All is done with the addition of a single annotation

The “Book” Service
• To use Spring Boot you need to include the respected
dependency and plugin to your dependency management
tool (e.g., mvn).

The “Book” Model

The “Book” Model
No mention of underlying data
store (e.g., MySQL, MariaDB)!
Create relational entity (a.k.a table)
Auto-generated primary key!
Define properties that cannot be
left blank!

The “Book” Repository

• All CRUD operations are auto-generated and made
available by Spring Boot.
• Create new record.
• Update existing record.
• Delete record.
• Find one record.
• Find All records.

• Repository functionality can be extended.
Extend lookup queries to use book title
--secondary index-- not just book id.

Configuring Data Store via Spring
Through your microservice config file -> NO code changes!

Spring Data for Entity Model and Repository
• In JPA are the modeling
handlers for persistent
storage (entity, etc.).
• Connector to use MySQL
(not part of Spring Boot!).

The “Books” Controller

• Handles all “book” requests.
• Provides REST API for accessing book repository.
• Maps requests to database/repository queries.

REST API landing URI
Automatic dependency injection
Microservices in microservices!
Book Controller returns a JSON array of ALL Book entities in DB
Change from JSON to XML, plain
text or HTML by only changing
parameter!

Microservices only Rely on Each Others
Public API
• Database technology or schema can change but the client
doesn’t care as access to data is provided via the API.
Microservice Y
Public
API
Microservice X
Public
API
Nope!

Books
Ratings
Reviews
Cart

• The Ratings, Reviews and Cart Microservices
developed following similar design pattern.
• Omitting due to time.
• But… code publically available:
https://github.com/UCY-LINC-LAB/CloudCom2018-Tutorial

Service Discovery and Load
Balancing
How do services find each other

Service Discovery
• Service discovery uses a registry to keep a real-time list
of services, their location, and their health.
• Services query the registry to discover the location of
other services and then connect directly.
• This allows services to scale up/down, load balancing
and gracefully handle failure in dynamic infrastructures.

The Self-Registration Pattern
• A service instance is responsible for (de-)registering
itself with the service registry.
• The service instance sends
heartbeat requests to prevent
its registration from expiring.

The Client-Side Discovery Pattern
• The client is responsible for determining the network
locations of available services and load balancing
requests across them.
• The client queries a service registry, which is essentially
a database with the available services locations.
• The client then selects one of the available services
(e.g. load-balancing algorithm) to make its request.

The Client-Side Discovery Pattern
Drawback #1 -> The client is highly
coupled with the service registry.
Drawback #2 -> Service discovery logic
MUST be implemented for each
programming language and frameworks
used by service clients.
• Client knows about all available services -> intelligent and
application-specific load-balancing decisions.
https://github.com/Netflix/eureka

• The client makes a request to a service via a load balancer.
• The load balancer queries the service registry and routes
each request to an available service instance.
The Server-Side Discovery Pattern
Service Registry + Load Balancer
two for the price of one!

The Server-Side Discovery Pattern
• etcd - Highly available key-value store used for
shared configuration and service discovery
(CloudFoundry, Kubernetes).
• Consul - provides API that allows clients to
register and discover services. Performs health
checks to determine service availability.
https://coreos.com/etcd/
https://www.consul.io/

Books
Ratings
Cart
Service
Registry

Books
Ratings
Reviews
Cart
Service
Registry
Register
“Reviews” as a service

Books
Ratings
Reviews
Cart
Service
Registry

Service Discovery with Consul
• Integrated well with Spring Boot with code annotations
and auto-configuration enabled.
All is done with the addition of a single annotation

• To use Consul annotations simply add the respected
dependency to mvn.

Configuring Service Discovery
• Through your microservice config file -> NO code changes!
Auto-generated Instance IDs for
dynamic resource provisioning!
Tags to categorize your services!

The tags declared in microservice
configuration!

Same Service ID
But… unique InstanceID

Heath Check
Report

• Map requests to services and retrieve resources.
• Distribute load among services.
Reverse Proxy and Load Balancing
Horizontal
Scalability

• Traditional reverse-proxies (e.g., HAproxy) require that
you statically configure each route that will connect
paths and sub-domains to each microservice.
• In a dynamic environment you add, remove, kill,
upgrade, or scale; your services many times a day.
• keeping routes up-to-date becomes tedious.

Books
Ratings
Reviews
Carts
Load
Balancer
Service
Registry
Request Route
Books#3
Ratings#1
Reviews#1
Carts#2 Services
Books{1,2,3}
Ratings{1}
Reviews{1,2}
Carts{1,2,3}

Traefik
• Reverse proxy and load balancer.
• Auto-configurable and supports dynamic discovery.
• Traefik listens to your service registry (e.g., Consul) or
orchestrator API (e.g., Kubernetes).
• Instantly generates the routes so your microservices
are connected to the outside world -- without manual
intervention.
https://traefik.io/

Traefik – Proxy Service Catalog
Micro-services
categorized by
declared tags

Traefik – Load Balancing
Define Load Balancing Strategies
Adjust weighting to skew the load
A/B testing
10% of traffic to test beta version
There is even option for dynamic
weighting which increases weights
on servers that perform better
than others.

• Load balancer can change anytime without any code
changes in to the microservices!
• Other Reverse Proxies and Load Balancers
• Google’s Envoy:
• Adopts side-car paradigm for containers
• Netflix OSS:
• Eureka (SD) + Ribbon (LB) + Hystrix (CB)
• Must be configured through app.
https://www.envoyproxy.io/

API Gateway
“Homogenizing” resource access.

The Book Store

API Gateway and Identity Service
• Coordinates how all client requests are processed and also
performs request validation.
• Allows for clean clients - no need to implement different
request technologies (e.g., REST, SOAP, etc).
• Gateway authenticates user via identity service which
provides access token.
• All requests going to microservices without a valid token
are rejected.

API Gateway
• Developed also by using microservices design pattern.

API Gateway
• Endpoints to services are NOT hardcoded, instead they
are “autowired” -> microservices in microservices.
API version request mapping
Autowired endpoints to
book store services

Books
Ratings
Reviews
Cart
Load
Balancer
API
Gateway
Service
Registry
Identity
Service

Books
Ratings
Reviews
Cart
Load
Balancer
API
Gateway
Service
Registry
Identity
Service
The
“front”
door
Behind private
network

Circuit Breaking
• Prevents high loads on failing servers based on various
strategies:
• One service cannot bring down the entire application.
Books
Ratings
CartCB
Bug in latest version
of ratings service.

The Book Store Frontend
One service cannot bring down
the entire application.

Circuit Breaking
ASYNChronous resource fetching
with 3 second timeout.

Other Circuit Breaking Strategies
• Error-ratio over 10s sliding window:
• Latency at quantile in milliseconds:
• Ratio of response code range:
NetworkErrorRatio() > 0.5
LatencyAtQuantileMS(50.0) > 50
ResponseCodeRatio(500, 600) > 0.5

Circuit Breaking
Without taking steps to ensure fault tolerance, 30
dependencies each with 99.99% uptime would result in
2+ hours downtime/month
(99.99%^30 = 99.7% uptime = 2+ hours in a month)
Ben Christensen,
Netflix Engineer

API Gateway - Versioning
• A key factor of resilience is that when you make upgrades
to your services, your clients don’t suffer from downtime.
• In the "old" days, upgrades were notorious for breaking
clients.
• An upgrade to the server required an update to the client.
• Nowadays, minutes of downtime for doing an upgrade can
cost millions in revenue.

API Gateway - Versioning
• For example:
• Instead of author, management decides to change field to
author_firstname, author_lastname.
• Easy update for db engineer but what about your clients?
• Remember: downtime -> lose of money
• Naïve solution… wake up at 2am to do upgrades.
• But… it‘s not 2am everywhere when offering a service over
the internet.

API Versioning – Backwards Compatible
• Old strategy that precedes microservices by many years.
• Never delete a column in a database (M. Stonebraker)
• You can add columns to a DB but never take them away.
• Not every change is possible with duplication. In such case v2 is
introduced and clients not upgraded are served from v1.
“Database Decay and How to Avoid It”. M. Stonebraker, D. Deng, M. L. Brodie, IEEE Big Data, 2016
Duplicate info but no client
downtime

Token-Based Authentication
• Spring Security extends Spring Boot functionality to
support authentication.
• Each request includes token (ideally in request header),
allowing access to routes, services, and resources.
• Token based authentication: JWT, OAuth2

JSON Web Token Authentication
• JWT is open standard (RFC 7519) for secure data
transmission between parties as a JSON object.
• Data can be trusted because it is digitally signed.
• Secret key (HMAC algorithm) or a private/public RSA key pair.
• Token Signature is calculated using the header and the
token payload -> verify the integrity of the content.
https://jwt.io/

Books Microservices
Ratings Microservices
Reviews Microservices
Cart Microservices
Token validated at the
microservice level so that
auth server is never
stressed.
Only creates new tokens
and refreshes expired
ones.

JWT validation in Spring is just a WebFilter that checks token signature.

API Documentation
• Documentation is a headache to maintain.
• A single API change and the documentation is out-of-
date.
• API documentation should be updated and visualized
automatically.
• If the API is public then the “public” must be able to
access it.

Swagger API Documentation
https://swagger.io/

Swagger API and UI dependency
for mvn.
Swagger API Documentation
REST controller and
methods decorated
with API annotations.

Containerization
Dockerizing the microservices

Dockerizing the Microservices
• Package and deploy microservices in container
runtimes for portability across private and public
clouds.
• Prepare Dockerfile for each service to configure the
bundling of the service with the container.
• Prepare Docker Compose description to automate the
configuration and deployment of the entire platform.

The Dockerfile
• A text file that contains all commands needed to
prepare a Docker image = container + service config.
• A Docker image is comprised of read-only layers with
each layer representing a Dockerfile instruction.
• The layers are stacked and each one is a delta of the
changes from the previous layer.

Book Service Dockerfile
Use as a base for this image another image or start FROM scratch
Add files to image from your local machine
What commands to run within the container
Instructions to build the container
(e.g., install java 8, set environmental
variables)

Book Service Dockerfile
These commands are used in ALL our
services (e.g., rating, review, cart)
Hello… BookBase Docker image

Ratings Service Dockerfile
A container built from
other containers!

Docker Compose
• Putting everything together.
• Multi-container/Multi-service
declaration and configuration.
• Common subnet network.
• Persistent volume for consul and
databases.
• CI workflow integration.

Docker Compose
• Books, Ratings, Reviews
and Cart Services
Put Books service behind cloudcom
private network!

Docker Compose
• Consul Service Registry
No dockerfile… consul image available in docker hub
Also part of the private cloudcom subnet

Microservices Challenges

Microservices at Scale
• In the cloud era, as applications grow by adding more
services, dynamic resource allocation and security
enforcement become significant challenges.
• These challenges must be fostered through autonomicity.

Monitoring and Diagnostics
• Containers ease deployment for users – no need to configure
(virtual) infrastructure, network(s), storage.
• Nightmare for monitoring tools -> no guest OS to deploy agents
side-by-side with application.
Monitoring must be deployed and
run through container engine or
be part of the application
Monitoring is integral part of app design and cannot be decided after deployment.

• Granularly slicing an application into services inherently
introduces heterogeneity.
• Book service is latency sensitive while books accessed are logged for
analytics (e.g., popular titles).
• Ratings service is cpu-intensive.
• Reviews service is memory-bound while reviews are monitored for
helpful/deceitful reviews.
• Requires full customization of the monitoring process to
perform diagnostics and receive helpful insights.

• Monitoring ephemeral, decomposed and highly dynamic
applications -> Non-neglectable monitoring cost!
• Overhead on monitoring source itself.
• Overhead (network, storage) and actual cost on monitoring
topology.
• Overhead to process real-time data and trawl historic monitoring
data.
D. Trihinas and G. Pallis and M. D. Dikaiakos, "Low-Cost Adaptive Monitoring Techniques for the Internet of Things", IEEE Transactions
on Services Computing, 2018.

Auto-Scaling and Optimization
• Scaling to meet demand is a challenge for most
applications and microservices are no exception.
• But I thought… microservices and containers are
inherently easier to scale:
• Simply create more copies of the services overwhelmed by
demand (horizontal scaling) and fastly boot your containers.
• So which one is it?
Lopez et al. “Towards quantifiable boundaries for elastic horizontal scaling of microservices”, IEEE/ACM UCC, 2017.

Auto-Scaling and Optimization
• At scale, significant profiling is required to optimize
performance, cost and quality:
• Identify what should be monitored
• When and How to scale
• Investigate, in a distributed and granular deployment, if
one service is currently affecting the performance of
another service?
• Anticipate demand?
“[Best Paper] ADVISE – a Framework for Evaluating Cloud Service Elasticity Behavior.” Copil, G. and Trihinas, D. et al., ICSOC 2014.

Orchestration in Hybrid Cloud Deployments
• Software teams are increasingly choosing to work with
multiple cloud offerings and/or cloud providers.
• Although containers –ideally can run anywhere- this
does not mean that a deployment can span across:
• Geographic regions
• Cloud availability zones
• Different cloud sites

Orchestration in Hybrid Cloud Deployments
• Constructing and managing a cross site overlay
network?
• Network performance issues (e.g., propagation delays)
for service communication especially when network
traffic must pass across datacenter boundaries.
• Service placement strategies:
• Suitability, cost, availability, latency?

Security Enforcement and Privacy Protection
• For a monolith there is only one thing to protect… and
its still a challenge.
• Decomposing an app into services with each service
inter-communicating over the network raises
significant security risks.

• Maintaining identity and access management across
the entire deployment.
• Mitigating new sets of security rules at runtime
without service disruption?
• We cant just shut down thousands service instances for
security rule alteration.
Sun et al, Security-as-a-Service for Microservices-Based Cloud Applications, IEEE CloudCom 2015.

• Privacy leaks due to code vulnerabilities in diversified
service stack.
• In-house developed code.
• Third-party libraries, frameworks, dependencies.
• Runtime overhead for security enforcement and
privacy preserving mechanisms?
Shu et al., “A Study of Security Vulnerabilities on Docker Hub”, ACM CODASPY, 2017.
Docker Security Scanning, https://docs.docker.com/v17.12/docker-cloud/builds/image-scan/

DevOps as a Service Platform
http://unicorn-project.eu/

Unicorn Vision
Increase SME and Startup productivity by offering a DevOps-as-a-
Service platform to ease the design and deployment of secure and
elastic microservices.

Unicorn Dashboard
• One collaborative and unified environment:
• Develop apps
• Share workspaces
• Ship coded artefacts to the cloud
• Manage the entire deployment lifespan

Unicorn Dashboard: Developer Perspective
• Design libraries for policy and constraint definition through
code annotations for minimum code intrusion.
• Eclipse Che IDE plugin for collaborative development.

Unicorn Dashboard: Manager Perspective
• Annotations mapped to Service Graph description.
• Administrators can enrich graph with additional policies
without having to use code annotations.

Unicorn Dashboard: App Description
• Service graph description extends Docker Compose model.
• No need to learn another proprietary technology.
• Description can still be used in any other Docker runtime.

Unicorn Platform
• Service graph validation (e.g. antagonizing policies).
• Smart and interoperable multi-cloud orchestration.
• Runtime policy and constrain enforcement.

Unicorn Technology Stack
Developed on top of popular and open-source frameworks
including Kubernetes, Docker, CoreOS to support multi-cloud
application runtime management.

Kubernetes to support the orchestration of large-scale distributed
containerized deployments spanning across multiple hosts.
Kubernetes Limitations
- (De-)provisioning
infrastructure resources.
- Auto-scaling.
- Cross-cloud deployments.

Unicorn Smart Orchestrator extends open-source Arcadia
framework to enable Kubernetes across multiple cloud sites.
- Cloud adaptors to probe
and program underlying
infrastructure.
- Taps into auto-scaling
offered by cloud offerings
to estimate and assess
app elasticity behavior
and scaling effects.
- Cross-cloud network
overlay management to
reliably handle SDN
accessibility.

Underlying kernel for the containerized environment is CoreOS
which enables fast boot times and secure-out-of-the Docker runtime.
Unicorn “side-car”
services
- Orchestrator service for
HA host management.
- Low-cost and self-
adaptive monitoring to
reduce network traffic
propagation.
- Security service to filter
network traffic and apply
privacy preserving
ruling.

Sign-in to Unicorn

Unicorn Dashboard

Unicorn Design Libraries
• Available for downloading through Unicorn Dashboard.
• Design libraries for monitoring, elastic scaling, security
enforcement and vulnerability assessment.
• Code annotations -> no business-logic bloating.
• Currently available for java apps.

Monitoring Design Library
• Maven Dependency also available to ease design
library usage:

Monitoring Design Library
• Simply add the Unicorn Monitoring annotation and
monitoring is enabled automatically with default
configuration!

Customize Monitoring – Add Probes
Create your own
Monitoring
Probes

Customize Monitoring – Add Probes
• Spring Boot Probe (avg response time, avg throughput,
requests since last update):
Customize and
extend monitoring

Customize Monitoring – Create Probe

Register Cloud Providers
• Register and manage cloud enpoints and access tokens.

Manage Cloud Availability Regions
Enable and disable cloud availability regions for app
deployments.

Container Registry
• Users are allowed to push their docker images to the
Unicorn private registry instead of public docker hub.

Add Microservices

Creating an Application Description

Service Graph Model
Unicorn handles service registry,
load balancing, elastic scaling and
cluster management!
Only configure and
drag n’ drop your
services.

Application Deployment

Runtime Policy and Service Graph Config

Elastic Scaling

Service Graph Description Auto-Updated

Runtime Policy and Service Graph Config (2)

Security Rules and Alerts

Unicorn Validation Contest
• Startups with an idea for an app or have already
developed a cloud app… and are interested in:
• Converting their app to a microservices architecture by using
the Unicorn design libraries
• Deploying their app through the Unicorn platform
• Startups can apply for a Unicorn Validation Contract.
• 12 contracts -> 10.000 euros
http://unicorn-project.eu/index.php/contest/

http://unicorn-project.eu/index.php/contest/
Unicorn Validation Contest

Summary
• Microservices are applications decomposed into single-
function services that inter-communicate over lightweight
communication protocols.
• Designed and developed independently.
• Decentralized governance.
• Communication only relies on API’s.
• Continuous delivery.
• Failure isolation.
• Portability and flexibility via containerization.

Summary
• Autonomicity can foster microservices scalability but it
bears challenges:
• Monitoring: part of app development process, service
heterogeneity, significant overheads.
• Auto-Scaling: determine optimization strategies, detect
bottlenecks.
• Cross-cloud Orchestration: service placement strategies.
• Security: must secure multiple services instead of singleton.

Summary
• Unicorn DevOps-as-a-Service Platform
• Design libraries for microservices development
• Cloud IDE plugin for collaborative development
• Policy and constraint definition via service graph model
• Deployment via containerized runtimes (enhanced Docker
Compose)
• Multi-cloud orchestration
• Real-time monitoring, auto-scaling and security enforcement
http://unicorn-project.eu/

Unicorn Team
http://unicorn-project.eu
@unicorn_H2020
http://linc.ucy.ac.cy/
Dr. Marios
D. Dikaiakos
Dr. George
Pallis
Dr. Demetris
Trihinas
Athanasios
Tryfonos
Zacharias
Georgiou
Moysis
Simeonidis
Maria
Poveda

Designing Scalable and Secure Microservices
by Embracing DevOps-as-a-Service Offerings
Thank you!
http://unicorn-project.eu
@unicorn_H2020

Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings

Similar a Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings (20)

Más de Demetris Trihinas

Más de Demetris Trihinas (18)

Último

Último (20)

Designing Scalable and Secure Microservices by Embracing DevOps-as-a-Service Offerings