August 22, 2017 presentation at the Sydney Cloud Native Meetup.

1. Application Automation with Habitat
Sydney Cloud Native Meetup
August 22, 2017

2. Matt Ray
Manager, Solutions Architect – APJ
Chef Software
matt@chef.io
@mattray

3. Consumption Models have Changed
● Standard “IT Shared Services” approach no longer viable
● Dev teams have spoken; they want agility and self-service
● Various dev teams assemble their own toolchains to move quickly
● Thrashing on toolchains prevents cross-organizational success
● Traditional centralized decision making isn’t going to work

4. Problems with Shared Services
● Slow
● Inflexible
● Enforce “control” vs enabling “access”
● Creates a barrier that slows down time to market

5. Consumption Models
Old
Create a helpdesk ticket
Send an email
Submit a Change Request
New
API-driven interaction
SDK for developers to build on top of APIs
CLI tools for ease of use
Slack (searchable real-time communication)

6. LOBLOB
FOCUS ON SPEED
Tension caused by the
demands placed on teams…
…can be resolved by vertical
integration and automation…
…to deliver a future of developer
services and software at speed
SHARED SERVICES
Vertical Integration is key to velocity
A balance of increasing speed, improved efficiency and decreasing risk
FOCUS ON RISK
SHARED SERVICES
LOB
Line of
Business
LOB
Line of
Business
LOB
SHARED SERVICES
LOB
DEVELOPER SERVICES
BUILD • DEPLOY • MANAGE • COLLABORATE

7. ”
Business Value with Developer Services
Shifting capabilities to match business requirements
Developer Services Engineer
Line of Business Development Team
I provide services that developers and development
teams use to build and deliver applications.
Developer Services TeamsTraditional Central IT
System Administrator
Centralized Enterprise IT Team
I manage and deliver infrastructure required to run
software in my organization.
Enabling development teams to get insights into speed,
efficiency and risk of delivery of their software
Reducing risk to my organization from my infrastructure
and software that runs on it
Enabling development teams to ship software at speed while
maintaining quality and reducing risk
Reliably managing changes to infrastructure
requirements
Providing on-demand, self-service infrastructure and services
tailored to developer needs
Managing and lowering costs of running, configuring,
and maintaining infrastructure
DEPLOY
MANAGE
Don’t measure me on traditional IT metrics, but on the metrics of the business
Jim Fowler, CIO, GE Capital
BUILD
Reducing friction by enabling full stack transparency and
management capability with separation of duties
Delivering configured infrastructure and handing off
responsibility to development teams
COLLABORATE

8. Modern Architecture Trends

9. Modern Architecture Principles
12 Factor Apps/Cloud Native
Microservices
Serverless
Containers

10. 12 Factor/Cloud Native
https://12factor.net/
“The twelve-factor app is a methodology for building software-as-a-service apps that:
● Use declarative formats for setup automation, to minimize time and cost for new developers joining the project;
● Have a clean contract with the underlying operating system, offering maximum portability between execution environments;
● Are suitable for deployment on modern cloud platforms, obviating the need for servers and systems administration;
● Minimize divergence between development and production, enabling continuous deployment for maximum agility;
● And can scale up without significant changes to tooling, architecture, or development practices.”

11. Microservices
“…the microservice architectural style is an approach to developing a single
application as a suite of small services, each running in its own process and
communicating with lightweight mechanisms…”
- Martin Fowler and James Harris
https://martinfowler.com/articles/microservices.html

12. Microservices Characteristics
● Componentization via Services
● Organized around Business Capabilities
● Products not Projects
● Smart endpoints and dumb pipes
● Decentralized Governance
● Decentralized Data Management
● Infrastructure Automation
● Design for failure
● Evolutionary Design

13. Serverless
“Serverless architecture replaces long-running virtual machines with
ephemeral compute power that comes into existence on request and
disappears immediately after use.”
- Thoughtworks Tech Radar, March 2017

14. Containers
Package the application and dependencies in a deployable format
Reduce/remove dependency on underlying OS/Infra
Small Area of Concern
Single process

15. Common themes
Reduced Area of Concern
Service Independence
Infrastructure Independence
Decentralized Management

16. Monolith ServerlessMicroservicesApplication
Application
Services
Centralized Highly Distributed

17. Monolith ServerlessMicroservicesApplication
Application
Services
Few Large Artifacts Thousands of
Small Artifacts

18. Complexity Cliff
Scale Increases -
Number of services under management increases
Efficiency Declines -
Number of ways to manage each service is different
Risk Exposure Increases -
Confidence that each service is secure, compliant, updated, etc.

19. Developer Services
“The services required for application developers to effectively run
applications. This consists of the underlying infrastructure, compute runtimes,
language runtimes, and application services to persist and retrieve data.
These services should be available via a common API, with a focus on making
the service easy to consume.”
- Me

20. Application Services
What Developers Need
Application Runtimes

21. Habitat is automation for
Application Runtimes & Application Services
to deliver Cloud Native Developer Services

22. Building Developer Services
With Habitat

23. Lifecycle of any software
Build Deploy Run

24. Build Lifecycle

25. The dreaded build cycle
wget https://some.place/package.tar.gz
tar xfzv package.tar.gz
cd package
./configure
make
make install

26. The dreaded build cycle

27. The dreaded build cycle

28. Habitat Defines the Build Lifecycle
● Default implementation is C/C++ Build Lifecycle
● Other default implementations provided by Scaffolding
○ core/scaffolding-ruby
○ core/scaffolding-node
○ core/scaffolding-go
● Build dependencies explicitly declared
● Runtime dependencies explicitly declared.

29. Habitat Artifacts
● Dependencies declared
○ Libraries
○ External Services
● Services/Port-bindings Exported
● Signed

30. Habitat Build Cycle
USER ARTIFACTPLAN DEPOT

31. DEMO

32. Deploy Lifecycle

33. Deploying Artifacts
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
DEPOT ARTIFACT
BARE METAL
CONTAINERS
AMI
VM

34. Libraries
Operating System
Application
Traditionally
the domain of
the OS Vendor
Application Libraries
Config Changes
BottomtoTopDependencyResolution

35. Application &
Libraries
Minimized OS
ToptoBottomDependencyResolution

37. Export Deployable Artifacts
● Docker
● ACI
● Mesos
● TAR

38. Run Lifecycle

39. What do applications need to run?

40. What do applications need to run?
• Process Supervisor
• Lifecycle events
• Start, Stop, Reconfigure, etc
• Environment specific configuration
• Knowledge of peers
• Knowledge of dependent services

41. The Habitat Supervisor
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR
SERVICE
SUPERVISOR

42. Lifecycle Hooks
• init
• run
• health_check
• file_updated
• reload
• reconfigure
• post-run
• suitability

43. Peers
Service Groups
Gossip
Availability
increases with
scale
Supervisors Form a Ring

44. Configuration
• Config files stored in ./config
• Default values provided by
• Scaffolding
• default.toml
• Override defaults through
• ENV variable
• user.toml
• Over the network

45. What do applications need to run?

46. Dynamic
configuration
Service group
level
Uses the ring
Supervisors Support Clustering Topologies

47. Self Organizing Applications
• Habitat applications can self organize
• Typical container pattern
• Bake intent into the container
• Redis-master container image
• Redis-follower container image
• Introduces image sprawl

48. Self Organizing Applications
•Habitat pattern
• One container image
• Launch images with knowledge of peers
• Application self organizes

49. External Actors
Health and
Status
Supervisor
Debugging
Supervisors provide a REST API

50. What Habitat brings to Containers:
• Build containers from the Application down
• Build the minimal viable container by default
• Export containers in a variety of deployable formats
• Automatically export containers with:
• Service Discovery
• Configuration Management
• Supervisor API
• Clustering Topology Support

51. What about those Developer Services?

52. Habitat and Developer Services
● Define How Software is Built
○ Language Agnostic
● Define How Software is Deployed
○ Export software to a variety of deployable formats (like containers)
● Define How Software is Run
○ Common management plane for any software
○ Common protocol for
■ Config
■ Cluster
○ Common API for monitoring

53. Habitat and Developer Services
Quickly and easily build, deploy, and run any
software across a variety of compute runtimes.

54. Habitat is automation for
Application Runtimes & Application Services
to deliver Cloud Native Developer Services

55. It’s all open source
Apache License

56. Questions
• Join the Habitat Slack Team - http://slack.habitat.sh/
• Work through the tutorial at https://www.habitat.sh/tutorials/
• Explore Habitat packages on the depot - https://app.habitat.sh/
• Explore the Habitat projects - https://github.com/habitat-sh
• Read Habitat Blog posts - https://blog.chef.io/?s=habitat
• Join the Habitat Forums - https://forums.habitat.sh/