SlideShare una empresa de Scribd logo
1 de 56
Descargar para leer sin conexión
Prepared by
Build a modern infrastructure in 45 min!
Matthew Barr
Sr. Systems Engineer
Is your
infrastructure a
mess?
Let’s fix it :)
What we’re going to do:
• Define a modern infrastructure
• Glance at their architectures
• Demonstrate how to do this yourselves
• … And then the details..
What is a modern infrastructure?
It includes:
• Centralized logging
• Monitoring
• Orchestration
• CI (continuous integration)
• Metrics*
What we’ll do today: Setup
• Mcollective
• Sensu (ideal for cloud infra)
• Logstash + ElasticSearch + Kibana
• Jenkins
MCollective (mco)
• Orchestration
• Uses ActiveMQ or RabbitMQ
• Maintained by Puppet Labs
• http://puppetlabs.com/mcollective
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
• Distributed monitoring system
• Uses RabbitMQ
• has a easy API
• Adding/remove servers without restarting or changing config files
on server
• http://sensuapp.org
Sensu!
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Logstash
http://logstash.net/docs/1.4.1/tutorials/getting-started-with-logstash
Elastic Search & Kibana
• Elasticsearch (http://www.elasticsearch.com) is a “distributed
restful search and analytics tool”
• It’s used as a datastore for Logstash. (it’s not the only one, but
one of the most used.)
• Kibana is a dashboard for use with Elasticsearch & Logstash.
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
What we’re actually doing:
• Show how to use a set of forge modules to build an infrastructure
out.
• using the mbarr/moderninfra as an opinionated profile module
• download the necessary modules using librarian-puppet
We’ll:
• Build a RabbitMQ server + sensu server
• the admin host (has the mco client)
• Build a logstash server
• Build a Jenkins host
Each server will also:
• be sending logs via logstash-forwarder
• run Sensu client checks
• run a mco server
Moderninfra module
A forge module just for you!
• Sets up the basics of each service
• Sets up the requirements correctly to all work together
• Has… opinions.
Install from the forge:
puppet module install mbarr-moderninfra
The code!
---!
moderninfra::rmqserver: 'rabbitmq.aws.mbarr.net'!
moderninfra::mco_password: 'shhhh..its.a.secret.'!
moderninfra::sensu_password: 'whatsupdoc'!
moderninfra::logstash_server: 'logstash.aws.mbarr.net'
Hiera data, to make life easier:
class moderninfra (!
$rmqserver,!
$logstash_server,!
$rmq=false,!
$mco_client=false,!
$mco_server=false,!
$sensu_client=false,!
$sensu_server=false,!
$logstash=false,!
$logstash_forwarder=true,!
$mco_password=undef,!
$sensu_password=undef,!
) {...}
node default {!
if $role == "mco" {!
class {'moderninfra':!
rmq => true,!
mco_client => true,!
sensu_server => true,!
}!
include profiles::sensuchecks !
}!
!
if $role == "puppet" {!
class {'moderninfra':!
mco_server => true,!
sensu_client => true,!
}!
}
if $role == "logstash" {!
class {'moderninfra':!
logstash => true,!
mco_server => true,!
sensu_client => true,!
}!
include profiles::logstash!
}!
!
if $role == "jenkins" {!
class {'moderninfra':!
mco_server => true,!
sensu_client => true,!
}!
include jenkins!
}!
}
Site.pp
RabbitMQ, Sensu & Mcollective
RabbitMQ
• This is the middle ware that is used by both mco & sensu.
• Our module uses the Puppet SSL certs for connections
• Adds a second cert for the host, via the puppet-certificate
module.
Code
class {'moderninfra':!
rmq => true,!
mco_client => true,!
sensu_server => true,!
}!
include profiles::sensuchecks !
}
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
RMQ Note
• To be fair: Sensu isn’t running w/ SSL certs
• I’ve used other self signed certs before without issue
• Looks like there’s a bug that hopefully is actually fixed in Erlang
OTP 17.1
Mcollective
• Using SSL to secure PSK connections between mco & RabbitMQ
• Installs the package, service & puppet agents.
root@rmq-us-east-1b-i-6a9bda41:~# mco package status puppet
!
* [ ============================================================> ] 4 / 4
!
puppet-us-east-1b-i-346b2a1f.ec2.mbarr.net: puppet-purged.
rmq-us-east-1b-i-6a9bda41.ec2.mbarr.net: puppet-3.6.2-1puppetlabs1.
logstash-us-east-1b-i-979adbbc.ec2.mbarr.net: puppet-3.6.2-1puppetlabs1.
jenkins-us-east-1b-i-969adbbd.ec2.mbarr.net: puppet-3.6.2-1puppetlabs1.
!
Summary of Arch:
!
No aggregate summary could be computed
!
Summary of Ensure:
!
3.6.2-1puppetlabs1 = 3
purged = 1
!
!
Finished processing 4 / 4 hosts in 1172.09 ms
Sensu
• Client on all 4 hosts
• Server on RMQ box
• Distributed checks
• Dashboard on 8080
• profiles::sensuchecks installs various checks. (not in module)
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Actually making sensu GO: (on server)
class profiles::sensuchecks {!
sensu::check { 'check_ntp':!
command => 'PATH=$PATH:/usr/lib/nagios/plugins check_ntp_time -H
pool.ntp.org -w 20 -c 40',!
handlers => 'default',!
subscribers => 'general',!
standalone => false,!
custom => { occurrences => 2 },!
}!
sensu::check { 'check_cron':!
command => '/etc/sensu/plugins/check-procs.rb -p cron -C 1 -c 10 -w 10 ',!
handlers => 'default',!
subscribers => 'general',!
interval => 60,!
standalone => false,!
custom => { occurrences => 2 },!
}!
}!
Logstash
• Centralized logging system
• Inputs, Outputs, Filters
• Inputs: syslog, files, redis..
• Outputs:elasticsearch, etc
• Filters: Grok, many others
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Logstash profile
class profiles::logstash {!
!
logstash::configfile { 'basic_config':!
source => 'puppet:///modules/profiles/logstash/basic_config',!
order => 10!
}!
!
include kibana3!
!
}!
Logstash config
input {
lumberjack {
port => 12345
ssl_certificate => "/etc/logstash/ssl/cert.pem"
ssl_key => "/etc/logstash/ssl/key.pem"
type => "lumberjack"
}
}
!
input {
tcp {
port => 5000
type => syslog
}
udp {
port => 5000
type => syslog
}
}
!
output {
elasticsearch { host => localhost }
stdout { codec => rubydebug }
}
Logstash-forwarder
• Data is sent from logs on client to Logstash server via SSL
• Keeps track of log positions and what’s been sent
• Server listens on 12345, for now.
Elasticsearch & Kibana
• This is what Kibana looks like with data from logstash fed into
elasticsearch
• (It’s zoomed a bit, so you can see the good parts.)
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!
Jenkins
Jenkins
• Continuous integration tool
• There is code to set up slaves in the Jenkins module.
• https://forge.puppetlabs.com/rtyler/jenkins
include jenkins
Things this module doesn’t do:
• Build your puppet master
• DNS names for Puppet master, RMQ, Logstash, etc
• Although the cloud formation templates do!
But it might let
you sleep at
night…
Appendix:!
Puppet Master
• Built w/ CloudFormations template
• Sorry, not vagrant. Might be added soon.
• uses cloud-init to provision puppet & code base
• Uses puppet 3.6.2
• Librarian-puppet
Puppet Master
• Set host name & domain
• Install puppet
• rm -rf /etc/puppet
• git clone REPO /etc/puppet
Appendix: !
Librarian-puppet
Librarian Puppet
• Lets you take a Puppetfile, and manage modules & dependencies
• can use forge or git repos
• Takes over your modules directory, though.
• adds to .gitignore & regenerates the directory from the
Puppetfile
• I’ve used a pattern of a second directory (modules-local) to allow
a slow migration & local files to stay in your existing repo
Modules-local pattern
Old:
modulepath = $confdir/modules:$confdir/modules-local
!
3.6+ directory environments: environment.conf
modulepath = modules:modules-local
Puppetfile
forge "https://forgeapi.puppetlabs.com"
!
mod "reidmv/puppet_certificate"
mod "elasticsearch/logstash"
mod "elasticsearch/elasticsearch"
mod "sensu/sensu"
!
mod "rtyler/jenkins"
!
mod "puppetlabs/mcollective"
!
mod "thejandroman/kibana3", "0.0.3"
!
# mod "mbarr/moderninfra",
# :git => "git://github.com/matthewbarr/moderninfra.git"
!
#mod "garethr/graphite"
modules
├── activemq
├── apache
├── apt
├── concat
├── datacat
├── elasticsearch
├── epel
├── erlang
├── file_concat
├── git
├── java
├── java_ks
├── jenkins
├── kibana3
├── logstash
├── mcollective
├── puppet_certificate
├── rabbitmq
├── sensu
├── staging
├── stdlib
├── vcsrepo
└── zypprepo
modules-local
├── moderninfra
└── profiles
We’re hiring! (in Boston)!
!
!
Matthew Barr!
@matthewbarr (github & twitter)!
matthew.barr@here.com!
mbarr@mbarr.net!
http://github.com/matthewbarr/build-modern-infra

Más contenido relacionado

La actualidad más candente

Ansible for beginners ...?
Ansible for beginners ...?Ansible for beginners ...?
Ansible for beginners ...?shirou wakayama
 
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Puppet
 
Managing Puppet using MCollective
Managing Puppet using MCollectiveManaging Puppet using MCollective
Managing Puppet using MCollectivePuppet
 
Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.Łukasz Proszek
 
Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Puppet
 
Getting started with Ansible
Getting started with AnsibleGetting started with Ansible
Getting started with AnsibleIvan Serdyuk
 
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpecMartin Etmajer
 
Spot Trading - A case study in continuous delivery for mission critical finan...
Spot Trading - A case study in continuous delivery for mission critical finan...Spot Trading - A case study in continuous delivery for mission critical finan...
Spot Trading - A case study in continuous delivery for mission critical finan...SaltStack
 
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...Puppet
 
OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013databus.pro
 
Go Faster with Ansible (PHP meetup)
Go Faster with Ansible (PHP meetup)Go Faster with Ansible (PHP meetup)
Go Faster with Ansible (PHP meetup)Richard Donkin
 
Automation with Ansible and Containers
Automation with Ansible and ContainersAutomation with Ansible and Containers
Automation with Ansible and ContainersRodolfo Carvalho
 
Ansible Devops North East - slides
Ansible Devops North East - slides Ansible Devops North East - slides
Ansible Devops North East - slides InfinityPP
 
Monitoring with sensu
Monitoring with sensuMonitoring with sensu
Monitoring with sensumiquelruizm
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet
 
Introduction to Python Celery
Introduction to Python CeleryIntroduction to Python Celery
Introduction to Python CeleryMahendra M
 
Puppet Camp DC 2014: Managing Puppet with MCollective
Puppet Camp DC 2014: Managing Puppet with MCollectivePuppet Camp DC 2014: Managing Puppet with MCollective
Puppet Camp DC 2014: Managing Puppet with MCollectivePuppet
 

La actualidad más candente (20)

Ansible for beginners ...?
Ansible for beginners ...?Ansible for beginners ...?
Ansible for beginners ...?
 
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
Testing for Ops: Going Beyond the Manifest - PuppetConf 2013
 
Managing Puppet using MCollective
Managing Puppet using MCollectiveManaging Puppet using MCollective
Managing Puppet using MCollective
 
Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.Take control of your Jenkins jobs via job DSL.
Take control of your Jenkins jobs via job DSL.
 
Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013Bootstrapping Puppet and Application Deployment - PuppetConf 2013
Bootstrapping Puppet and Application Deployment - PuppetConf 2013
 
Getting started with Ansible
Getting started with AnsibleGetting started with Ansible
Getting started with Ansible
 
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpecTest-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
Test-Driven Infrastructure with Ansible, Test Kitchen, Serverspec and RSpec
 
Spot Trading - A case study in continuous delivery for mission critical finan...
Spot Trading - A case study in continuous delivery for mission critical finan...Spot Trading - A case study in continuous delivery for mission critical finan...
Spot Trading - A case study in continuous delivery for mission critical finan...
 
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...
 
OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013OpenNebula and SaltStack - OpenNebulaConf 2013
OpenNebula and SaltStack - OpenNebulaConf 2013
 
Go Faster with Ansible (PHP meetup)
Go Faster with Ansible (PHP meetup)Go Faster with Ansible (PHP meetup)
Go Faster with Ansible (PHP meetup)
 
Ansible Crash Course
Ansible Crash CourseAnsible Crash Course
Ansible Crash Course
 
Automation with Ansible and Containers
Automation with Ansible and ContainersAutomation with Ansible and Containers
Automation with Ansible and Containers
 
Ansible Devops North East - slides
Ansible Devops North East - slides Ansible Devops North East - slides
Ansible Devops North East - slides
 
Celery with python
Celery with pythonCelery with python
Celery with python
 
Monitoring with sensu
Monitoring with sensuMonitoring with sensu
Monitoring with sensu
 
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
Puppet Camp Düsseldorf 2014: Continuously Deliver Your Puppet Code with Jenki...
 
Introduction to Python Celery
Introduction to Python CeleryIntroduction to Python Celery
Introduction to Python Celery
 
Puppet Camp DC 2014: Managing Puppet with MCollective
Puppet Camp DC 2014: Managing Puppet with MCollectivePuppet Camp DC 2014: Managing Puppet with MCollective
Puppet Camp DC 2014: Managing Puppet with MCollective
 
Puppet - an introduction
Puppet - an introductionPuppet - an introduction
Puppet - an introduction
 

Destacado

SharePoint Branding - 3 Most Common Mistakes
SharePoint Branding - 3 Most Common MistakesSharePoint Branding - 3 Most Common Mistakes
SharePoint Branding - 3 Most Common MistakesNicolePullin
 
Pitch advices - medialab session nantes 2013
Pitch advices - medialab session nantes 2013Pitch advices - medialab session nantes 2013
Pitch advices - medialab session nantes 2013Quentin Adam
 
Dotscale2013 : How to scale ?
Dotscale2013 : How to scale ?Dotscale2013 : How to scale ?
Dotscale2013 : How to scale ?Quentin Adam
 
Understand Immutable infrastructure - at Build Stuff Kiev 2016
Understand Immutable infrastructure  - at Build Stuff Kiev 2016Understand Immutable infrastructure  - at Build Stuff Kiev 2016
Understand Immutable infrastructure - at Build Stuff Kiev 2016Quentin Adam
 
Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...
Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...
Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...Na'Tosha Bard
 
Personal Branding Using Social Media
Personal Branding Using Social MediaPersonal Branding Using Social Media
Personal Branding Using Social MediaBrian Hollowaty
 
Rural branding
Rural brandingRural branding
Rural brandingSunil
 
Personal branding - do it yourself
Personal branding - do it yourselfPersonal branding - do it yourself
Personal branding - do it yourselfShivam Dhawan
 
Definition of Business Infrastructure
Definition of Business InfrastructureDefinition of Business Infrastructure
Definition of Business InfrastructureEquilibria, Inc.
 

Destacado (9)

SharePoint Branding - 3 Most Common Mistakes
SharePoint Branding - 3 Most Common MistakesSharePoint Branding - 3 Most Common Mistakes
SharePoint Branding - 3 Most Common Mistakes
 
Pitch advices - medialab session nantes 2013
Pitch advices - medialab session nantes 2013Pitch advices - medialab session nantes 2013
Pitch advices - medialab session nantes 2013
 
Dotscale2013 : How to scale ?
Dotscale2013 : How to scale ?Dotscale2013 : How to scale ?
Dotscale2013 : How to scale ?
 
Understand Immutable infrastructure - at Build Stuff Kiev 2016
Understand Immutable infrastructure  - at Build Stuff Kiev 2016Understand Immutable infrastructure  - at Build Stuff Kiev 2016
Understand Immutable infrastructure - at Build Stuff Kiev 2016
 
Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...
Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...
Build Infrastructure: What It Is, Why You Need It, and How to Use Python to F...
 
Personal Branding Using Social Media
Personal Branding Using Social MediaPersonal Branding Using Social Media
Personal Branding Using Social Media
 
Rural branding
Rural brandingRural branding
Rural branding
 
Personal branding - do it yourself
Personal branding - do it yourselfPersonal branding - do it yourself
Personal branding - do it yourself
 
Definition of Business Infrastructure
Definition of Business InfrastructureDefinition of Business Infrastructure
Definition of Business Infrastructure
 

Similar a Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!

Toplog candy elves - HOCM Talk
Toplog candy elves - HOCM TalkToplog candy elves - HOCM Talk
Toplog candy elves - HOCM TalkPatrick LaRoche
 
Sensu and Sensibility - Puppetconf 2014
Sensu and Sensibility - Puppetconf 2014Sensu and Sensibility - Puppetconf 2014
Sensu and Sensibility - Puppetconf 2014Tomas Doran
 
Automating Complex Setups with Puppet
Automating Complex Setups with PuppetAutomating Complex Setups with Puppet
Automating Complex Setups with PuppetKris Buytaert
 
Node object and roles - Fundamentals Webinar Series Part 3
Node object and roles - Fundamentals Webinar Series Part 3Node object and roles - Fundamentals Webinar Series Part 3
Node object and roles - Fundamentals Webinar Series Part 3Chef
 
What's new in Docker - InfraKit - Docker Meetup Berlin 2016
What's new in Docker - InfraKit - Docker Meetup Berlin 2016What's new in Docker - InfraKit - Docker Meetup Berlin 2016
What's new in Docker - InfraKit - Docker Meetup Berlin 2016Patrick Chanezon
 
Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow Puppet
 
Steamlining your puppet development workflow
Steamlining your puppet development workflowSteamlining your puppet development workflow
Steamlining your puppet development workflowTomas Doran
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentationSuresh Kumar
 
Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Pavel Chunyayev
 
Using Puppet in Small Infrastructures
Using Puppet in Small InfrastructuresUsing Puppet in Small Infrastructures
Using Puppet in Small InfrastructuresRachel Andrew
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationGiacomo Vacca
 
Introduction to Chef - Techsuperwomen Summit
Introduction to Chef - Techsuperwomen SummitIntroduction to Chef - Techsuperwomen Summit
Introduction to Chef - Techsuperwomen SummitJennifer Davis
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesHiroshi SHIBATA
 
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014Miguel Zuniga
 
Our Puppet Story (Linuxtag 2014)
Our Puppet Story (Linuxtag 2014)Our Puppet Story (Linuxtag 2014)
Our Puppet Story (Linuxtag 2014)DECK36
 
Intro To Node.js
Intro To Node.jsIntro To Node.js
Intro To Node.jsChris Cowan
 
Automated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. AnsibleAutomated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. AnsibleAlberto Molina Coballes
 
Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...
Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...
Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...Nagios
 

Similar a Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min! (20)

Wielding a cortana
Wielding a cortanaWielding a cortana
Wielding a cortana
 
Toplog candy elves - HOCM Talk
Toplog candy elves - HOCM TalkToplog candy elves - HOCM Talk
Toplog candy elves - HOCM Talk
 
Sensu and Sensibility - Puppetconf 2014
Sensu and Sensibility - Puppetconf 2014Sensu and Sensibility - Puppetconf 2014
Sensu and Sensibility - Puppetconf 2014
 
Automating Complex Setups with Puppet
Automating Complex Setups with PuppetAutomating Complex Setups with Puppet
Automating Complex Setups with Puppet
 
Node object and roles - Fundamentals Webinar Series Part 3
Node object and roles - Fundamentals Webinar Series Part 3Node object and roles - Fundamentals Webinar Series Part 3
Node object and roles - Fundamentals Webinar Series Part 3
 
What's new in Docker - InfraKit - Docker Meetup Berlin 2016
What's new in Docker - InfraKit - Docker Meetup Berlin 2016What's new in Docker - InfraKit - Docker Meetup Berlin 2016
What's new in Docker - InfraKit - Docker Meetup Berlin 2016
 
Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow Puppet Camp New York 2014: Streamlining Puppet Development Workflow
Puppet Camp New York 2014: Streamlining Puppet Development Workflow
 
Steamlining your puppet development workflow
Steamlining your puppet development workflowSteamlining your puppet development workflow
Steamlining your puppet development workflow
 
Ansible presentation
Ansible presentationAnsible presentation
Ansible presentation
 
Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015Ansible benelux meetup - Amsterdam 27-5-2015
Ansible benelux meetup - Amsterdam 27-5-2015
 
Using Puppet in Small Infrastructures
Using Puppet in Small InfrastructuresUsing Puppet in Small Infrastructures
Using Puppet in Small Infrastructures
 
Docker and Puppet for Continuous Integration
Docker and Puppet for Continuous IntegrationDocker and Puppet for Continuous Integration
Docker and Puppet for Continuous Integration
 
Introduction to Chef - Techsuperwomen Summit
Introduction to Chef - Techsuperwomen SummitIntroduction to Chef - Techsuperwomen Summit
Introduction to Chef - Techsuperwomen Summit
 
Hosting Ruby Web Apps
Hosting Ruby Web AppsHosting Ruby Web Apps
Hosting Ruby Web Apps
 
Large-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 MinutesLarge-scaled Deploy Over 100 Servers in 3 Minutes
Large-scaled Deploy Over 100 Servers in 3 Minutes
 
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
Configuration Management in the Cloud - Cloud Phoenix Meetup Feb 2014
 
Our Puppet Story (Linuxtag 2014)
Our Puppet Story (Linuxtag 2014)Our Puppet Story (Linuxtag 2014)
Our Puppet Story (Linuxtag 2014)
 
Intro To Node.js
Intro To Node.jsIntro To Node.js
Intro To Node.js
 
Automated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. AnsibleAutomated Deployment and Configuration Engines. Ansible
Automated Deployment and Configuration Engines. Ansible
 
Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...
Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...
Nagios Conference 2014 - Mike Merideth - The Art and Zen of Managing Nagios w...
 

Más de Puppet

Puppet camp2021 testing modules and controlrepo
Puppet camp2021 testing modules and controlrepoPuppet camp2021 testing modules and controlrepo
Puppet camp2021 testing modules and controlrepoPuppet
 
Puppetcamp r10kyaml
Puppetcamp r10kyamlPuppetcamp r10kyaml
Puppetcamp r10kyamlPuppet
 
2021 04-15 operational verification (with notes)
2021 04-15 operational verification (with notes)2021 04-15 operational verification (with notes)
2021 04-15 operational verification (with notes)Puppet
 
Puppet camp vscode
Puppet camp vscodePuppet camp vscode
Puppet camp vscodePuppet
 
Modules of the twenties
Modules of the twentiesModules of the twenties
Modules of the twentiesPuppet
 
Applying Roles and Profiles method to compliance code
Applying Roles and Profiles method to compliance codeApplying Roles and Profiles method to compliance code
Applying Roles and Profiles method to compliance codePuppet
 
KGI compliance as-code approach
KGI compliance as-code approachKGI compliance as-code approach
KGI compliance as-code approachPuppet
 
Enforce compliance policy with model-driven automation
Enforce compliance policy with model-driven automationEnforce compliance policy with model-driven automation
Enforce compliance policy with model-driven automationPuppet
 
Keynote: Puppet camp compliance
Keynote: Puppet camp complianceKeynote: Puppet camp compliance
Keynote: Puppet camp compliancePuppet
 
Automating it management with Puppet + ServiceNow
Automating it management with Puppet + ServiceNowAutomating it management with Puppet + ServiceNow
Automating it management with Puppet + ServiceNowPuppet
 
Puppet: The best way to harden Windows
Puppet: The best way to harden WindowsPuppet: The best way to harden Windows
Puppet: The best way to harden WindowsPuppet
 
Simplified Patch Management with Puppet - Oct. 2020
Simplified Patch Management with Puppet - Oct. 2020Simplified Patch Management with Puppet - Oct. 2020
Simplified Patch Management with Puppet - Oct. 2020Puppet
 
Accelerating azure adoption with puppet
Accelerating azure adoption with puppetAccelerating azure adoption with puppet
Accelerating azure adoption with puppetPuppet
 
Puppet catalog Diff; Raphael Pinson
Puppet catalog Diff; Raphael PinsonPuppet catalog Diff; Raphael Pinson
Puppet catalog Diff; Raphael PinsonPuppet
 
ServiceNow and Puppet- better together, Kevin Reeuwijk
ServiceNow and Puppet- better together, Kevin ReeuwijkServiceNow and Puppet- better together, Kevin Reeuwijk
ServiceNow and Puppet- better together, Kevin ReeuwijkPuppet
 
Take control of your dev ops dumping ground
Take control of your  dev ops dumping groundTake control of your  dev ops dumping ground
Take control of your dev ops dumping groundPuppet
 
100% Puppet Cloud Deployment of Legacy Software
100% Puppet Cloud Deployment of Legacy Software100% Puppet Cloud Deployment of Legacy Software
100% Puppet Cloud Deployment of Legacy SoftwarePuppet
 
Puppet User Group
Puppet User GroupPuppet User Group
Puppet User GroupPuppet
 
Continuous Compliance and DevSecOps
Continuous Compliance and DevSecOpsContinuous Compliance and DevSecOps
Continuous Compliance and DevSecOpsPuppet
 
The Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick Maludy
The Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick MaludyThe Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick Maludy
The Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick MaludyPuppet
 

Más de Puppet (20)

Puppet camp2021 testing modules and controlrepo
Puppet camp2021 testing modules and controlrepoPuppet camp2021 testing modules and controlrepo
Puppet camp2021 testing modules and controlrepo
 
Puppetcamp r10kyaml
Puppetcamp r10kyamlPuppetcamp r10kyaml
Puppetcamp r10kyaml
 
2021 04-15 operational verification (with notes)
2021 04-15 operational verification (with notes)2021 04-15 operational verification (with notes)
2021 04-15 operational verification (with notes)
 
Puppet camp vscode
Puppet camp vscodePuppet camp vscode
Puppet camp vscode
 
Modules of the twenties
Modules of the twentiesModules of the twenties
Modules of the twenties
 
Applying Roles and Profiles method to compliance code
Applying Roles and Profiles method to compliance codeApplying Roles and Profiles method to compliance code
Applying Roles and Profiles method to compliance code
 
KGI compliance as-code approach
KGI compliance as-code approachKGI compliance as-code approach
KGI compliance as-code approach
 
Enforce compliance policy with model-driven automation
Enforce compliance policy with model-driven automationEnforce compliance policy with model-driven automation
Enforce compliance policy with model-driven automation
 
Keynote: Puppet camp compliance
Keynote: Puppet camp complianceKeynote: Puppet camp compliance
Keynote: Puppet camp compliance
 
Automating it management with Puppet + ServiceNow
Automating it management with Puppet + ServiceNowAutomating it management with Puppet + ServiceNow
Automating it management with Puppet + ServiceNow
 
Puppet: The best way to harden Windows
Puppet: The best way to harden WindowsPuppet: The best way to harden Windows
Puppet: The best way to harden Windows
 
Simplified Patch Management with Puppet - Oct. 2020
Simplified Patch Management with Puppet - Oct. 2020Simplified Patch Management with Puppet - Oct. 2020
Simplified Patch Management with Puppet - Oct. 2020
 
Accelerating azure adoption with puppet
Accelerating azure adoption with puppetAccelerating azure adoption with puppet
Accelerating azure adoption with puppet
 
Puppet catalog Diff; Raphael Pinson
Puppet catalog Diff; Raphael PinsonPuppet catalog Diff; Raphael Pinson
Puppet catalog Diff; Raphael Pinson
 
ServiceNow and Puppet- better together, Kevin Reeuwijk
ServiceNow and Puppet- better together, Kevin ReeuwijkServiceNow and Puppet- better together, Kevin Reeuwijk
ServiceNow and Puppet- better together, Kevin Reeuwijk
 
Take control of your dev ops dumping ground
Take control of your  dev ops dumping groundTake control of your  dev ops dumping ground
Take control of your dev ops dumping ground
 
100% Puppet Cloud Deployment of Legacy Software
100% Puppet Cloud Deployment of Legacy Software100% Puppet Cloud Deployment of Legacy Software
100% Puppet Cloud Deployment of Legacy Software
 
Puppet User Group
Puppet User GroupPuppet User Group
Puppet User Group
 
Continuous Compliance and DevSecOps
Continuous Compliance and DevSecOpsContinuous Compliance and DevSecOps
Continuous Compliance and DevSecOps
 
The Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick Maludy
The Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick MaludyThe Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick Maludy
The Dynamic Duo of Puppet and Vault tame SSL Certificates, Nick Maludy
 

Último

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 

Último (20)

UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 

Puppet Camp NYC 2014: Build a Modern Infrastructure in 45 min!

  • 1. Prepared by Build a modern infrastructure in 45 min! Matthew Barr Sr. Systems Engineer
  • 3. What we’re going to do: • Define a modern infrastructure • Glance at their architectures • Demonstrate how to do this yourselves • … And then the details..
  • 4. What is a modern infrastructure?
  • 5. It includes: • Centralized logging • Monitoring • Orchestration • CI (continuous integration) • Metrics*
  • 6. What we’ll do today: Setup • Mcollective • Sensu (ideal for cloud infra) • Logstash + ElasticSearch + Kibana • Jenkins
  • 7. MCollective (mco) • Orchestration • Uses ActiveMQ or RabbitMQ • Maintained by Puppet Labs • http://puppetlabs.com/mcollective
  • 9. • Distributed monitoring system • Uses RabbitMQ • has a easy API • Adding/remove servers without restarting or changing config files on server • http://sensuapp.org Sensu!
  • 12. Elastic Search & Kibana • Elasticsearch (http://www.elasticsearch.com) is a “distributed restful search and analytics tool” • It’s used as a datastore for Logstash. (it’s not the only one, but one of the most used.) • Kibana is a dashboard for use with Elasticsearch & Logstash.
  • 14. What we’re actually doing: • Show how to use a set of forge modules to build an infrastructure out. • using the mbarr/moderninfra as an opinionated profile module • download the necessary modules using librarian-puppet
  • 15. We’ll: • Build a RabbitMQ server + sensu server • the admin host (has the mco client) • Build a logstash server • Build a Jenkins host
  • 16. Each server will also: • be sending logs via logstash-forwarder • run Sensu client checks • run a mco server
  • 18. A forge module just for you! • Sets up the basics of each service • Sets up the requirements correctly to all work together • Has… opinions.
  • 19. Install from the forge: puppet module install mbarr-moderninfra
  • 20. The code! ---! moderninfra::rmqserver: 'rabbitmq.aws.mbarr.net'! moderninfra::mco_password: 'shhhh..its.a.secret.'! moderninfra::sensu_password: 'whatsupdoc'! moderninfra::logstash_server: 'logstash.aws.mbarr.net' Hiera data, to make life easier: class moderninfra (! $rmqserver,! $logstash_server,! $rmq=false,! $mco_client=false,! $mco_server=false,! $sensu_client=false,! $sensu_server=false,! $logstash=false,! $logstash_forwarder=true,! $mco_password=undef,! $sensu_password=undef,! ) {...}
  • 21. node default {! if $role == "mco" {! class {'moderninfra':! rmq => true,! mco_client => true,! sensu_server => true,! }! include profiles::sensuchecks ! }! ! if $role == "puppet" {! class {'moderninfra':! mco_server => true,! sensu_client => true,! }! } if $role == "logstash" {! class {'moderninfra':! logstash => true,! mco_server => true,! sensu_client => true,! }! include profiles::logstash! }! ! if $role == "jenkins" {! class {'moderninfra':! mco_server => true,! sensu_client => true,! }! include jenkins! }! } Site.pp
  • 22. RabbitMQ, Sensu & Mcollective
  • 23. RabbitMQ • This is the middle ware that is used by both mco & sensu. • Our module uses the Puppet SSL certs for connections • Adds a second cert for the host, via the puppet-certificate module.
  • 24. Code class {'moderninfra':! rmq => true,! mco_client => true,! sensu_server => true,! }! include profiles::sensuchecks ! }
  • 26. RMQ Note • To be fair: Sensu isn’t running w/ SSL certs • I’ve used other self signed certs before without issue • Looks like there’s a bug that hopefully is actually fixed in Erlang OTP 17.1
  • 27. Mcollective • Using SSL to secure PSK connections between mco & RabbitMQ • Installs the package, service & puppet agents.
  • 28. root@rmq-us-east-1b-i-6a9bda41:~# mco package status puppet ! * [ ============================================================> ] 4 / 4 ! puppet-us-east-1b-i-346b2a1f.ec2.mbarr.net: puppet-purged. rmq-us-east-1b-i-6a9bda41.ec2.mbarr.net: puppet-3.6.2-1puppetlabs1. logstash-us-east-1b-i-979adbbc.ec2.mbarr.net: puppet-3.6.2-1puppetlabs1. jenkins-us-east-1b-i-969adbbd.ec2.mbarr.net: puppet-3.6.2-1puppetlabs1. ! Summary of Arch: ! No aggregate summary could be computed ! Summary of Ensure: ! 3.6.2-1puppetlabs1 = 3 purged = 1 ! ! Finished processing 4 / 4 hosts in 1172.09 ms
  • 29. Sensu • Client on all 4 hosts • Server on RMQ box • Distributed checks • Dashboard on 8080 • profiles::sensuchecks installs various checks. (not in module)
  • 33. Actually making sensu GO: (on server) class profiles::sensuchecks {! sensu::check { 'check_ntp':! command => 'PATH=$PATH:/usr/lib/nagios/plugins check_ntp_time -H pool.ntp.org -w 20 -c 40',! handlers => 'default',! subscribers => 'general',! standalone => false,! custom => { occurrences => 2 },! }! sensu::check { 'check_cron':! command => '/etc/sensu/plugins/check-procs.rb -p cron -C 1 -c 10 -w 10 ',! handlers => 'default',! subscribers => 'general',! interval => 60,! standalone => false,! custom => { occurrences => 2 },! }! }!
  • 35. • Centralized logging system • Inputs, Outputs, Filters • Inputs: syslog, files, redis.. • Outputs:elasticsearch, etc • Filters: Grok, many others
  • 37. Logstash profile class profiles::logstash {! ! logstash::configfile { 'basic_config':! source => 'puppet:///modules/profiles/logstash/basic_config',! order => 10! }! ! include kibana3! ! }!
  • 38. Logstash config input { lumberjack { port => 12345 ssl_certificate => "/etc/logstash/ssl/cert.pem" ssl_key => "/etc/logstash/ssl/key.pem" type => "lumberjack" } } ! input { tcp { port => 5000 type => syslog } udp { port => 5000 type => syslog } } ! output { elasticsearch { host => localhost } stdout { codec => rubydebug } }
  • 39. Logstash-forwarder • Data is sent from logs on client to Logstash server via SSL • Keeps track of log positions and what’s been sent • Server listens on 12345, for now.
  • 40. Elasticsearch & Kibana • This is what Kibana looks like with data from logstash fed into elasticsearch • (It’s zoomed a bit, so you can see the good parts.)
  • 44. Jenkins • Continuous integration tool • There is code to set up slaves in the Jenkins module. • https://forge.puppetlabs.com/rtyler/jenkins
  • 46. Things this module doesn’t do: • Build your puppet master • DNS names for Puppet master, RMQ, Logstash, etc • Although the cloud formation templates do!
  • 47. But it might let you sleep at night…
  • 49. • Built w/ CloudFormations template • Sorry, not vagrant. Might be added soon. • uses cloud-init to provision puppet & code base • Uses puppet 3.6.2 • Librarian-puppet
  • 50. Puppet Master • Set host name & domain • Install puppet • rm -rf /etc/puppet • git clone REPO /etc/puppet
  • 52. Librarian Puppet • Lets you take a Puppetfile, and manage modules & dependencies • can use forge or git repos • Takes over your modules directory, though. • adds to .gitignore & regenerates the directory from the Puppetfile • I’ve used a pattern of a second directory (modules-local) to allow a slow migration & local files to stay in your existing repo
  • 53. Modules-local pattern Old: modulepath = $confdir/modules:$confdir/modules-local ! 3.6+ directory environments: environment.conf modulepath = modules:modules-local
  • 54. Puppetfile forge "https://forgeapi.puppetlabs.com" ! mod "reidmv/puppet_certificate" mod "elasticsearch/logstash" mod "elasticsearch/elasticsearch" mod "sensu/sensu" ! mod "rtyler/jenkins" ! mod "puppetlabs/mcollective" ! mod "thejandroman/kibana3", "0.0.3" ! # mod "mbarr/moderninfra", # :git => "git://github.com/matthewbarr/moderninfra.git" ! #mod "garethr/graphite"
  • 55. modules ├── activemq ├── apache ├── apt ├── concat ├── datacat ├── elasticsearch ├── epel ├── erlang ├── file_concat ├── git ├── java ├── java_ks ├── jenkins ├── kibana3 ├── logstash ├── mcollective ├── puppet_certificate ├── rabbitmq ├── sensu ├── staging ├── stdlib ├── vcsrepo └── zypprepo modules-local ├── moderninfra └── profiles
  • 56. We’re hiring! (in Boston)! ! ! Matthew Barr! @matthewbarr (github & twitter)! matthew.barr@here.com! mbarr@mbarr.net! http://github.com/matthewbarr/build-modern-infra