This document provides an overview of Apache Ambari, an open source framework for provisioning, managing and monitoring Hadoop clusters. It discusses Ambari's architecture and features for provisioning clusters, managing services, monitoring metrics and alerts, and extensibility through Ambari stacks, views and blueprints. The document also outlines Ambari's release cadence and upcoming features around operations, extensibility and troubleshooting insights.

1. © Hortonworks Inc. 2014
Apache Ambari
Provisioning, Managing and Monitoring Hadoop

2. © Hortonworks Inc. 2014
Agenda
• Ambari Overview and the Community
• Ambari Architecture
• Provisioning Clusters and Services
• Extensibility Features
– Ambari Stacks
– Ambari Views
– Ambari Blueprints
• Releases
• Learn More
• Q & A
Page 2

3. © Hortonworks Inc. 2014
Hadoop Operations with Ambari
Provision: Simplified deployment across platforms
Manage: Consistent controls across the Stack
Monitor: Visibility into key cluster metrics
- Single pane of glass for Hadoop & System status
- Pre-configured metrics & alerts
- Single point for cluster operations
- Advanced configurations and host controls
- Wizard-driven cluster install experience
- Cloud, virtual and physical environments
Apache Ambari is a 100% open source framework for
provisioning, managing and monitoring Hadoop clusters

4. © Hortonworks Inc. 2014
100% Apache Open Source
• Active Community
• 60+ Contributors / 30+ Committers
• 200+ Ambari User Group Members
Page 4
2013
Oct
Apache Ambari 1.4.1 Released
Adds operations for Hadoop 2 Stack
Ambari Ambari 1.2.0 Released
Operations for Hadoop 1 Stack
2013
Jan
2013
Dec
Apache Ambari Graduates to Top Level Project

5. © Hortonworks Inc. 2014
Ambari Release Cadence
Page 5
Ambari 1.4.3
Jan 2014
Ambari 1.4.4
Feb 2014
Ambari 1.5.0
Apr 2014
Ambari 1.7.0
2H2014
Ambari 1.6.0
May 2014

6. © Hortonworks Inc. 2014
Demo #1
Cluster Management
Page 6

7. © Hortonworks Inc. 2014
System Architecture
7
Ambari Server
Host
Agent
Host
Agent
Host
Agent
Ambari
Web
DB
/clusters!
Host
Agent

8. © Hortonworks Inc. 2014
Ambari Server Architecture
8
DB
Orchestrator Monitoring
REST API
Request Dispatcher
Ambari
Web
Ambari
Server
Ambari
Agents
Metrics
Providers
AuthProvider
/clusters!
User!
Repo!
java!
python!
JS!
RDBMS!
LDAP!
REST API
Configurable
Auth Provider
Cluster
Configurations
and Topology
Web Client
AD!

9. © Hortonworks Inc. 2014
Installing Ambari
• Install Ambari Server
• Setup Options
– JDK (Oracle JDK 6, Oracle JDK 7, Custom JDK)
– Database (PostgreSQL, MySQL or Oracle)
– Server Port and HTTPS
– User Authentication (Local, LDAP or AD)
– Server-Agent Two-way SSL
• Start Ambari Server
• Browse to Ambari Web for Cluster Install Wizard
• Install Ambari Agents
– Option #1: Automate via SSH
– Option #2: Manual
Page 9

10. © Hortonworks Inc. 2014
Installing Agents
Page 10
OPTION 1!
automate!
provide SSH Key!
OPTION 2!
manual!

11. © Hortonworks Inc. 2014
Option #1: Ambari Agent via SSH
Page 11
Ambari Server Ambari Agent
1) Configure Ambari Repo!
2) Copy Ambari Agent Setup script!
3) Execute Ambari Agent Setup script!
4) Install ambari-agent!
5) Configure ambari-agent.ini!
6) Start ambari-agent!
7) Begin Ambari Agent Registration!
Uses
the SSH

12. © Hortonworks Inc. 2014
Option #2: Ambari Agent via Manual
Page 12
Ambari Server Ambari Agent
2) Install ambari-agent!
3) Configure ambari-agent.ini!
4) Start ambari-agent!
5) Begin Ambari Agent Registration!
1) Configure Ambari repo!
** User executes all ops on the Agent host(s). No requirement for SSH Private Key.
No SSH

13. © Hortonworks Inc. 2014
Ambari Agent Registration
Page 13
Ambari Server Ambari Agent
11) Agent Heartbeat Begins!
1) Connect on Handshake port 8441!
4) Sign Agent Cert!
2) Download Server Cert!
6) Connect on Registration port 8440!
7) Perform 2WAY auth using Agent Cert!
10) Complete Host Registration!
Agent Host
8) Get FQDN!
9) Register host!
8a)
Host
Script!
3) Request to Sign Agent Cert!
5) Download Agent Cert + Disconnect!

14. © Hortonworks Inc. 2014
Demo #2
Cluster Install
Page 14

15. © Hortonworks Inc. 2014
Extensibility Features
Ambari Stacks
Ambari Blueprints
Ambari Views
Page 15

16. © Hortonworks Inc. 2014
Ambari Stacks
Page 16

17. © Hortonworks Inc. 2014
Ambari Stacks
• Separate “the Stack” from “Ambari”
• Make it easier to bring more than a traditional
“Hadoop Stack” under Ambari management
• Defining a consistent Service lifecycle management
interface that can be extended
• Dynamically add Stacks + Services definitions
Page 17

18. © Hortonworks Inc. 2014
Stack Details
• Stacks define Services + Repos
– What is in the Stack, and where to get the bits
• Each Service has a definition
– What Components are part of the Service
• Each Service has defined lifecycle commands
– start, stop, status, install, configure
• Lifecycle is controlled via command scripts
• Ability to define “custom” commands
• Ability to “extend” Stacks
Page 18
AMBARI
SERVER
Stack
Command
Scripts
Service
Defini:ons
AMBARI
AGENT/S
AMBARI
AGENT/S
AMBARI
AGENT/S
python!xml!
Repos

19. © Hortonworks Inc. 2014
Stack Mechanics
• Ambari Server reads Stack definitions on start
• Ambari Server sends a command to Agents
• Agents download Stack definition + command scripts
• Agent executes command
• If the Stack definition changes, Agent will request
latest Stack definition + command scripts
Page 19

20. © Hortonworks Inc. 2014
Ambari Blueprints
Page 20

21. © Hortonworks Inc. 2014
Ambari Blueprints
Page 21
STACK
HOSTS
BLUEPRINT

22. © Hortonworks Inc. 2014
Automating Installs
• Blueprint defines a cluster layout and
component configuration for a Stack
• Provide Blueprint to Ambari to perform
cluster installation (no wizard required)
• Export Blueprint from existing cluster
BLUEPRINT!
Ambari
Submit Blueprint
to Ambari Hadoop
Cluster
Ambari provisions
cluster
BLUEPRINT!
<stack>!
<host>!
<service>!
<component>!
<config>!
HOST!
MANIFEST!
<host>!
<meta>!
SERVICE!
CONFIGS!
<props>!
Page 22

23. © Hortonworks Inc. 2014
Ambari + OpenStack
Page 23
OpenStack
Infrastructure
Horizon
Savanna
AMBARI
SERVER
Hadoop
Cluster

24. © Hortonworks Inc. 2014
Ambari Views
Page 24

25. © Hortonworks Inc. 2014
Ambari Views
• Goal: Customize the Ambari Web experience
• Views compliment Stack Extensibility
– Stack Extensibility makes custom Stack Services available to Ambari
– Views expose custom UI features for Services
• Ambari Admins can entitle “views” to Ambari Web users
– Entitlements framework for finer-grained permissions control for Ambari users
Page 25

26. © Hortonworks Inc. 2014
View Usage
• Developer Creates a View
• Deploys View to Ambari Server
• Ambari Admin creates instances of Views
• Ambari Admin entitles Views to users
Page 26

27. © Hortonworks Inc. 2014
View Configuration + Entitlement
Page 27

28. © Hortonworks Inc. 2014
Customized Views
Page 28
“admin” gets operator controls “joe.user” gets specific views
based on entitlements

29. © Hortonworks Inc. 2014
Releases
What’s Up Next
Page 29

30. © Hortonworks Inc. 2014
What’s New in Ambari 1.5.0
• Maintenance Mode
• Rolling Restarts
• Bulk Host Operations
• Decommission TTs, NMs, RSs
• Add Service
• Refresh Client Configs
• Default JDK 7
• For a complete list of changes:
https://issues.apache.org/jira/browse/AMBARI/fixforversion/12325019
Page 30

31. © Hortonworks Inc. 2014
Ambari Futures
• Operations
– Ambari Blueprints
– Component log access + search
– Ubuntu and Windows support
• Extensibility
– Finer-grain user entitlements
– Ambari Views
• Insight
– Troubleshooting + Performance Tuning
– Capacity Planning
Page 31

32. © Hortonworks Inc. 2014
Learn More
Page 32
Resource Location
Apache Ambari Project Page http://ambari.apache.org
Ambari Project Wiki https://cwiki.apache.org/confluence/display/AMBARI
Ambari Project JIRA https://issues.apache.org/jira/browse/AMBARI

33. © Hortonworks Inc. 2014
Q & A
Page 33