Key takeaways:
How Bright Cluster Manager allows you to monitor HPC and Hadoop clusters
How Bright Cluster Manager allows you to monitor public and private clouds
How Bright Cluster Manager enables monitoring with alerts and health checks
How Bright Cluster Manager enables customized monitoring - including how to incorporate your own monitors
This recording (http://hubs.ly/y0JtjX0) includes live-product demonstrations of Bright Cluster Manager.
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
Bright Topics Webinar April 15, 2015 - Modernized Monitoring for Cluster and Clouds of All Types
1. Modernized Monitoring for Clusters
and Clouds of All Types
Ian Lumb
Product Marketing Manager
Bright Topics Webinar
April 15, 2015
RECORDING
2. 2
Modernized monitoring
Monitoring HPC and Hadoop clusters
Monitoring public and private clouds
Monitoring with alerts and health checks
Customized monitoring - including how to incorporate
your own monitors
Key Takeaways
RECORDING
3. The Five Essential Strategies
1. Plan to manage the impact of software complexity
2. Plan for scalable growth
3. Plan to manage heterogeneous hardware/software
solutions
4. Be ready for the Cloud
5. Have an answer for the Hadoop question
http://insidehpc.com/2014/05/five-essential-strategies-successful-hpc-clusters/
5. 5
The problem with “Toolkits”
Toolkits — “A patchwork of disparate tools”
• Tools typically used: Ganglia, Nagios, Cfengine, System Imager,
Puppet, Chef, Cobbler, Hobbit, Big Brother, Zabbix, etc.
• Scripts
Issues with the “toolkit” approach:
• Scripts poorly documented and hard to maintain
• Tools not designed to work together
• Each tool has its own user interface (CLI/GUI)
• Each tool has its own agent and database
Hidden assumptions and biases re: sampling and more
• Tools rarely designed for scale & high performance
• Accelerators and coprocessors often not supported
Making a collection of unrelated tools work together
• Requires a lot of expertise and scripting
• Rarely leads to a really easy-to-use and scalable solution
6. 6
The problem with “Meta-Toolkits”
Meta-Toolkits likely to obfuscate
• Assumptions and biases involved in sampling and processing
Was interpolation or extrapolation required?
• Scalability limitations
• Existing capabilities within a specific toolkit
User beware the LCD effect!
• The ongoing burden of management and maintenance
http://insidehpc.com/2014/11/monitoring-hpc-clusters-modernized/
7. 7
Pressing concerns, real implications
Significant toolkit legacy in HPC
• Use of meta-toolkits escalating
Hadoop deployments rediscovering
toolkit legacy
• Hadoop monitoring +
{ NAGIOS || Ganglia || ??? }
• Apache Ambari an evolving meta-toolkit
‘Modernized’ monitoring with meta-toolkits?
http://www.hpcwire.com/2014/09/18/modernizing-hpc-cluster-monitoring/
"Those who cannot
remember the past
are condemned to
repeat it“
George Santayana
The Life of Reason, Vol. 1
1905
Hadoop Users: Stop Settling for the Santayana Effect TODAY!
https://www.linkedin.com/pulse/hadoop-users-stop-settling-santayana-effect-today-ian-lumb
11. 11
Pressing concerns, real implications
Significant toolkit legacy in HPC
• Use of meta-toolkits escalating
Hadoop deployments rediscovering
toolkit legacy
• Hadoop monitoring +
{ NAGIOS || Ganglia || ??? }
• Apache Ambari an evolving meta-toolkit
OpenStack on track to also
rediscover the toolkit legacy
‘Modernized’ monitoring with meta-toolkits?
http://www.hpcwire.com/2014/09/18/modernizing-hpc-cluster-monitoring/
"Those who cannot
remember the past
are condemned to
repeat it“
George Santayana
The Life of Reason, Vol. 1
1905
Hadoop Users: Stop Settling for the Santayana Effect TODAY!
https://www.linkedin.com/pulse/hadoop-users-stop-settling-santayana-effect-today-ian-lumb
13. 13
Pressing concerns, real implications
Significant toolkit legacy in HPC
• Use of meta-toolkits escalating
Hadoop deployments rediscovering
toolkit legacy
• Hadoop monitoring +
{ NAGIOS || Ganglia || ??? }
• Apache Ambari an evolving meta-toolkit
OpenStack on track to also
rediscover the toolkit legacy
‘Modernized’ monitoring with meta-toolkits?
http://www.hpcwire.com/2014/09/18/modernizing-hpc-cluster-monitoring/
"Those who cannot
remember the past
are condemned to
repeat it“
George Santayana
The Life of Reason, Vol. 1
1905
Hadoop Users: Stop Settling for the Santayana Effect TODAY!
https://www.linkedin.com/pulse/hadoop-users-stop-settling-santayana-effect-today-ian-lumb
14. 14
A Better Solution
Bright Cluster Manager takes a much more fundamental
& integrated approach
• Designed and written from the ground up
• Single cluster management agent provides all functionality
• Single, central database for configuration and monitoring data
• Single UI for ALL cluster management functionality
Which makes Bright Cluster Manager …
• Extremely easy to use
• Extremely scalable
• Secure & reliable
• Complete
• Flexible
• Maintainable
15. Bright Cluster
Architecture — Monitoring
CMDaemon
head node
node001
node003
node002
data
Cluster
Management
GUI
Cluster
Management
Shell
Web-Based
User Portal
Third-Party
Applications
BMC
BMC
BMCraw data consolidated
data
metrics
metrics
metrics
metrics
metrics
16. 16
Native Metrics for Clusters & Clouds
Over 160 relating to HPC
• From bare metal to workload managers to apps
Includes accelerators and coprocessors
• On-the-ground and in-the-public-cloud
Over 400 relating to Hadoop
• From distros, HDFS & YARN to data-platform apps
Almost 90 relating to OpenStack
• Tenant-specific plus private cloud as-a-whole
Over 60 relating to Ceph
http://www.brightcomputing.com/Linux-Cluster-Monitoring
17.
18.
19. 19
Monitoring++
Proactive alert-based monitoring
• Define thresholds for any metric
• Associate actions with thresholds
Actions execute when thresholds exceeded
Health checks
• Invasive plus dynamic diagnostics
Cluster monitoring vs. health checking: What’s the difference?
http://info.brightcomputing.com/blog/cluster-monitoring-vs.-health-checking-whats-the-difference
http://www.brightcomputing.com/Linux-Cluster-Health
23. 23
Modernized monitoring
Monitoring HPC and Hadoop clusters
Monitoring public and private clouds
Monitoring with alerts and health checks
Customized monitoring - including how to incorporate
your own monitors
Key Takeaways
RECORDING
24. Q & A
Ian Lumb, ian.lumb@brightcomputing.com
http://www.brightcomputing.com/