SlideShare una empresa de Scribd logo
1 de 23
Installation
  7/6/2012

© 2012 MapR Technologies   Installation 1
Installation
   Agenda
   • Installation Overview
   • Installation Methods
   • Installation Details
   • High Availability Considerations
   • LAB: Installation




© 2012 MapR Technologies       Installation 2
Installation
   Objectives
   At the end of this module you will be able to:
   • Describe the steps involved in MapR installation
   • Install a cluster using the maprinstall script
   • Install MapR on a single node manually and with the maprinstall script
   • Install MapR Metrics
   • Explain what is involved in setting up HA for MapR




© 2012 MapR Technologies            Installation 3
Installation Overview




© 2012 MapR Technologies     Installation 4
Installation Overview

     What needs to happen to install MapR on each node ?
      –   Add MapR repository
      –   Install MapR services and packages from repository
      –   Configure the node (configure.sh)
      –   Format disks (disksetup)
      –   Install and configure MapR Metrics (optional)
      –   Start the cluster
      –   Additional configuration steps after install
          • Set up HA
          • Set up node topology (covered in Configuration module)




© 2012 MapR Technologies               Installation 5
Installation Methods




© 2012 MapR Technologies     Installation 6
Installation Methods

     Cluster installation
      –   Bring up an entire cluster with a script from one node
          • Download the maprinstall script and move to /opt/install-mapr
          •   Run/opt/install-mapr/maprinstall –R <roles_file>
              – Performs node validation (OS partition, ports, memory, swap space, etc.)
              – Assumes ability to SSH from node running script to other nodes
              – Provide a roles file with comma-delimited lists of:
                  »   Nodes
                  »   Packages
                  »   Disks
                  10.10.100.2 mapr-fileserver,mapr-tasktracker /dev/sdb
                  10.10.100.3 mapr-fileserver,mapr-tasktracker /dev/sdb
                  10.10.100.1 mapr-cldb,mapr-zookeeper,mapr-jobtracker,mapr-
                  webserver,mapr-fileserver,mapr-tasktracker /dev/sdb




© 2012 MapR Technologies                     Installation 7
Installation Methods

     Individual node installation
      –   Scripted
          • Download the maprinstall script and move to /opt/install-mapr
          •   Run/opt/install-mapr/maprinstall
              – Assumes you can provide mandatory data (package list, CLDB, Zookeeper)
              – Performs node validation (OS partition, ports, memory, swap space, etc.)
              – Executes all steps below from one script

      –   Manual
          •   Add MapR repository
          •   Install MapR services (apt-get, yum, etc.)
          •   configure.sh
          •   disksetup
          •   Bring up the cluster



© 2012 MapR Technologies                     Installation 8
Installation Details




© 2012 MapR Technologies         Installation 9
MapR Repository

     From MapR repositories
      –   apt-get or yum
     From local repositories
      –   Download .tar.gz
      –   Set up repository
      –   apt-get or yum
     From local packages
      –   Use dpkg or rpm
      –   Must resolve dependencies manually




© 2012 MapR Technologies           Installation 10
Configuration Script

     configure.sh
      –   Run on each node
      –   Sets up mapr-clusters.conf
      –   Writes ZK node IPs several places (mapr-cldb.conf is one)
      –   Registers node with ZooKeeper and CLDB




© 2012 MapR Technologies           Installation 11
Choosing Disks for MapR

     MapR formats and uses disks for MapR-FS
      –   Should use physical drives/partitions
     Disk ready for MapR?
      –   mount: partition in use?
      –   df: partition in use?
      –   Entry in /etc/fstab?
      –   Not accessible to mkfs? mkfs fails?
     If any above are true, disk is not ready




© 2012 MapR Technologies             Installation 12
Disk Setup Script

     disksetup
      –   Takes input from a text file listing usable disks. E.g.
          /dev/sdb
          /dev/sdc1 /dev/sdc2 /dev/sdc4
          /dev/sdd
      –   Adds disks to disktab
      –   Use –F option to force formatting
      –   Creates storage pools (-w option specifies stripe width)
      –   Run after configure.sh




© 2012 MapR Technologies               Installation 13
Installing MapR Metrics
     Prerequisites
      –   MySQL server (on a node or separate machine)
      –   EPEL repository
      –   M5 license for all features (M3: no charts or histograms)


     Installation procedure:
      –   Install MapR Metrics package on all JobTracker and Webserver nodes
          •   e.g. apt-get install mapr-metrics
      –   MCS: System Settings > Metrics
          • Enter <hostname/IP>:<port> of MySQL server
          • Enter MySQL credentials
      –   Source /opt/mapr/bin/setup.sql at mysql> prompt on a node
          where MapR Metrics is installed


© 2012 MapR Technologies              Installation 14
Starting the Cluster

     Bring up ZooKeeper on ZooKeeper nodes
     Bring up Warden
      –   Wait for services to start (use jps to see)
     Set up administrative user
     Log onto MapR Control System
     Add license




© 2012 MapR Technologies              Installation 15
High Availability
                           Considerations



© 2012 MapR Technologies       Installation 16
High Availability (HA)

     CLDB
     JobTracker
     NFS




© 2012 MapR Technologies   Installation 17
High Availability CLDB

     Warm standby
      –   One Primary CLDB active at a time
      –   Secondary CLDB supports reads
     Configure multiple CLDBs
      –   Up to 6
     Install across physical topology
     Can isolate from all other services
     Can also isolate CLDB volume on its own topology
     Don’t run active JobTracker on primary CLDB node




© 2012 MapR Technologies            Installation 18
High Availability JobTracker

     Cold standby
      –   One active JobTracker at a time
      –   Stateful failover preserves running jobs
     In most cases 3
      –   Unless > 100 nodes, then 5




© 2012 MapR Technologies               Installation 19
High Availability NFS

     Active standby
      –   Multiple concurrent NFS servers allowed
      –   VIPs provide immediate failover
      –   Failover triggered by CLDB if NFS server fails
      –   Reads/writes continue after failover




© 2012 MapR Technologies              Installation 20
Setting Up NFS HA

     Install and run mapr-nfs service on assigned nodes
     Create a pool of VIPs
     Use MapR Control System or the API to assign VIPs to NFS nodes
      –   One VIP across multiple NFS nodes
      –   Multiple VIPs across multiple NFS nodes




© 2012 MapR Technologies            Installation 21
LAB: Installation




© 2012 MapR Technologies       Installation 22
Questions




© 2012 MapR Technologies   Installation 23

Más contenido relacionado

La actualidad más candente

Gluster fs tutorial part 2 gluster and big data- gluster for devs and sys ...
Gluster fs tutorial   part 2  gluster and big data- gluster for devs and sys ...Gluster fs tutorial   part 2  gluster and big data- gluster for devs and sys ...
Gluster fs tutorial part 2 gluster and big data- gluster for devs and sys ...Tommy Lee
 
Docker and coreos20141020b
Docker and coreos20141020bDocker and coreos20141020b
Docker and coreos20141020bRichard Kuo
 
Divide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloudDivide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloudStephen Gordon
 
Ceph Day Beijing: Big Data Analytics on Ceph Object Store
Ceph Day Beijing: Big Data Analytics on Ceph Object Store Ceph Day Beijing: Big Data Analytics on Ceph Object Store
Ceph Day Beijing: Big Data Analytics on Ceph Object Store Ceph Community
 
Gluster Storage
Gluster StorageGluster Storage
Gluster StorageRaz Tamir
 
SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)Lars Marowsky-Brée
 
Ceph and Mirantis OpenStack
Ceph and Mirantis OpenStackCeph and Mirantis OpenStack
Ceph and Mirantis OpenStackMirantis
 
Quick-and-Easy Deployment of a Ceph Storage Cluster with SLES
Quick-and-Easy Deployment of a Ceph Storage Cluster with SLESQuick-and-Easy Deployment of a Ceph Storage Cluster with SLES
Quick-and-Easy Deployment of a Ceph Storage Cluster with SLESJan Kalcic
 
Your 1st Ceph cluster
Your 1st Ceph clusterYour 1st Ceph cluster
Your 1st Ceph clusterMirantis
 
Linux High Availability Overview - openSUSE.Asia Summit 2015
Linux High Availability Overview - openSUSE.Asia Summit 2015 Linux High Availability Overview - openSUSE.Asia Summit 2015
Linux High Availability Overview - openSUSE.Asia Summit 2015 Roger Zhou 周志强
 
Tutorial ceph-2
Tutorial ceph-2Tutorial ceph-2
Tutorial ceph-2Tommy Lee
 
Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt Ceph Community
 
Build an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on CephBuild an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on CephRongze Zhu
 
Storing VMs with Cinder and Ceph RBD.pdf
Storing VMs with Cinder and Ceph RBD.pdfStoring VMs with Cinder and Ceph RBD.pdf
Storing VMs with Cinder and Ceph RBD.pdfOpenStack Foundation
 
Stig Telfer - OpenStack and the Software-Defined SuperComputer
Stig Telfer - OpenStack and the Software-Defined SuperComputerStig Telfer - OpenStack and the Software-Defined SuperComputer
Stig Telfer - OpenStack and the Software-Defined SuperComputerDanny Abukalam
 
How to Increase Performance of Your Hadoop Cluster
How to Increase Performance of Your Hadoop ClusterHow to Increase Performance of Your Hadoop Cluster
How to Increase Performance of Your Hadoop ClusterAltoros
 
Rhel cluster basics 3
Rhel cluster basics   3Rhel cluster basics   3
Rhel cluster basics 3Manoj Singh
 

La actualidad más candente (20)

Gluster fs tutorial part 2 gluster and big data- gluster for devs and sys ...
Gluster fs tutorial   part 2  gluster and big data- gluster for devs and sys ...Gluster fs tutorial   part 2  gluster and big data- gluster for devs and sys ...
Gluster fs tutorial part 2 gluster and big data- gluster for devs and sys ...
 
52 nfs
52 nfs52 nfs
52 nfs
 
Docker and coreos20141020b
Docker and coreos20141020bDocker and coreos20141020b
Docker and coreos20141020b
 
Divide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloudDivide and conquer: resource segregation in the OpenStack cloud
Divide and conquer: resource segregation in the OpenStack cloud
 
Ceph on arm64 upload
Ceph on arm64   uploadCeph on arm64   upload
Ceph on arm64 upload
 
Ceph Day Beijing: Big Data Analytics on Ceph Object Store
Ceph Day Beijing: Big Data Analytics on Ceph Object Store Ceph Day Beijing: Big Data Analytics on Ceph Object Store
Ceph Day Beijing: Big Data Analytics on Ceph Object Store
 
Gluster Storage
Gluster StorageGluster Storage
Gluster Storage
 
SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)SUSE Storage: Sizing and Performance (Ceph)
SUSE Storage: Sizing and Performance (Ceph)
 
ceph-barcelona-v-1.2
ceph-barcelona-v-1.2ceph-barcelona-v-1.2
ceph-barcelona-v-1.2
 
Ceph and Mirantis OpenStack
Ceph and Mirantis OpenStackCeph and Mirantis OpenStack
Ceph and Mirantis OpenStack
 
Quick-and-Easy Deployment of a Ceph Storage Cluster with SLES
Quick-and-Easy Deployment of a Ceph Storage Cluster with SLESQuick-and-Easy Deployment of a Ceph Storage Cluster with SLES
Quick-and-Easy Deployment of a Ceph Storage Cluster with SLES
 
Your 1st Ceph cluster
Your 1st Ceph clusterYour 1st Ceph cluster
Your 1st Ceph cluster
 
Linux High Availability Overview - openSUSE.Asia Summit 2015
Linux High Availability Overview - openSUSE.Asia Summit 2015 Linux High Availability Overview - openSUSE.Asia Summit 2015
Linux High Availability Overview - openSUSE.Asia Summit 2015
 
Tutorial ceph-2
Tutorial ceph-2Tutorial ceph-2
Tutorial ceph-2
 
Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt Scaling Ceph at CERN - Ceph Day Frankfurt
Scaling Ceph at CERN - Ceph Day Frankfurt
 
Build an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on CephBuild an High-Performance and High-Durable Block Storage Service Based on Ceph
Build an High-Performance and High-Durable Block Storage Service Based on Ceph
 
Storing VMs with Cinder and Ceph RBD.pdf
Storing VMs with Cinder and Ceph RBD.pdfStoring VMs with Cinder and Ceph RBD.pdf
Storing VMs with Cinder and Ceph RBD.pdf
 
Stig Telfer - OpenStack and the Software-Defined SuperComputer
Stig Telfer - OpenStack and the Software-Defined SuperComputerStig Telfer - OpenStack and the Software-Defined SuperComputer
Stig Telfer - OpenStack and the Software-Defined SuperComputer
 
How to Increase Performance of Your Hadoop Cluster
How to Increase Performance of Your Hadoop ClusterHow to Increase Performance of Your Hadoop Cluster
How to Increase Performance of Your Hadoop Cluster
 
Rhel cluster basics 3
Rhel cluster basics   3Rhel cluster basics   3
Rhel cluster basics 3
 

Similar a 20a installation

Big Data in Container; Hadoop Spark in Docker and Mesos
Big Data in Container; Hadoop Spark in Docker and MesosBig Data in Container; Hadoop Spark in Docker and Mesos
Big Data in Container; Hadoop Spark in Docker and MesosHeiko Loewe
 
How to configure the cluster based on Multi-site (WAN) configuration
How to configure the clusterbased on Multi-site (WAN) configurationHow to configure the clusterbased on Multi-site (WAN) configuration
How to configure the cluster based on Multi-site (WAN) configurationAkihiro Kitada
 
Deployment and Management of Hadoop Clusters
Deployment and Management of Hadoop ClustersDeployment and Management of Hadoop Clusters
Deployment and Management of Hadoop ClustersAmal G Jose
 
3 map r installation & setup administration course description
3 map r installation & setup administration course description3 map r installation & setup administration course description
3 map r installation & setup administration course descriptionmapr-academy
 
30a accessing your cluster
30a accessing your cluster30a accessing your cluster
30a accessing your clustermapr-academy
 
VDCF Overview
VDCF OverviewVDCF Overview
VDCF OverviewJomaSoft
 
4. v sphere big data extensions hadoop
4. v sphere big data extensions   hadoop4. v sphere big data extensions   hadoop
4. v sphere big data extensions hadoopChiou-Nan Chen
 
Drill into Drill – How Providing Flexibility and Performance is Possible
Drill into Drill – How Providing Flexibility and Performance is PossibleDrill into Drill – How Providing Flexibility and Performance is Possible
Drill into Drill – How Providing Flexibility and Performance is PossibleMapR Technologies
 
Increase Efficiency of Solaris Operations & Hardware Life Cycle
Increase Efficiency of Solaris Operations & Hardware Life CycleIncrease Efficiency of Solaris Operations & Hardware Life Cycle
Increase Efficiency of Solaris Operations & Hardware Life CycleJomaSoft
 
TechEvent Operating MapR Hadoop Cluster for a year
TechEvent Operating MapR Hadoop Cluster for a yearTechEvent Operating MapR Hadoop Cluster for a year
TechEvent Operating MapR Hadoop Cluster for a yearTrivadis
 
Hadoop over rgw
Hadoop over rgwHadoop over rgw
Hadoop over rgwzhouyuan
 
Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)MapR Technologies
 
Securing Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys BotzumSecuring Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys BotzumMapR Technologies
 
Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)
Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)
Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)BigDataEverywhere
 
Accelerated dataplanes integration and deployment
Accelerated dataplanes integration and deploymentAccelerated dataplanes integration and deployment
Accelerated dataplanes integration and deploymentOPNFV
 

Similar a 20a installation (20)

22 configuration
22 configuration22 configuration
22 configuration
 
HBase with MapR
HBase with MapRHBase with MapR
HBase with MapR
 
Big Data in Container; Hadoop Spark in Docker and Mesos
Big Data in Container; Hadoop Spark in Docker and MesosBig Data in Container; Hadoop Spark in Docker and Mesos
Big Data in Container; Hadoop Spark in Docker and Mesos
 
How to configure the cluster based on Multi-site (WAN) configuration
How to configure the clusterbased on Multi-site (WAN) configurationHow to configure the clusterbased on Multi-site (WAN) configuration
How to configure the cluster based on Multi-site (WAN) configuration
 
RHadoop - beginners
RHadoop - beginnersRHadoop - beginners
RHadoop - beginners
 
Deployment and Management of Hadoop Clusters
Deployment and Management of Hadoop ClustersDeployment and Management of Hadoop Clusters
Deployment and Management of Hadoop Clusters
 
Exp-3.pptx
Exp-3.pptxExp-3.pptx
Exp-3.pptx
 
3 map r installation & setup administration course description
3 map r installation & setup administration course description3 map r installation & setup administration course description
3 map r installation & setup administration course description
 
30a accessing your cluster
30a accessing your cluster30a accessing your cluster
30a accessing your cluster
 
Yarns About Yarn
Yarns About YarnYarns About Yarn
Yarns About Yarn
 
VDCF Overview
VDCF OverviewVDCF Overview
VDCF Overview
 
4. v sphere big data extensions hadoop
4. v sphere big data extensions   hadoop4. v sphere big data extensions   hadoop
4. v sphere big data extensions hadoop
 
Drill into Drill – How Providing Flexibility and Performance is Possible
Drill into Drill – How Providing Flexibility and Performance is PossibleDrill into Drill – How Providing Flexibility and Performance is Possible
Drill into Drill – How Providing Flexibility and Performance is Possible
 
Increase Efficiency of Solaris Operations & Hardware Life Cycle
Increase Efficiency of Solaris Operations & Hardware Life CycleIncrease Efficiency of Solaris Operations & Hardware Life Cycle
Increase Efficiency of Solaris Operations & Hardware Life Cycle
 
TechEvent Operating MapR Hadoop Cluster for a year
TechEvent Operating MapR Hadoop Cluster for a yearTechEvent Operating MapR Hadoop Cluster for a year
TechEvent Operating MapR Hadoop Cluster for a year
 
Hadoop over rgw
Hadoop over rgwHadoop over rgw
Hadoop over rgw
 
Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)Map r hadoop-security-mar2014 (2)
Map r hadoop-security-mar2014 (2)
 
Securing Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys BotzumSecuring Hadoop by MapR's Senior Principal Technologist Keys Botzum
Securing Hadoop by MapR's Senior Principal Technologist Keys Botzum
 
Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)
Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)
Big Data Everywhere Chicago: Getting Real with the MapR Platform (MapR)
 
Accelerated dataplanes integration and deployment
Accelerated dataplanes integration and deploymentAccelerated dataplanes integration and deployment
Accelerated dataplanes integration and deployment
 

Más de mapr-academy

Más de mapr-academy (9)

53 lab-nfs
53 lab-nfs53 lab-nfs
53 lab-nfs
 
51 lab-volumes
51 lab-volumes51 lab-volumes
51 lab-volumes
 
50a volumes
50a volumes50a volumes
50a volumes
 
48a tuning
48a tuning48a tuning
48a tuning
 
42 lab-managing services
42 lab-managing services42 lab-managing services
42 lab-managing services
 
41a managing services
41a managing services41a managing services
41a managing services
 
14 lab-planing
14 lab-planing14 lab-planing
14 lab-planing
 
12a architecture
12a architecture12a architecture
12a architecture
 
10c introduction
10c introduction10c introduction
10c introduction
 

Último

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.YounusS2
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 

Último (20)

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.Basic Building Blocks of Internet of Things.
Basic Building Blocks of Internet of Things.
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 

20a installation

  • 1. Installation 7/6/2012 © 2012 MapR Technologies Installation 1
  • 2. Installation Agenda • Installation Overview • Installation Methods • Installation Details • High Availability Considerations • LAB: Installation © 2012 MapR Technologies Installation 2
  • 3. Installation Objectives At the end of this module you will be able to: • Describe the steps involved in MapR installation • Install a cluster using the maprinstall script • Install MapR on a single node manually and with the maprinstall script • Install MapR Metrics • Explain what is involved in setting up HA for MapR © 2012 MapR Technologies Installation 3
  • 4. Installation Overview © 2012 MapR Technologies Installation 4
  • 5. Installation Overview  What needs to happen to install MapR on each node ? – Add MapR repository – Install MapR services and packages from repository – Configure the node (configure.sh) – Format disks (disksetup) – Install and configure MapR Metrics (optional) – Start the cluster – Additional configuration steps after install • Set up HA • Set up node topology (covered in Configuration module) © 2012 MapR Technologies Installation 5
  • 6. Installation Methods © 2012 MapR Technologies Installation 6
  • 7. Installation Methods  Cluster installation – Bring up an entire cluster with a script from one node • Download the maprinstall script and move to /opt/install-mapr • Run/opt/install-mapr/maprinstall –R <roles_file> – Performs node validation (OS partition, ports, memory, swap space, etc.) – Assumes ability to SSH from node running script to other nodes – Provide a roles file with comma-delimited lists of: » Nodes » Packages » Disks 10.10.100.2 mapr-fileserver,mapr-tasktracker /dev/sdb 10.10.100.3 mapr-fileserver,mapr-tasktracker /dev/sdb 10.10.100.1 mapr-cldb,mapr-zookeeper,mapr-jobtracker,mapr- webserver,mapr-fileserver,mapr-tasktracker /dev/sdb © 2012 MapR Technologies Installation 7
  • 8. Installation Methods  Individual node installation – Scripted • Download the maprinstall script and move to /opt/install-mapr • Run/opt/install-mapr/maprinstall – Assumes you can provide mandatory data (package list, CLDB, Zookeeper) – Performs node validation (OS partition, ports, memory, swap space, etc.) – Executes all steps below from one script – Manual • Add MapR repository • Install MapR services (apt-get, yum, etc.) • configure.sh • disksetup • Bring up the cluster © 2012 MapR Technologies Installation 8
  • 9. Installation Details © 2012 MapR Technologies Installation 9
  • 10. MapR Repository  From MapR repositories – apt-get or yum  From local repositories – Download .tar.gz – Set up repository – apt-get or yum  From local packages – Use dpkg or rpm – Must resolve dependencies manually © 2012 MapR Technologies Installation 10
  • 11. Configuration Script  configure.sh – Run on each node – Sets up mapr-clusters.conf – Writes ZK node IPs several places (mapr-cldb.conf is one) – Registers node with ZooKeeper and CLDB © 2012 MapR Technologies Installation 11
  • 12. Choosing Disks for MapR  MapR formats and uses disks for MapR-FS – Should use physical drives/partitions  Disk ready for MapR? – mount: partition in use? – df: partition in use? – Entry in /etc/fstab? – Not accessible to mkfs? mkfs fails?  If any above are true, disk is not ready © 2012 MapR Technologies Installation 12
  • 13. Disk Setup Script  disksetup – Takes input from a text file listing usable disks. E.g. /dev/sdb /dev/sdc1 /dev/sdc2 /dev/sdc4 /dev/sdd – Adds disks to disktab – Use –F option to force formatting – Creates storage pools (-w option specifies stripe width) – Run after configure.sh © 2012 MapR Technologies Installation 13
  • 14. Installing MapR Metrics  Prerequisites – MySQL server (on a node or separate machine) – EPEL repository – M5 license for all features (M3: no charts or histograms)  Installation procedure: – Install MapR Metrics package on all JobTracker and Webserver nodes • e.g. apt-get install mapr-metrics – MCS: System Settings > Metrics • Enter <hostname/IP>:<port> of MySQL server • Enter MySQL credentials – Source /opt/mapr/bin/setup.sql at mysql> prompt on a node where MapR Metrics is installed © 2012 MapR Technologies Installation 14
  • 15. Starting the Cluster  Bring up ZooKeeper on ZooKeeper nodes  Bring up Warden – Wait for services to start (use jps to see)  Set up administrative user  Log onto MapR Control System  Add license © 2012 MapR Technologies Installation 15
  • 16. High Availability Considerations © 2012 MapR Technologies Installation 16
  • 17. High Availability (HA)  CLDB  JobTracker  NFS © 2012 MapR Technologies Installation 17
  • 18. High Availability CLDB  Warm standby – One Primary CLDB active at a time – Secondary CLDB supports reads  Configure multiple CLDBs – Up to 6  Install across physical topology  Can isolate from all other services  Can also isolate CLDB volume on its own topology  Don’t run active JobTracker on primary CLDB node © 2012 MapR Technologies Installation 18
  • 19. High Availability JobTracker  Cold standby – One active JobTracker at a time – Stateful failover preserves running jobs  In most cases 3 – Unless > 100 nodes, then 5 © 2012 MapR Technologies Installation 19
  • 20. High Availability NFS  Active standby – Multiple concurrent NFS servers allowed – VIPs provide immediate failover – Failover triggered by CLDB if NFS server fails – Reads/writes continue after failover © 2012 MapR Technologies Installation 20
  • 21. Setting Up NFS HA  Install and run mapr-nfs service on assigned nodes  Create a pool of VIPs  Use MapR Control System or the API to assign VIPs to NFS nodes – One VIP across multiple NFS nodes – Multiple VIPs across multiple NFS nodes © 2012 MapR Technologies Installation 21
  • 22. LAB: Installation © 2012 MapR Technologies Installation 22
  • 23. Questions © 2012 MapR Technologies Installation 23