migrate hadoop, hadoop migration, hadoop upgradation, upgrade hadoop

1. (c) Infinity
1

2. Preparation
Phase
• Access Grants
• Assessment of Current Infrastructure
• Inventory of Current Setup
• Meta-Data Backup
• New Cluster Setup with Hadoop 2.x . OR Inline Migration to Hadoop 2.x from Hadoop
1.x
Methodology
Staging
Environment
Production
Rollout
• Replica of Current Setup in Staging Environment
• Upgrade in Staging Environment
• Validation of Migration in Staging Environment
• Upgrade of Production Environment
• Production Environment validation.
(c) Infinity
2

3. 
Upgrade a cluster by configuring a new cluster with same capacity and
newer Hadoop version and then migrate the files/data from old cluster
to new one.
(c) Infinity
3

4. CopyToLocal /CopyFromLocal

The process flows, the files are copied to local drive using Hadoop command
CopyToLocal and the files are then pushed to the new cluster using
CopyFromLocal, and the older cluster can be decommissioned.
Using Hadoop CP command

This is a kind of cluster to cluster copy, using Hadoop ‘cp’ command the files
are transferred from one HDFS to other HDFS. As the version are different we
need a mechanism called copy from HFTP where the command is executed from
the target cluster by defining source as old cluster with HFTP protocol and
target as HDFS protocol.
Using Hadoop DISTCP command

This is a kind of cluster to cluster copy, using Hadoop ‘distcp’ command the files
are transferred from one HDFS to other HDFS. As the version are
different we need a mechanism called copy from HFTP where the command is
executed from the target cluster by defining source as old cluster with HFTP
protocol and target as HDFS protocol.
(c) Infinity
4

5.  Upgrading
existing cluster from V1 to V2 inline by
Installing/Configuring new cluster and updating
metadata.
Hadoop V2
Hadoop V1
HDFS  Upgraded Metadata HDFS
(c) Infinity
5

6. Shutdown of
Cluster/Node
s
Validation of
Upgrade
Activity
Software
Stack Stage
Upgrade
Phase
(c) Infinity
Production
Rollout
6

7.  http://wiki.apache.org/hadoop/Hadoop_Upgrade
(c) Infinity
7

8.  Drop
me mail @
 dwivedishashwat@gmail.com
 Visit
my blog
 http://helpmetocode.blogspot.com
(c) Infinity
8