1. © Hortonworks Inc. 2013
HDFS NameNode HA
Jing Zhao
Hortonworks
Page 1

2. © Hortonworks Inc. 2013
Background and Motivation
• Prior to Hadoop 2.0
– One NameNode for metadata management
– Single point of failure
• HDFS High Availability
– Two NameNodes in the same cluster
– Active NameNode: responsive for all client operations
– Standby NameNode: maintain enough state to provide a fast failover
• Shared storage
– Active NN writes edit log
– Standby NN reads edit log and applies to its own namespace
– During failover, Standby NN reads all the edits and transitions to Active state
Page 2
Architecting the Future of Big Data

3. © Hortonworks Inc. 2013
Quorum Journal Management
• Before QJM
– NFS directory as shared storage
– Limitation: custom hardware, complicated deployment…
• QJM
– Quorum commits of edit log to separate daemons (JournalNodes)
– Paxos-like protocol to guarantee correctness
• JournalNodes
– A successful edit log commit must write to a majority of JNs
– Lightweight, can be co-located on machines for NN, JT, or RM
– Recommended number of JNs: 3 or 5
Page 3
Architecting the Future of Big Data

4. © Hortonworks Inc. 2013
Page 4
Architecting the Future of Big Data
DataNode DataNode DataNode
. . . . .
Standby
NameNode
Active
NameNode
Shared Edits
JournalNode JournalNode JournalNode
Block Reports to
Active & Standby
Write Read

5. © Hortonworks Inc. 2013
Automatic Failover
• Manual failover
– Administrative commands
• Use ZKFailoverController for Automatic Failover
– ZKFC: a ZooKeeper client that monitors and manages the NN state
– Health monitoring
– ZooKeeper session management
– ZooKeeper-base election
Page 5
Architecting the Future of Big Data

6. © Hortonworks Inc. 2013
Page 6
Architecting the Future of Big Data
DataNode DataNode DataNode
. . . . .
Standby
NameNode
Write
Active
NameNode
Shared Edits
JournalNode JournalNode JournalNode
ZKFC ZKFC
ZooKeeper Service
Cluster
Monitor and try to
take active lock
Monitor
liveness and
health.
Manage HA
state.
Write
Read
Monitor
liveness and
health.
Manage HA
state.
Monitor and maintain
active lock