以下笔记为简略记录，详细实战内容待补充。yarn-ha测试未通过！！！

【HDFS-HA规划】

 机器一       机器二      机器三

namenode    namenode

datanode datanode    datanode

journalnode journalnode journalnode

【HDFS-HA配置】

备份分布式配置

[zeal@data1 etc]$ cp -r hadoop/ dist-hadoop

[zeal@data1 data]$ mv tmp/ dist-tmp

[zeal@data1 data]$ mkdir tmp

参考文档

https://hadoop.apache.org/docs/r2.5.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html

配置hdfs-site.xml

配置core-site.xml

分发各服务器

[zeal@data1 etc]$ scp -r hadoop/ zeal@data2.zeal.name:/opt/modules/hadoop-2.5.0/etc/

[zeal@data1 etc]$ scp -r hadoop/ zeal@data3.zeal.name:/opt/modules/hadoop-2.5.0/etc/

【启动hadoop ha测试】

启动journalnode

[zeal@data1 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start journalnode

[zeal@data2 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start journalnode

[zeal@data3 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start journalnode

格式化hdfs，启动nn1上的namenode

[zeal@data1 hadoop-2.5.0]$ bin/hdfs namenode -format

[zeal@data1 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start namenode

同步nn1上元数据到nn2

[zeal@data2 hadoop-2.5.0]$ bin/hdfs namenode -bootstrapStandby

启动nn2上的namenode

[zeal@data2 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start namenode

访问监控地址

http://data1.zeal.name:50070/

http://data2.zeal.name:50070/

将nn1设置为active模式

[zeal@data1 hadoop-2.5.0]$ bin/hdfs haadmin -transitionToActive nn1

【手动Hadoop故障转移测试】

[zeal@data1 hadoop-2.5.0]$ sbin/stop-dfs.sh

[zeal@data1 hadoop-2.5.0]$ sbin/start-dfs.sh

[zeal@data1 hadoop-2.5.0]$ bin/hdfs haadmin -transitionToActive nn1

[zeal@data1 hadoop-2.5.0]$ bin/hdfs dfs -mkdir -p /user/zeal/data

[zeal@data1 hadoop-2.5.0]$ bin/hdfs dfs -put /opt/datas/wc.input /user/zeal/data

[zeal@data1 hadoop-2.5.0]$ bin/hdfs dfs -text /user/zeal/data/wc.input

[zeal@data1 hadoop-2.5.0]$ jps

3493 Jps

2799 DataNode

2699 NameNode

2987 JournalNode

[zeal@data1 hadoop-2.5.0]$ kill -9 2699

[zeal@data2 hadoop-2.5.0]$ bin/hdfs haadmin -transitionToActive nn2 --forceactive

[zeal@data1 hadoop-2.5.0]$ bin/hdfs dfs -text /user/zeal/data/wc.input

【自动Hadoop故障转移测试】

1.配置hdfs-site.xml

2.集群data2配置SSH无秘钥登录其他机器（同data1）

3.分发配置

[zeal@data1 hadoop-2.5.0]$ scp etc/hadoop/hdfs-site.xml zeal@data2.zeal.name:/opt/modules/hadoop-2.5.0/etc/hadoop/

4.启停服务

[zeal@data1 hadoop-2.5.0]$ sbin/stop-dfs.sh

[zeal@data1 zookeeper-3.4.5-cdh5.10.0]$ bin/zkServer.sh start

[zeal@data2 zookeeper-3.4.5-cdh5.10.0]$ bin/zkServer.sh start

[zeal@data3 zookeeper-3.4.5-cdh5.10.0]$ bin/zkServer.sh start

[zeal@data1 hadoop-2.5.0]$ bin/hdfs zkfc -formatZK

[zeal@data1 hadoop-2.5.0]$ sbin/start-dfs.sh

[zeal@data1 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start zkfc

[zeal@data2 hadoop-2.5.0]$ sbin/hadoop-daemon.sh start zkfc

5.故障转移测试

①停止data1上的namenode进程，查看data2上namenode是否自动切换active

②停止data2上的namenode进程，查看data1上namenode是否自动切换active

③读取数据测试

[zeal@data1 hadoop-2.5.0]$ bin/hdfs dfs -text /user/zeal/data/core-site.xml

【yarn-HA规划】

机器一    机器二      机器三

resourcemanager resourcemanager

nodemanager     nodemanager     nodemanager

配置yarn-site.xml

分发配置

[zeal@data1 hadoop]$ scp yarn-site.xml zeal@data2.zeal.name:/opt/modules/hadoop-2.5.0/etc/hadoop/

[zeal@data1 hadoop]$ scp yarn-site.xml zeal@data3.zeal.name:/opt/modules/hadoop-2.5.0/etc/hadoop/

测试MapReduce故障转移

分别启动机器上的resourcemanager和nodemanager

查看进程

[zeal@data1 hadoop-2.5.0]$ jps

5081 NodeManager

5116 Jps

2224 QuorumPeerMain

2535 DataNode

4091 NameNode

2722 JournalNode

2875 DFSZKFailoverController

4822 ResourceManager

[zeal@data2 hadoop-2.5.0]$ jps

4232 NodeManager

3691 NameNode

4267 Jps

1984 QuorumPeerMain

2223 JournalNode

2125 DataNode

4156 ResourceManager

2350 DFSZKFailoverController

[zeal@data3 hadoop-2.5.0]$ jps

2223 DataNode

2147 QuorumPeerMain

2315 JournalNode

2983 NodeManager

3013 Jps

访问监控地址

http://data1.zeal.name:8088/cluster

http://data2.zeal.name:8088/cluster

其中standby服务自动重定向到active上

[zeal@data1 hadoop-2.5.0]$ bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount /user/zeal/data/wc.input /user/zeal/data/output/1

#######################################未测试通过##################################

https://hadoop.apache.org/docs/r2.5.2/

java.io.IOException: org.apache.hadoop.yarn.exceptions.ApplicationNotFoundException: Application with id 'application_1555599511366_0001' doesn't exist in RM.