Shulou(Shulou.com)06/02 Report--

I. Summary

This article records the deployment process of hbase high availability cluster. You need to deploy the hadoop cluster before deploying hbase, because the data of hbase needs to be stored on hdfs. The deployment of hadoop cluster will be recorded in a subsequent article. This article assumes that hadoop cluster has been deployed, distributed hbase cluster needs to rely on zk, and zk can be hosted by hbase itself or built by ourselves. Here we use our own zk cluster. Our hadoop cluster is a distribution of cdh, so hbase also uses the source of cdh.

II. Environment

1. Software version

Centos6

Zookeeper-3.4.5+cdh6.9.0+98-1.cdh6.9.0.p0.30.el6.x86_64

Hadoop-2.6.0+cdh6.9.0+1799-1.cdh6.9.0.p0.30.el6.x86_64

Hbase-1.2.0+cdh6.9.0+205-1.cdh6.9.0.p0.30.el6.x86_64

2. Role

A, zk cluster

10.10.20.64:218110.10.40.212:218110.10.102.207:2181

B 、 hbase

10.10.40.212 HMaster10.10.20.64 HMaster10.10.10.114 HRegionServer10.10.40.169 HRegionServer10.10.30.174 HRegionServer

III. Deployment

1. Configure the yum source of cdh

Vim / etc/yum.repos.d/cloudera-cdh.repo [cloudera-cdh6] # Packages for Cloudera's Distribution for Hadoop, Version 5.4.4, on RedHat or CentOS 6 x86_64name=Cloudera's Distribution for Hadoop, Version 5.4.8 baseurl = http://archive.cloudera.com/cdh6/redhat/6/x86_64/cdh/5.9.0/gpgkey=http://archive.cloudera.com/cdh6/redhat/6/x86_64/cdh/RPM-GPG-KEY-cloudera gpgcheck=1 [cloudera-gplextras5b2] # Packages for Cloudera's GPLExtras Version 5.4.4, on RedHat or CentOS 6 x86_64name=Cloudera's GPLExtras, Version 5.4.8 baseurl = http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/5.9.0/gpgkey=http://archive.cloudera.com/gplextras5/redhat/6/x86_64/gplextras/RPM-GPG-KEY-cloudera gpgcheck=1

2. Install zk cluster (all zk nodes are operated)

1. Installation

Yum-y install zookeeper zookeeper-server

B, configuration

Vim / etc/zookeeper/conf/zoo.cfg tickTime=2000initLimit=10syncLimit=5dataDir=/data/lib/zookeeperclientPort=2181maxClientCnxns=0server.1=10.10.20.64:2888:3888server.2=10.10.40.212:2888:3888server.3=10.10.102.207:2888:3888autopurge.snapRetainCount=3autopurge.purgeInterval=1mkdir-p / data/lib/zookeeper # create the dir directory of zk echo 1 > / data/lib/zookeeper/myid # 10.10.20.64 operate on echo 2 > / data/lib/zookeeper/myid # 10.10.40.212 Operation on echo 3 > / data/lib/zookeeper/myid # 10.10.102.207

C. Start the service

/ etc/init.d/zookeeper-server start

3. Install and configure hbase cluster

A, installation

Yum-y install hbase hbase-master # HMaster Node Operation yum-y install hbase hbase-regionserver # HRegionServer Node Operation

B, configuration (all base node operations)

Vim / etc/hbase/conf/hbase-site.xml hbase.zookeeper.quorum 10.10.20.64:2181,10.10.40.212:2181,10.10.102.207:2181 hbase.zookeeper.property.clientPort 2181 hbase.zookeeper.property.dataDir / data/lib/zookeeper/ hbase.rootdir hdfs://mycluster:8020/hbase The mode of hbase.cluster.distributed true cluster Distributed or stand-alone mode. If set to false, the HBase process and the Zookeeper process are in the same JVM process echo "export HBASE_MANAGES_ZK=false" > > / etc/hbase/conf/hbase-env.sh# setting hbase uses a separate zk cluster vim / etc/hbase/conf/regionservers ip-10-10-30-174.ec2.internalip-10-10-10-114.ec2.internalip-10-10-40-169.ec2.internal# to add the hostname of HRegionServer to regionservers Instead of mapping the hostname under / etc/hosts, I directly used the default hostname of ec2

C. Start the service

/ etc/init.d/hbase-master start # HMaster node operation / etc/init.d/hbase-regionserver start # HRegionServer node operation

4. Verification

A. Verify basic functions

[root@ip-10-10-20-64] # hbase shell 2017-05-10 16 INFO 31 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.availableHBase Shell Enter 'help' for list of supported commands.Type "exit" to leave the HBase ShellVersion 1.2.0-cdh6.9.0, rUnknown, Fri Oct 21 01:19:47 PDT 2016hbase (main): 001 backup masters 0 > status1 active master, 1 backup masters, 3 servers, 0 dead 1.3333 average loadhbase (main): 002VO > listTABLE test Test1 2 row (s) in 0.0330 seconds= > ["test" "test1"] hbase (main): 003main 0 > describe 'test'Table test is ENABLED test COLUMN FAMILIES DESCRIPTION {NAME = > 'id' BLOOMFILTER = > 'ROW', VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 0mm, BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536' REPLICATION_SCOPE = >'0'} {NAME = > 'name', BLOOMFILTER = >' ROW', VERSIONS = >'1', IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 0' BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} {NAME = > 'text', BLOOMFILTER = >' ROW', VERSIONS = >'1', IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER' COMPRESSION = > 'NONE', MIN_VERSIONS = >' 013, BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} 3 row (s) in 0.1150 secondshbase (main): 004BLOCKCACHE 0 >

B. Verify the HA function

1. The default web management port of hbase is 60010. Whoever starts the two HMaster first is the main active node. 10.10.40.212 starts first, and 10.10.20.64 starts later. The web screenshot is as follows:

2. Stop the HMaster process of 10.10.40.212 and see if 10.10.20.64 will be promoted to master.

/ etc/init.d/hbase-master stop

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.