Shulou(Shulou.com)05/31 Report--

This article mainly introduces how to build a distributed environment in hadoop, which has a certain reference value. Interested friends can refer to it. I hope you can learn a lot after reading this article.

one. Distributed environment building

Download: http://www.apache.org/dyn/closer.cgi/hbase/, hbase-0.98.0-hadoop2-bin.tar.gz.

1. Install HBase1 on the master main control computer) decompress

SHELL$ tar-zxvf hbase-0.98.0-hadoop2-bin.tar.gz

SHELL$ mv hbase-0.98.0-hadoop2 ~ / hbase0.98.0hadoop2

2) configure environment variables

(1) modify / etc/profile file

SHELL$ sudo gedit / etc/profile

(2) Verification

3) modify% HBASE%/conf/hbase-env.sh

4) modify $HBASE_HOME/conf/hbase-site.xml hbase.master hdfs://192.168.1.240:60000 hbase.rootdir hdfs://192.168.1.240:9000/hbase Hbase.cluster.distributed true hbase.zookeeper.quorum 192.168.1.241192.168.1.242192.168.1.243 hbase.tmp.dir / home/hadoop/hbase0.98.0hadoop2/hbase-tmp hbase.zookeeper.property.dataDir / home/hadoop/hbase0.98.0hadoop2/zookeeper-temp

5) $HBASE_HOME/conf/regionservers file increased

two。 Copy the HBase to the slave slave

SHELL$ sudo scp-rpv / home/hadoop/hbase0.98.0hadoop2/ hadoop@hapslave*:/home/hadoop/

3. Start the HBase cluster

After the Hadoop cluster starts, start the HBase cluster again.

SHELL$ start-hbase.sh

View through the web interface in the host computer (in this example, 4 nodes are configured):

4. Stop the HBase cluster

SHELL$ stop-hbase.sh

two. HBase Shell

SHELL$ hbase shell

1. Build table create

two。 List all tables list

3. Table description describe

4. Delete table disable,drop

5. Insert entry put

6. Show full table scan

7. Query entry get

8. Update entry put

9. Delete entry delete

Clear the table:

Truncate is a SQL syntax that can quickly empty all data in a table. And it can recalculate the count reset to zero for the fields with automatic increment.

10. Statistical parameter

three. JavaAPI

All the API is in the% HBase%/docs directory, all in English.

All the jar required for this example can be found in the% HBase installation directory% / lib directory. To save trouble, I imported it all.

1. Create a table package com.cuiweiyou.test;import java.io.IOException;import java.util.Iterator;import java.util.List;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.hbase.Cell;import org.apache.hadoop.hbase.CellUtil;import org.apache.hadoop.hbase.HBaseConfiguration;import org.apache.hadoop.hbase.HColumnDescriptor;import org.apache.hadoop.hbase.HTableDescriptor;import org.apache.hadoop.hbase.TableName;import org.apache.hadoop.hbase.client.Delete Import org.apache.hadoop.hbase.client.Get;import org.apache.hadoop.hbase.client.HBaseAdmin;import org.apache.hadoop.hbase.client.HTable;import org.apache.hadoop.hbase.client.Put;import org.apache.hadoop.hbase.client.Result;import org.apache.hadoop.hbase.client.ResultScanner;import org.apache.hadoop.hbase.client.Scan;import org.apache.hadoop.hbase.util.Bytes;import org.junit.Test Public class HBaseTest {/ / create table @ Test public void creatTable () throws Exception {String strTBName = "tb_test"; / / table String strColFamily = "cf"; / / column family / / configure Configuration conf = HBaseConfiguration.create () Conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243"); / / administrator HBaseAdmin hbaseAdmin = new HBaseAdmin (conf) / / addColumn (String tableName, HColumnDescriptor column) / / add / / checkHBaseAvailable (HBaseConfiguration hbaseConf) / / static function to an existing table Check to see if HBase is running / / deleteTable (byte [] tableName) / / delete an existing table / / enableTable (byte [] tableName) / / make the table in a valid state / / disableTable (byte [] tableName) / / invalidate the table / / HTableDescriptor [] listTables () / / list There are user control table items / / modifyTable (byte [] tableName) HTableDescriptor tableDesc) / / modify the schema of the table Is an asynchronous operation that takes time / / tableExists (String tableName) / / checks whether the table exists / / the table name TableName tableName = TableName.valueOf (strTBName) / / Table descriptor HTableDescriptor tableDesc = new HTableDescriptor (tableName) / / removeFamily (byte [] column) / / remove a column family / / getName () / / get the name of the table / / getValue (byte [] key) / / get the value of the attribute / / setValue (String key String value) / / sets the value of the property tableDesc.addFamily (new HColumnDescriptor (strColFamily)) / / add column families / / create a table and synchronize hbaseAdmin.createTable (tableDesc); System.out.println ("create table" + strTBName + "success");}}

two。 Add a record / / add data @ Test public void addData () throws IOException {String strTBName = "tb_test"; String strColFamily = "cf"; String strColumn = "col"; / / column name String strRowKey = "row1"; / / Row number String strValue = "values" / / value Configuration conf = HBaseConfiguration.create (); conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243"); / / Table instance HTable table = new HTable (conf, strTBName) / / close () frees all resources or suspends updates in the internal buffer / / exists (Get get) Check whether the value specified by the Get instance exists in the column of HTable / / get (Get get) to get the specified row The values corresponding to some cells of / / getEndKeys () get the end key value of each region of the currently open table / / getScanner (byte [] family) Get the scanner instance of the current given column family / / getTableDescriptor () get the HTableDescriptor instance / / getTableName () of the current table Get the table name / / isTableEnabled (HBaseConfiguration conf String tableName) check whether the table is valid / / get all column families HColumnDescriptor [] columnFamilies = table.getTableDescriptor () .getColumnFamilies () / / the common method of HColumnDescriptor: / / getName () / / get the name of the column family / / getValue (byte [] key) / / get the value of the corresponding attribute / / setValue (String key) String value) / / set the value of the corresponding property / / inserter Put put = new Put (Bytes.toBytes (strRowKey)) / / set the line number RowKey / / add (byte [] family, byte [] qualifier, byte [] value) adds the specified column and corresponding values to the Put instance / / add (byte [] family, byte [] qualifier, long ts) Byte [] value) adds the specified column and corresponding value and timestamp to the Put instance / / getRow () to get the row / / getRowLock () of the Put instance Get the row lock / / getTimeStamp () of the Put instance to get the Put reality The timestamp / / isEmpty () of the example checks whether familyMap is empty / / setTimeStamp (long timeStamp) Set the timestamp for of the Put instance (int I = 0) I < columnFamilies.length; iTunes +) {String familyName = columnfamilies [I] .getNameAsString () / / get the column family name / / specify the column family if (familyName.equals (strColFamily)) {/ / insert put.add (Bytes.toBytes (familyName), Bytes.toBytes (strColumn), Bytes.toBytes (strValue)) }} table.put (put); / / run the inserter System.out.println ("data storage completed");}

3. Read the specified row record / / query the whole row @ Test public void getRow () throws IOException {String strTBName = "tb_test"; String strRowKey = "row1"; Configuration conf = HBaseConfiguration.create (); conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243") according to RowKey HTable table = new HTable (conf, strTBName); / / get table instance / / querier Get get = new Get (Bytes.toBytes (strRowKey)) / / query specified row / / addColumn (byte [] family, byte [] qualifier) get the column corresponding to the specified column family and column modifier / / addFamily (byte [] family) get all the columns corresponding to its corresponding column through the specified column family / / setTimeRange (long minStamp Long maxStamp) gets the version number of the column of the specified pickup / / setFilter (Filter filter) sets the server-side filter Result result = table.get (get) when performing the Get operation / / containsColumn (byte [] family, byte [] qualifier) checks whether the specified column exists / / getFamilyMap (byte [] family) gets the key-value pair / / getValue (byte [] family) of the modifiers and values contained in the corresponding column family. Byte [] qualifier) gets the latest value of the corresponding column List listCells = result.listCells () / / specify all columns of for (Cell cell: listCells) {System.out.println ("column family:" + Bytes.toString (CellUtil.cloneFamily (cell); System.out.println ("column name:" + Bytes.toString (CellUtil.cloneQualifier (cell) System.out.println ("column value:" + Bytes.toString (CellUtil.cloneValue (cell); System.out.println ("timestamp:" + cell.getTimestamp ());}}

4. Show all data / / traverse all entries @ Test public void getAllRows () throws IOException {String strTBName = "tb_test"; Configuration conf = HBaseConfiguration.create (); conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243"); HTable table = new HTable (conf, strTBName) / / get table instance / / Scanner ResultScanner resultScanner = table.getScanner (new Scan ()); / / Querier for full table Iterator results = resultScanner.iterator (); while (results.hasNext ()) {Result result = results.next (); List cells = result.listCells () For (Cell cell: cells) {System.out.println ("column Family:" + Bytes.toString (CellUtil.cloneFamily (cell); System.out.println ("column name:" + Bytes.toString (CellUtil.cloneQualifier (cell) System.out.println ("column value:" + Bytes.toString (CellUtil.cloneValue (cell); System.out.println ("timestamp:" + cell.getTimestamp () + "\ n -");}

5. Update entry / / update a column of a row in the table @ Test public void updateTable () throws IOException {String strTBName = "tb_test"; String strColFamily = "cf"; String strColumn = "col"; String strRowKey = "row1"; String strNewValue = "NewValues"; Configuration conf = HBaseConfiguration.create () Conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243"); HTable table = new HTable (conf, strTBName); / / get table instance Put put = new Put (Bytes.toBytes (strRowKey)) / / still insert operation (known column family, known column, new value) put.add (Bytes.toBytes (strColFamily), Bytes.toBytes (strColumn), Bytes.toBytes (strNewValue)); table.put (put); System.out.println ("end of update");}

6. Delete cell / / Delete the specified column of the specified row (delete cell) @ Test public void deleteColumn () throws IOException {String strTBName = "tb_test"; String strColFamily = "cf"; String strColumn = "col"; String strRowKey = "row1"; Configuration conf = HBaseConfiguration.create () Conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243"); HTable table = new HTable (conf, strTBName); / / get table instance / / deleter Delete del = new Delete (Bytes.toBytes (strRowKey)) Del.deleteColumns (Bytes.toBytes (strColFamily), Bytes.toBytes (strColumn)); table.delete (del); System.out.println ("Row:" + strRowKey + ", column family:" + strColFamily + ", column:" + strColumn + ", deletion completed");}

7. Delete the whole line / / delete the whole line @ Test public void deleteAllColumn () throws IOException {String strTBName = "tb_test"; String strRowKey = "row1"; Configuration conf = HBaseConfiguration.create (); conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243") HTable table = new HTable (conf, strTBName); / / get table instance Delete deleteAll = new Delete (Bytes.toBytes (strRowKey)); table.delete (deleteAll); System.out.println ("this row is all deleted");}

8. Delete form / / Delete table @ Test public void deleteTable () throws IOException {String strTBName = "tb_test"; Configuration conf = HBaseConfiguration.create (); conf.set ("hbase.zookeeper.quorum", "192.168.1.241192.168.1.242192.168.1.243"); HBaseAdmin admin = new HBaseAdmin (conf) Admin.disableTable (strTBName); admin.deleteTable (strTBName); System.out.println (strTBName + "Table deleted");}

Thank you for reading this article carefully. I hope the article "how to build a distributed Environment in hadoop" shared by the editor will be helpful to you. At the same time, I also hope that you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.