Shulou(Shulou.com)05/31 Report--

Editor to share with you hbase-0.96.x relative to hbase-0.94.x what has changed, I believe that most people do not know much about it, so share this article for your reference, I hope you will learn a lot after reading this article, let's go to know it!

Environment:

Hadoop:hadoop-2.2.0

Hbase:hbase-0.96.0

1.org.apache.hadoop.hbase.client.Put

Canceled the construction method without parameters.

The Put class no longer inherits the Writable class

Public class Put extends Mutation implements HeapSize, Writable, Comparable at 0.94.6

Public class Put extends Mutation implements HeapSize, Comparable at 0.96.0

Solution:

By public class MonthUserLoginTimeIndexReducer extends Reducer {

Change to public class MonthUserLoginTimeIndexReducer extends Reducer {

2.org.apache.hadoop.hbase.client.Mutation.familyMap

Org.apache.hadoop.hbase.client.Mutation.familyMap type change:

/ * *

* 0.94.6

* protected Map familyMap

*

* 0.96.*

* protected NavigableMap familyMap

* not available in org.apache.hadoop.hbase.Cell hbase-0.94.*

, /

Changes in org.apache.hadoop.hbase.KeyValue:

/ * *

* 0.94.*

* public class KeyValue extends Object implements Writable, HeapSize

*

* 0.96.0

* public class KeyValue extends Object implements Cell, HeapSize, Cloneable

, /

Solution: change the List in the code to List

3. Org.apache.hadoop.hbase.KeyValue

Method getFamily in 0.96.0 has been deprecated (Deprecated) and changed to method getFamilyArray ()

4.org.apache.hadoop.hbase.HTableDescriptor

The constructor public HTableDescriptor (String name) of class org.apache.hadoop.hbase.HTableDescriptor has been deprecated (Deprecated)

Solution: use public HTableDescriptor (TableName name)

Old: HTableDescriptor tableDesc = new HTableDescriptor (tableName)

New: HTableDescriptor tableDesc = new HTableDescriptor (TableName.valueOf (tableName))

5.org.apache.hadoop.hbase.client.HTablePool

Class org.apache.hadoop.hbase.client.HTablePool is completely deprecated (Deprecated)

Solution: use HConnection.getTable (String) instead, HConnection is an interface, and class CoprocessorHConnection is its only implementation class:

HRegionServer hRegionServer = new HRegionServer (conf)

HConnection connection = HConnectionManager.createConnection (conf)

HConnection = new CoprocessorHConnection (connection,hRegionServer)

6.org.apache.hadoop.hbase.client.Result

Method public KeyValue [] raw () is deprecated (Deprecated), and it is recommended to use public Cell [] rawCells ()

Method getRow is deprecated (Deprecated)

Method getFamily is deprecated (Deprecated)

Method getQualifier is deprecated (Deprecated)

Method getValue is deprecated (Deprecated)

Method public List getColumn (byte [] family,byte [] qualifier) is deprecated (Deprecated)

Method public KeyValue getColumnLatest (byte [] family,byte [] qualifier) is deprecated (Deprecated)

In Cell: change to the following method

GetRowArray ()

GetFamilyArray ()

GetQualifierArray ()

GetValueArray ()

In Result: add the following methods

Public List getColumnCells (byte [] family,byte [] qualifier)

Public KeyValue getColumnLatestCell (byte [] family,byte [] qualifier)

Changes: all ipeijian_data items related to "new users, active users and lost users] are changed as follows:

Old code: if (value.raw (). Length = = 1

New code: if (value.rawCells (). Length = = 1

Set TableInputFormat.SCAN in 7.job

The method is removed from 0.96.0: public void write (DataOutput out) throws IOException

Previous versions used conf.set (TableInputFormat.SCAN, StatUtils.convertScanToString (scan)); to set up

The specific implementation of StatUtils.convertScanToString is as follows:

Public static String convertScanToString (Scan scan) throws IOException {

ByteArrayOutputStream out = new ByteArrayOutputStream ()

DataOutputStream dos = new DataOutputStream (out)

Scan.write (dos)

Return Base64.encodeBytes (out.toByteArray ())

}

The implementation of this method is the same as TableMapReduceUtil.convertScanToString (Scan scan).

But when hbase is upgraded to 0.96. * the method write is deprecated for class Scan (not just Deprecated, but Deleted), so the above

The implementation becomes incorrect

This method is reimplemented in hbase0.96.*:

Public static String convertScanToString (Scan scan) throws IOException {

ClientProtos.Scan proto = ProtobufUtil.toScan (scan)

Return Base64.encodeBytes (proto.toByteArray ())

}

So make the following changes:

The implementation of the method convertScanToString in the StatUtils class has been changed as above to fit the hbase0.96.*

8.cn.m15.ipj.db.hbase.MyPut

Custom Put class, one more length than traditional Put class, comparison between original and new code:

Original: (the red font is API and becomes the wrong place in the new edition of Times)

Public class MyPut extends Put {

Public MyPut (byte [] row, int length) {

/ / the reason is that the parameter-free construction method of put has disappeared in the new book.

If (row = = null | | length > HConstants.MAX_ROW_LENGTH) {

Throw new IllegalArgumentException ("Row key is invalid")

}

This.row = Arrays.copyOf (row, length)

This.ts = HConstants.LATEST_TIMESTAMP

}

Public MyPut add (byte [] family, byte [] qualifier, long ts, byte [] value,int length) {

List list = getKeyValueList (family)

KeyValue kv = createPutKeyValue (family, qualifier, ts, value, length)

List.add (kv)

FamilyMap.put (kv.getFamily (), list)

/ / the type of familyMap has changed

Return this

}

Private List getKeyValueList (byte [] family) {

List list = familyMap.get (family)

/ / the type of familyMap has changed

If (list = = null) {

List = new ArrayList (0)

}

Return list

}

Private KeyValue createPutKeyValue (byte [] family, byte [] qualifier,long ts, byte [] value, int length) {

Return new KeyValue (this.row, 0, this.row.length, family, 0

Family.length, qualifier, 0, qualifier.length, ts

KeyValue.Type.Put, value, 0, length)

}

}

After the change:

Public MyPut (byte [] row, int length) {

Super (row,length)

/ / New addition

If (row = = null | | length > HConstants.MAX_ROW_LENGTH) {

Throw new IllegalArgumentException ("Row key is invalid")

}

This.row = Arrays.copyOf (row, length)

This.ts = HConstants.LATEST_TIMESTAMP

}

Public MyPut add (byte [] family, byte [] qualifier, long ts, byte [] value,int length) {

List list = getCellsList (family)

KeyValue kv = createPutKeyValue (family, qualifier, ts, value, length)

List.add (kv)

FamilyMap.put (CellUtil.cloneFamily (kv), list)

Return this

}

Private List getCellsList (byte [] family) {

List list = familyMap.get (family)

If (list = = null) {

List = new ArrayList (0)

}

Return list

}

Private KeyValue createPutKeyValue (byte [] family, byte [] qualifier,long ts, byte [] value, int length) {

Return new KeyValue (this.row, 0, this.row.length, family, 0 focus family.length, qualifier, 0, qualifier.length, ts

KeyValue.Type.Put, value, 0, length)

}

}

The above is all the contents of this article entitled "what are the changes in hbase-0.96.x compared to hbase-0.94.x?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.