Shulou(Shulou.com)06/01 Report--

Frontier:

Mongodb used to be used, but the amount was too large. Mongodb seemed not so reliable. It was changed to hbase to support a magnitude.

HBase is an Apache Hadoop database that provides random, real-time read and write access to large data sets. The goal of HBase is to store and process large amounts of data. HBase is an open source, distributed, multi-version, column-oriented storage model. It stores loose data.

HBase provides rich access interfaces.

HBase Shell

Java clietn API

Jython、Groovy DSL、Scala

REST

Thrift（Ruby、Python、Perl、C++…）

MapReduce

Hive/Pig

hbase(main):001:0>

#Creating tables

hbase(main):002:0* create 'blog','info','content'

0 row(s) in 2.0290 seconds

#View table

hbase(main):003:0> list

TABLE

blog

test_standalone

2 row(s) in 0.0270 seconds

#Add data

hbase(main):004:0> put 'blog','1','info:editor','liudehua'

0 row(s) in 0.1340 seconds

hbase(main):005:0> put 'blog','1','info:address','bj'

0 row(s) in 0.0070 seconds

hbase(main):006:0> put 'blog','1','content:header','this is header'

0 row(s) in 0.0070 seconds

hbase(main):007:0>

hbase(main):008:0*

hbase(main):009:0* get 'blog','1'

COLUMN CELL

content:header timestamp=1407464302384, value=this is header

info:address timestamp=1407464281942, value=bj

info:editor timestamp=1407464270098, value=liudehua

3 row(s) in 0.0360 seconds

hbase(main):010:0> get 'blog','1','info'

COLUMN CELL

info:address timestamp=1407464281942, value=bj

info:editor timestamp=1407464270098, value=liudehua

2 row(s) in 0.0120 seconds

#This can be queried according to the conditions.

hbase(main):012:0* scan 'blog'

ROW COLUMN+CELL

1 column=content:header, timestamp=1407464302384, value=this is header

1 column=info:address, timestamp=1407464281942, value=bj

1 column=info:editor, timestamp=1407464270098, value=liudehua

1 row(s) in 0.0490 seconds

hbase(main):013:0>

hbase(main):014:0* put 'blog','1','content:header','this is header2'

0 row(s) in 0.0080 seconds

hbase(main):015:0>

hbase(main):016:0*

hbase(main):017:0* put 'blog','1','content:header','this is header3'

0 row(s) in 0.0050 seconds

hbase(main):018:0> scan 'blog'

ROW COLUMN+CELL

1 column=content:header, timestamp=1407464457128, value=this is header3

1 column=info:address, timestamp=1407464281942, value=bj

1 column=info:editor, timestamp=1407464270098, value=liudehua

1 row(s) in 0.0180 seconds

hbase(main):020:0> get 'blog','1','content:header'

COLUMN CELL

content:header timestamp=1407464457128, value=this is header3

1 row(s) in 0.0090 seconds

hbase(main):021:0>

Original text: rfyiamcool.blog.51cto.com/1030776/1537505

#You can see the version history

hbase(main):022:0* get 'blog','1',{COLUMN => 'content:header',VERSIONS => 2}

COLUMN CELL

content:header timestamp=1407464457128, value=this is header3

content:header timestamp=1407464454648, value=this is header2

2 row(s) in 0.0100 seconds

#You can see the version history

hbase(main):023:0> get 'blog','1',{COLUMN => 'content:header',VERSIONS => 3}

COLUMN CELL

content:header timestamp=1407464457128, value=this is header3

content:header timestamp=1407464454648, value=this is header2

content:header timestamp=1407464302384, value=this is header

3 row(s) in 0.0490 seconds

hbase(main):024:0>

Java is the most convenient and efficient way to operate base. Java is not lightweight, however, and it is inconvenient to debug in any environment. Moreover, different developers are familiar with different languages, and their development efficiency is also different. hbase can also be operated with python,ruby,cpp,perl and other languages through thrift.

Thrift is an open-source remote invocation component developed by facebook that resembles google's protobuf. But protobuf only serializes data and supports only binary protocols, with no remote invocation component. Protobuf natively supports cpp,python,java, as well as third-party implementations of objectc, ruby and other languages. Thrift implements serialization, transmission, protocol definition, remote call and other functions, and has more cross-language capabilities. Some aspects may be substituted for each other, but others have their own scope of application.

Original text: rfyiamcool.blog.51cto.com/1030776/1537505

Thrift Installation and Thrift Python Related Modules ~

http://www.apache.org/dist//thrift/0.9.1/thrift-0.9.1.tar.gztar zxvf thrift-0.8.0.tar.gzcd thrift-0.8.0./ configure -with-cpp=nomakesudo make installsudo pip install thrift

Here are the thrift and hbase modules that can generate python ~

thrift -gen py /home/ubuntu/hbase-0.98.1/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift

from thrift.transport import TSocketfrom thrift.protocol import TBinaryProtocolfrom hbase import Hbasetransport=TSocket.TSocket('localhost',9090)protocol=TBinaryProtocol.TBinaryProtocol(transport)client=Hbase.Client(protocol)transport.open()client.getTableNames()

Original text: rfyiamcool.blog.51cto.com/1030776/1537505

hbase version 0.98 does not seem to have thrift related components, I use version 0.94 here to get it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.