Shulou(Shulou.com)06/01 Report--

Compression formats supported by Hbase:

Compression formats supported by hbase: GZ (GZIP), LZ0,LZ4,Snappy

GZ: for cold data compression, GZIP has a higher compression ratio than Snappy and LZ0, but consumes more CPU, and decompresses / compresses more slowly.

Snappy and LZ0: used for hot data compression, using less CPU, decompression / compression speed is faster than GZ, but the compression ratio is not as high as GZ.

Compared with LZ0, the overall performance of Snappy is better than that of LZ0,Snappy and the compression ratio of Snappy is lower than that of LZ0, but the decompression / compression speed is faster.

Compared with LZ0, the compression ratio of LZ4 is not much different from that of LZ0, but the decompression / compression speed of LZ4 is faster.

In most cases, Snppy or LZ0 is a better choice because of their low compression overhead and space savings.

Specify the compression format when creating the table

Hbase (main): 013test3', 0 > create 'test3', {NAME= >' f1'}, {NAME= > 'f2' COMPRESSION= > 'Snappy'} 0 row (s) in 1.2740 seconds = > Hbase::Table-test3hbase (main): 014 in 0 > desc' test3'Table test3 is ENABLED test3 COLUMN FAMILIES DESCRIPTION {NAME = >'F1' BLOOMFILTER = > 'ROW',VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 01mm 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} {NAME = > f2mm, BLOOMFILTER = > 'ROW',VERSIONS = >' 1' IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'SNAPPY', MIN_VERSIONS = >' 0prima BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} 2 row (s) in 0.0300 secondshbase (main): 002 main > create 'test4', {NAME= >' F1'}, {NAME= > 'f2' COMPRESSION= > 'GZ'} 0 row (s) in 1.4900 seconds= > Hbase::Table-test4hbase (main): 003row 0 > desc' test4'Table test4 is ENABLED test4 COLUMN FAMILIES DESCRIPTION {NAME = >'F1' BLOOMFILTER = > 'ROW', VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 013', BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} {NAME = >'f2', BLOOMFILTER = > 'ROW', VERSIONS = >' 1' IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'GZ', MIN_VERSIONS = >' 013', BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} 2 row (s) in 0.1290 seconds

Modify columnfamily compression format after table creation

The correct way is to first disable the table, then modify the column family compression format, and then do major_compact operation after the enbale table.

As follows:

Hbase (main): 004desc 0 > test1'Table test1 is ENABLED test1 COLUMN FAMILIES DESCRIPTION {NAME = >'F1' BLOOMFILTER = > 'ROW',VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 01mm 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} {NAME = > f2mm, BLOOMFILTER = > 'ROW',VERSIONS = >' 1' IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 0prima BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536' REPLICATION_SCOPE = >'0'} 2 row (s) in 0.0230 seconds hbase (main): 005test1' 0 > disable 'test1'0 row (s) in 2.2870 seconds hbase (main): 006test1' 0 > alter' test1', {NAME= >'F1' COMPRESSION= > 'Snappy'} Updating all regions with the new schema...1/1 regions updated.Done.0 row (s) in 1.9510 seconds hbase (main): 007 test1'0 row 0 > enable' test1'0 row (s) in 1.2820 seconds hbase (main): 008 desc 0 > desc 'test1'Table test1 is ENABLED Test1 COLUMN FAMILIES DESCRIPTION {NAME = >'F1' BLOOMFILTER = > 'ROW',VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'SNAPPY', MIN_VERSIONS = >' 01mm 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} {NAME = > f2mm, BLOOMFILTER = > 'ROW',VERSIONS = >' 1' IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 0prima BLOCKCACHE = > 'true', BLOCKSIZE = >' 65536' REPLICATION_SCOPE = >'0'} 2 row (s) in 0.0310 seconds hbase (main): 009 test1'0 row 0 > major_compact 'test1'0 row (s) in 0.1380 seconds hbase (main): 010 seconds hbase 0 > desc' test1'Table test1 is ENABLED Test1 COLUMN FAMILIES DESCRIPTION {NAME = >'F1' BLOOMFILTER = > 'ROW',VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'SNAPPY', MIN_VERSIONS = >' 01mm 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} {NAME = > f2mm, BLOOMFILTER = > 'ROW',VERSIONS = >' 1' IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE',DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 0mm > 'true', BLOCKSIZE = >' 65536', REPLICATION_SCOPE = >'0'} 2 row (s) in 0.0260 seconds

But there is no disable table, no major_compact, and the column family compression format has been modified successfully (I don't know why).

Hbase (main): 001desc 0 > test'Table test is ENABLED test COLUMN FAMILIES DESCRIPTION {NAME = > 'fam1' BLOOMFILTER = > 'ROW', VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'NONE', MIN_VERSIONS = >' 0536', BLOCKCACHE = > 'true', BLOCKSIZE = >' 6553613', REPLICATION_SCOPE = >'0'} 1 row (s) in 0.3680 secondshbase (main): 002false', KEEP_DELETED_CELLS 0 > alter 'test' {NAME= > 'fam1' COMPRESSION= > 'LZ4'} Updating all regions with the new schema...1/1 regions updated.Done.0 row (s) in 2.0460 secondshbase (main): 003Updating all regions with the new schema...1/1 regions updated.Done.0 row 0 > desc' test'Table test is ENABLED test COLUMN FAMILIES DESCRIPTION {NAME = > 'fam1' BLOOMFILTER = > 'ROW', VERSIONS = >' 1century, IN_MEMORY = > 'false', KEEP_DELETED_CELLS = >' FALSE', DATA_BLOCK_ENCODING = > 'NONE', TTL = >' FOREVER', COMPRESSION = > 'LZ4', MIN_VERSIONS = >' 0,0,0001 row (s) in 0.0280 seconds = > 'true', BLOCKSIZE = >' 65536mm, REPLICATION_SCOPE = >'0'} 1 row (s)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.