Shulou(Shulou.com)06/02 Report--

Editor to share with you what the default configuration of HBase is, I believe most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Hbase.rootdir

This directory is a shared directory for region server and is used to persist HBase. The URL needs to be 'completely correct' and also contains the scheme of the file system. For example, to represent the'/ hbase' directory in hdfs, namenode runs on port 9090 of namenode.example.org. Then you need to set it to hdfs://namenode.example.org:9000/hbase. By default, HBase is written to / tmp. If you do not change this configuration, the data will be lost when you restart.

Default: file:///tmp/hbase-${user.name}/hbase

Hbase.master.port

The port of HBase's Master.

Default: 60000

Hbase.cluster.distributed

The operation mode of HBase. False is in stand-alone mode and true is in distributed mode. If false,HBase and Zookeeper run in the same JVM.

Default: false

Hbase.tmp.dir

A temporary folder for the local file system. Can be modified to a more persistent directory. (/ tmp will be clear when restarting)

Default: ${java.io.tmpdir} / hbase-$ {user.name}

Hbase.local.dir

As local storage, the path to the local file system.

Default: ${hbase.tmp.dir} / local/

Hbase.master.info.port

HBase Master web interface port. Setting to-1 means you don't want him to run.

Default: 60010

Hbase.master.info.bindAddress

Ports bound by the HBase Master web interface

Default: 0.0.0.0

Hbase.client.write.buffer

The default size of the write buffer for HTable clients. The higher this value, the more memory you need to consume. Because buffering has instances on both the client and the server, it consumes memory in both the client and the server. The benefit is that the number of RPC can be reduced. The memory occupied on the server side can be estimated as follows: hbase.client.write.buffer * hbase.regionserver.handler.count

Default: 2097152

Hbase.regionserver.port

Port bound by HBase RegionServer

Default: 60020

Hbase.regionserver.info.port

Setting the HBase RegionServer web interface binding port to-1 means you don't want to run the RegionServer interface.

Default: 60030

Hbase.regionserver.info.port.auto

Whether Master or RegionServer should dynamically search for an available port to bind the interface. When hbase.regionserver.info.port is already occupied, you can search for a free port binding. This feature is useful in testing. Off by default.

Default: false

Hbase.regionserver.info.bindAddress

The IP address of the HBase RegionServer web interface

Default: 0.0.0.0

Hbase.regionserver.class

The interface used by RegionServer. It is used when the client opens a proxy to connect to the region server.

Default: org.apache.hadoop.hbase.ipc.HRegionInterface

Hbase.client.pause

The usual client pause time. The most common usage is the waiting time of the client before retrying. For example, failed get operations and region query operations are likely to be used.

Default: 1000

Hbase.client.retries.number

Maximum number of retries. The maximum of all operations that need to be retried. For example, get the root region,Get unit value from the root region server, line Update operation, and so on. This is the value of the maximum retry error. Default: 10.

Default: 10

Hbase.bulkload.retries.number

Maximum number of retries. The maximum number of iterations of atomic batch loading attempts. Never give up. Default: 0.

Default: 0

Hbase.client.scanner.caching

The number of rows fetched from the server at a time when the next method of Scanner is called and the value is not in the cache. A higher value means that Scanner is faster, but takes up more memory. When the buffer is full, the next method call becomes slower and slower. Slow to a certain extent may lead to a timeout. For example, it exceeds the hbase.regionserver.lease.period.

Default: 100

Hbase.client.keyvalue.maxsize

The maximum size of a KeyValue instance. This is used to set the upper bound of the size of a single entry in the storage file. Because a KeyValue cannot be split, it can be avoided that the region is indivisible because the data is too large. It would be wise to set it to the number that is divisible by the maximum region size. If set to 0 or less, this check is disabled. Default 10MB.

Default: 10485760

Hbase.regionserver.lease.period

The client lease HRegion server period, that is, the timeout threshold. The unit is milliseconds. By default, the client must send a message within this time, otherwise it is considered dead.

Default: 60000

Hbase.regionserver.handler.count

The number of RPC Server instances offered by RegionServers. For Master, this attribute is the number of handler offered by Master

Default: 10

Hbase.regionserver.msginterval

Interval between RegionServer sending messages to Master (in milliseconds)

Default: 3000

Hbase.regionserver.optionallogflushinterval

The interval between synchronizing Hlog to HDFS. If the Hlog does not accumulate to a certain amount, synchronization will also be triggered when the time is up. The default is 1 second, in milliseconds.

Default: 1000

Hbase.regionserver.regionSplitLimit

When the number of region reaches this value, it will not split again. This is not a hard limit on the number of region. But it plays a guiding role, and at this value it is time to stop splitting. The default is MAX_INT. That is to say, do not stop the split.

Default: 2147483647

Hbase.regionserver.logroll.period

The interval at which the commit log is submitted, regardless of whether enough values have been written.

Default: 3600000

Hbase.regionserver.hlog.reader.impl

The realization of HLog file reader.

Default: org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader

Hbase.regionserver.hlog.writer.impl

The realization of HLog file writer.

Default: org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter

Hbase.regionserver.nbreservationblocks

The number of block stored in memory. When an out of memory exception occurs, we can use these memory RegionServer to clean up before stopping.

Default: 4

Hbase.zookeeper.dns.interface

When using DNS, the network interface name of the IP address that Zookeeper uses to report.

Default: default

Hbase.zookeeper.dns.nameserver

When using DNS, Zookeepr uses the domain name or IP address of DNS, which Zookeeper uses to determine the domain name used to communicate with master.

Default: default

Hbase.regionserver.dns.interface

When using DNS, the network interface name of the IP address that RegionServer uses to report.

Default: default

Hbase.regionserver.dns.nameserver

When using DNS, RegionServer uses the domain name or IP address of DNS, which RegionServer uses to determine the domain name used to communicate with master.

Default: default

Hbase.master.dns.interface

When using DNS, the network interface name of the IP address that Master uses to report.

Default: default

Hbase.master.dns.nameserver

When using DNS, RegionServer uses the domain name or IP address of DNS, which Master uses to determine the domain name used for communication.

Default: default

Hbase.balancer.period

The interval at which the Master executes the region balancer.

Default: 300000

Hbase.regions.slop

When any regional server has average + (average * slop) partitions, rebalancing will be performed. The default is 20% slop.

Default: 0.2

Hbase.master.logcleaner.ttl

The maximum time that Hlog exists in the. oldlogdir folder will be cleaned up by the thread of Master.

Default: 600000

Hbase.master.logcleaner.plugins

A set of LogCleanerDelegat that the LogsCleaner service executes. Values are represented by text spaced by commas. These WAL/HLog cleaners are called sequentially. You can put the ones that are called first first. You can implement your own LogCleanerDelegat, add it to Classpath, and write down the full name of the class here. It is usually added before the default value.

Default: org.apache.hadoop.hbase.master.TimeToLiveLogCleaner

Hbase.regionserver.global.memstore.upperLimit

The maximum of all memtores for a single region server. Beyond this value, a new update operation is suspended, forcing the flush operation.

Default: 0.4

Hbase.regionserver.global.memstore.lowerLimit

When the flush operation is enforced, flush stops when it falls below this value. The default is 35% of the heap size. If this value is the same as hbase.regionserver.global.memstore.upperLimit, it means that when the update operation is suspended due to memory constraints, the flush will be executed as little as possible.

Default: 0.35

Hbase.server.thread.wakefrequency

The sleep interval in milliseconds for service work. Can be used as the sleep interval for service threads, such as log roller.

Default: 10000

Hbase.server.versionfile.writeattempts

The number of attempts to write the version file before exiting. Each attempt is spaced by hbase.server.thread.wakefrequency milliseconds.

Default: 3

Hbase.hregion.memstore.flush.size

When the size of the memstore exceeds this value, it will be flush to disk. This value is checked by a thread every other hbase.server.thread.wakefrequency.

Default: 134217728

Hbase.hregion.preclose.flush.size

When the size of the memstore in a region is greater than this value, we trigger the close. It will first run the "pre-flush" operation, clean up the memstore that needs to be closed, and then take the region offline. When a region goes offline, we can't do any more writes. If a memstore is very large, the flush operation will take a lot of time. The "pre-flush" operation means that the memstore is emptied before the region goes offline. In this way, when the close operation is finally performed, the flush operation will be very fast.

Default: 5242880

Hbase.hregion.memstore.block.multiplier

If memstore has the size of a hbase.hregion.flush.size that is a multiple of hbase.hregion.memstore.block.multiplier, the update operation is blocked. This is to prevent the loss of control caused by the peak of update. If there is no upper bound, flush will take a long time to merge or split, and in the worst case, an out of memory exception will be thrown. (translator's note: the speed of memory operation does not match the disk, we need to wait. The original text seems to be incorrect)

Default: 2

Hbase.hregion.memstore.mslab.enabled

Experience feature: enable memStore to allocate local buffers. The purpose of this feature is to prevent excessive fragmentation of the heap when there is a large write load. This reduces the frequency of GC operations. (GC may Stop the world) (the principle of implementation is equivalent to pre-allocated memory, not every value has to be allocated from the heap)

Default: true

Hbase.hregion.max.filesize

Maximum HStoreFile size. If the HStoreFile growth of a column family reaches this value, the Hegion will be cut into two. Default: 10g.

Default: 10737418240

Hbase.hstore.compactionThreshold

When a HStore contains more than this value of HStoreFiles (each memstore flush produces a HStoreFile), a merge operation is performed to write the HStoreFiles into one. The higher this value, the longer it takes to merge.

Default: 3

Hbase.hstore.blockingStoreFiles

When a HStore contains more than this value of HStoreFiles (each memstore flush produces a HStoreFile), a merge operation is performed, and the update blocks until the merge is completed until the value of hbase.hstore.blockingWaitTime is exceeded.

Default: 7

Hbase.hstore.blockingWaitTime

The number of StoreFile limited by hbase.hstore.blockingStoreFiles will cause update blocking, which is used to limit the blocking time. When this time is exceeded, HRegion will stop blocking update operations, but the merge has not yet been completed. The default is 90s.

Default: 90000

Hbase.hstore.compaction.max

The maximum number of HStoreFiles merged per "small".

Default: 10

Hbase.hregion.majorcompaction

The interval between the major compactions of all HStoreFile in a Region. The default is 1 day. Setting to 0 disables this feature.

Default: 86400000

Hbase.storescanner.parallel.seek.enable

Allows StoreFileScanner to search StoreScanner in parallel, a feature that reduces latency under certain conditions.

Default: false

Hbase.storescanner.parallel.seek.threads

When the parallel search feature is turned on, the default thread pool size.

Default: 10

Hbase.mapreduce.hfileoutputformat.blocksize

HFileOutputFormat in MapReduce can write storefiles/hfiles. This value is the minimum blocksize for hfile. Usually when HBase writes Hfile, bloocksize is determined by table schema (HColumnDescriptor), but when mapreduce writes, we cannot get the blocksize in schema. The smaller the value, the larger your index, and the smaller the data you need to fetch for random access. If your cell is small and you need faster random access, you can lower this value.

Default: 65536

Hfile.block.cache.size

The percentage of block cache allocated to HFile/StoreFile to the largest heap (- Xmx setting). The default 0.25 means an allocation of 25%, and a setting of 0 is disabled, but it is not recommended.

Default: 0.25

Hbase.hash.type

The hash algorithm used by the hash function. You can choose two values: murmur (MurmurHash) and jenkins (JenkinsHash). This hash is for bloom filters.

Default: murmur

These are all the contents of the article "what is the default configuration of HBase?" Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.