Shulou(Shulou.com)06/03 Report--

   OpenTSDB currently supports Apache HBase as its primary storage backend. As of version 2.3 OpenT SDB can also use Bigtable on the Google cloud. Select the HBase link below to store the schema or Bigtable to find configurations and settings for use in the cloud.

HBase SchemaData Table Schema

All data points of    OpenTSDB are saved in a separate sea scale named tsdb by default. It takes advantage of HBase sorting and region distribution. All values are stored in the column family t.

   Row Key-Row key is a byte array of optional salt values, metric UID, timestamps, and UID of tagk/v pairs: [salt] […] . By default, UID is 3 bytes encoded.

   has enabled the salting feature since OpenTSDB version 2.2, and the first byte (or multiple bytes) is a hash salt value ID to better distribute data across multiple regions or region servers.

The    timestamp section is a Unix epoch value encoded in 4 bytes in seconds. One line per hour, reflected by the timestamp in each line. So each timestamp will be normalized (normalized) to the value of hour, for example: 2013-01-01 08:00:00. This is to avoid filling too many data points in a single row, as this can affect the distribution of region. In addition, because HBase sorts Row Key, data from the same metric and time period, but data with different tags, will also be grouped together for efficient query. (does the combination of different tags refer to the same Region? )

The example row key with no salt value for    is expressed in hexadecimal as follows:

00000150E2270000000100000100000150E2270000000100000100000200000400000150E2270000000100000200000150E2270000000100000300000150E2351000000100000100000150E2351000000100000100000200000400000150E2351000000100000200000150E2351000000100000300000150E2432000000100000100000150E2432000000100000100000200000400000150E2432000000100000200000150E24320000001000003

   where

00000150E22700000001000001'-'metric time tagk tagv

   represents a single indicator, but four time series over a three-hour period. Note that a time series consists of two sets of labels:

00000150E22700000001000001000002000004'-''-'metric time tagk tagv tagk tagv

The    tag name (tagk) is sorted in subtitle order before it is stored, so the host tag will always appear before owner in rowkey/TSUID.

Data point column

   is by far the most common column data point. These are the actual values recorded when the data is sent to TSD for storage.

The    Column Qualifier-qualifier consists of 2 or 4 bytes that encode the offset of the base time of the row and the identification that determines whether this value is an integer or decimal (decimal) value. Qualifier encodes the offset of the row base time and the format and length of the stored data.

The column of    with 2 bytes of Qualifier is offset in seconds. The first 12 bits of Qualifier (bit) represent an integer, which is an increment of the timestamp in row key. For example, if row key is normalized to 1292148000 and a data point comes at 1292148123, the record increment will be 123. The last four digits (bit) are format identifiers.

The column of    with 4 bytes of Qualifier is offset in milliseconds. The first 4 bits of Qualifier (bit) will be set to hexadecimal 1 or F. The next 22-bit (bit) encodes offset as a unsigned integer. The next 2 bits (bit) are reserved, and the last 4 bits (bit) are format identifiers.

The last 4 bits (bit) of any column type of    describe the stored data. The first bit is an identification indicating whether the value is an integer or a floating-point number. The value 0 identifies an integer and 1 represents a floating point number. The last three bits (bit) represent the length of the data, with an offset of 1. A value of 000 represents a 1-byte value, while 010 represents a 2-byte value. The length must reflect the values in 1, 2, 4 or 8. Anything else indicates an error.

   for example: 0100 indicates that the column value is an 8-byte signed integer. 1011 indicates that the column value is a 4-byte floating point value. So at 1292148123, the data point of Qualifier will have an integer value of 4294967296, and there will be Qualifier of 0000011110110100 or hexadecimal 07B4.

   Column Values-1 to 8 byte encoding specified by the Qualifier identity.

Test example:

{"metric": "sys.cpu.system", "tags": {"host": "web02", "dc": "lax"}, "timestamp": 1525336162539, "value": 4} query result: RowKey {metricUid='1', metricName='sys.cpu.system', ts=1525334400, datetime='2018-05-03 16 purse 0000, tags= [TagPair {tagKUid='1', tagK='host', tagVUid='3', tagV='web02'}, TagPair {tagKUid='2', tagK='dc', tagVUid='6' TagV='lax'}]} family: t qualifier: f6b93ac0 value: 4 calculation: 0xf6b93ac0 = 1111 0110 1001 0011 1010 1100 0000 (binary) before 4bit:111122bit: 0110 1011 1001 0011 1010 11 = 1762539 (decimal) last 4bit:0000 so: timestamp = 15253344001000 + 1762539 = 1525336162539

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.