Shulou(Shulou.com)06/01 Report--

Manifesto of the Lady

This article will introduce some data structures involved in Select query request results in Influxdb. Students who are not familiar with Influxsql query statements can learn about it first:

Https://docs.influxdata.com/influxdb/v1.7/query_language/data_exploration

Data structure

1 Series

Series is actually a combination of measurement and tags, and tags is the map of tag key and tag value. How does the id of Tags come into being? in fact, it encodes tag key and tag value to

[] byte: agkey1\ 0tagkey2\ 0...\ tagvalue1\ 0tagvalue2\ 0...

Implement the encodeTags defined in query/point.go.

2 Row

Row represents each row in the query result set, where the Values representation is the collection of returned Fields

3 bufFloatIterator

BufFloatIterator is equivalent to a linked list element in c, itr points to the next element, and buf represents the current element, that is, the iterator of a linked list of type FloatPoint.

FloatPoint

FloatPoint is defined in query/point.gen.go to represent a piece of data whose field is of type float

Next implementation

If the value of the current Iterator is not empty, the current buf is returned. If the current value is empty, itr.itr.Next () is returned, that is, the next element pointed to.

Unread: iterator fallback operation

4 floatMergeIterator

FloatMergeIterator combines multiple floatIterator

Because you want to do merge, you need to sort all the Interator elements it manages, and here you use golang's container/heap for stacking.

Because you want to use the container/heap of golang to manage, you need to implement the interfaces specified below

FloatMergeHeap in the definition of floatMergeIterator implements the above interface. Let's take a look at the implementation of the comparison function, which is actually FloatPoint.

The priority of the comparison is first the measurement name of FloatPoint, then tagset id, and finally time. We can know by comparing this function.

structure

The realization of Next function

A long picture

Combined with the above Less function, for all FloatPoint, the smallest unit of sorting is Window (composed of measurement name, tagset id, time window), and FloatPoint with the same Window attribute is no longer sorted. If it is traversed according to the upgrade rule, the traversal result is arranged from small to large by Window, but the time of multiple Point within the same Window is not necessarily from small to large.

five

FloatSortedMergeIterator

At the same time, it also uses heap in golang/container. Compared with floatMergeIterator, it implements the sort traversal of all Point. Let's take a look at how it is implemented.

Pop function:

All the FloatPoint contained in Iterator are sorted and there is no concept of Window.

6 floatIteratorScanner

FloatIteratorScanner scans the value of floatIterator into map.

ScanAt

Find the Point that satisfies the condition in floatIterator if ts, name and tags are all equal, so it is relatively simple to implement.

7 floatParallelIterator

Call floatIterator.Next in a separate goroutine to get the FloatPoint, and then write it to chan:

When in use, call Next to read the data from the Chan above:

8 floatLimitIterator

Limit the number of Point read per window

Next

9 floatFillIterator

Group by time fill (...) running in select, in the current window of interval, if no value is queried, the corresponding value is generated using the corresponding filling rule.

For more information, please see:

Group-by-time-intervals-and-fill

Definition:

10 floatInterruptIterator

After each traversal of N pieces of data, check whether the traversal needs to be interrupted.

Definition:

Next

11 floatReduceFloatIterator

Reduce the data in each interval

Definition:

Reduce ()

Returns the processed points with long function but simple logic

12 CallIterator

CallIterator implements Iterator of aggregate functions: count, min, max, sum, first, last, mean, distinct,Median.... It mainly uses a series of ReduceIterator introduced above, provides the corresponding Reducer, and implements the two functions AggregateFloat and Emit.

13 IteratorOptions

Some configuration options used when building Iterator, which contain a lot of content.

Definition:

14 Cursor

After select, you will get this cursor, which is used to traverse the query results.

Definition:

Scan

15 floatIteratorMapper

* IteratorMapper series, the main function is to traverse cursor

Define

Let's take a look at the Next interface and scan the current cursor to return FloatPoint

Summary

These are some of the data structures involved in the results of Influxdb's select query request

Discussion on HULK first-line Technology

The official account of technology sharing created by the 360Cloud platform team covers many technical areas, such as cloud computing, database, big data, monitoring, pan-front-end, automated testing, and so on. Through the accumulation of solid technology and rich first-line practical experience, to bring you the best technology sharing.

Original link: https://mp.weixin.qq.com/s/GzfNWEZ4wOOFVd8yrQpQOQ

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.