Shulou(Shulou.com)06/02 Report--

This article mainly explains the "introduction to the storage mode of lucene inverted index". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "introduction to the storage mode of lucene inverted index".

In talking about the storage mode of lucene inverted index, it only describes the storage of location-related information of inverted index, but does not specify how to design its index if random access to location information is needed. Lucene adopts the method of multi-level jump list. First, let's talk about the basic idea of jump list (actually mentioned in the previous article). Suppose that given a pile of ordered numbers, and the amount of data is so large that it cannot be put in memory, if you want to quickly and randomly access one of these values, one way is to record the corresponding values and corresponding file pointers to these numbers every other number, such as 1000. Then load these values and the corresponding file pointers into memory and use the binary search method to find the starting address of the data block in which you want to find the values, and then traverse and compare 1000 records in turn or load them into memory using binary lookup. These values and file pointers are also called first-level jump tables.

If the amount of data of the first-level jump table is still very large, then we have to establish another layer of jump table on this basis, and so on, there will be a multi-level jump table. It is worth mentioning that the more levels, the better, because the more levels, the more searches, and the default maximum level for lucene is 10.

The above picture is the official lucene diagram (a word represents the inverted position index), d represents the document, x represents every 128documents compressed file pointer is also the first level index records the corresponding document ID and the file pointer, c is the second level and the third level respectively. In this way, the more complex index structure in the code implementation is really very clever in the lucene implementation, because the total level can be calculated in advance, and then the document level can be calculated while writing. If you are interested, you should look at the code.

Thank you for your reading, the above is the "introduction to the storage of lucene inverted index", after the study of this article, I believe you have a deeper understanding of the storage of lucene inverted index, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.