Shulou(Shulou.com)06/01 Report--

How to use zookeeper in SolrCloud? in view of this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

Nodes of solr in zookeeper

1. Aliases.json has another ingenious use for colletion aliases (solrcloud's build search separation), and then write a blog description later.

2. Clusterstate.json important information file. Contains the specific description information of colletion and shard replica.

3. Live_nodes. The following are all instantaneous zk nodes, which represent the nodes in the current surviving solrcloud.

4. Overseer, an important role in solrcloud. Below are three important distributed queues that represent task queues for solrcloud-related zookeeper operations to be performed. Collection-queue-work stores special operations related to collection, such as createcollection, reloadcollection,createalias,deletealias, splitshard, etc.

5. Queue stores all operations that have nothing to do with collection, such as deletecore,removecollection,removeshard,leader,createshard,updateshardstate, as well as changes in the state of nodes (down, active, recovering).

6. Queue-work is a temporary queue that refers to messages that are being processed. The operation is saved to / overseer/queue, moved to / overseer/queue-work when overseser is processed, and the message is deleted from / overseer/queue-work after processing. If the overseer dies halfway, the newly elected overseer will choose to finish the operation in / overseer/queue-work and then process the operation in / overseer/queue.

Note: all the child nodes stored in the above queue are of type PERSISTENT_SEQUENTIAL.

7. Overseer_elect, for the election work of overseer

8. Colletcion, which stores some simple information about the current collection (the main information is in clusterstate.json). The following leader_elect is naturally used for leader election of replica sets in shard in collection.

Zk Writing process of Overseer

When looking at the official documentation of solrcloud, there is very little description of the role of overseer. I believe that many developers who have successfully configured solrcloud are not aware of the existence of this role.

Overseer, as its name implies, is a role that takes care of the overall situation and does the overall control work. Reflected in the related operations between the code and zk, that is, most of the write operations in zookeeper are handled by overseer, and the contents of the two zk nodes clusterstate.josn and aliases.json are well maintained. It is different from our "who creates, who modifies" approach. The actions initiated by each solr node will publish to the corresponding queue under the / overseer node, and then the overseer will go to some distributed queues to fetch the operation information, make corresponding zk modifications, and update the relevant specific status information in the entire solrcloud to the cluseterstate.json. Finally, an operation will be deleted from the queue, indicating that the operation is completed.

Take, for example, a solr node that marks its status as down. The node will publish the relevant information about this "state" operation into / overseer/queue. It is up to Overseer to get this operation from it, and then write the information with node state as down to clusterstate.json. Finally, delete this node in queue.

Of course, the role of overseer is elected internally by zookeeper in solrcloud.

General zk read operation

Solr puts the most important and informative content in cluseterstate.json. This reduces the number of zk nodes that a normal solr node needs to focus on. In addition to clusterstate.json, when an ordinary solr node needs the overall state of the current collection, it will also get the information in the / live_nodes of the zk, according to the information in the live_nodes, know the surviving node of the collection, and then obtain the information of the node from the clusterstate.json.

In fact, this kind of treatment is also easy to understand. If an solr node goes offline abnormally, there may not necessarily be a change in the clusterstate.json, but the zk node corresponding to the node in / live_nodes disappears (because it is instantaneous).

This is the answer to the question about how to use zookeeper in SolrCloud. I hope the above content can be of some help to you. If you still have a lot of doubts to solve, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.