Shulou(Shulou.com)05/31 Report--

How to achieve Zookeeper dynamic update server list, many novices are not very clear about this, in order to help you solve this problem, the following editor will explain in detail for you, people with this need can come to learn, I hope you can gain something.

Inside the zookeeper is a cluster, the primary node is elected, and the outside looks like there is only one, holding a copy of state data. When doing distributed application coordination, it can reduce the difficulty of development.

With high availability, loosely coupled interaction.

1. Zookeeper's API interface String create (String path, byte [] data, List acl, CreateMode createMode) Stat exists (String path, boolean watch) void delete (String path, int version) List getChildren (String path, boolean watch) List getChildren (String path, boolean watch) Stat setData (String path, byte [] data, int version) byte [] getData (String path, boolean watch, Stat stat) void addAuthInfo (String scheme, byte [] auth) Stat setACL (String path, List acl, List acl) String path (List acl, List acl)

Generally speaking, the data kept by zookeeper is no more than 1m. It is mainly to save some configuration information, and the main feature is to monitor the real-time update of data.

Second, main applications:

1. Cluster management: it is stipulated that the lowest number is master, so when we monitor the SERVERS node, we get a list of servers. As long as all cluster machines logically assume that the lowest numbered node is master, then master is selected. When the master goes down, the corresponding znode disappears, and then the new server list is pushed to the client, and then each node logically thinks that the lowest numbered node is master. In this way, dynamic master elections can be achieved.

2, configuration management: it is very common in a distributed application environment, for example, the same application system needs more than one PC Server to run, but some configuration items of the application system they run are the same. If you want to modify these same configuration items, you must modify the PC Server of each running application system at the same time, which is very troublesome and error-prone. Save the configuration information in a directory node of Zookeeper, and then monitor the status of the configuration information by all the application machines that need to be modified. once the configuration information changes, each application machine will receive a notification from Zookeeper, and then obtain the new configuration information from Zookeeper and apply it to the system.

3. Shared locks: easy to implement in the same process, but not easy to implement across processes or between different Server. However, Zookeeper is easy to implement this function, which also requires the Server to acquire the lock to create an EPHEMERAL_SEQUENTIAL directory node, and then call the getChildren method to get whether the smallest directory node in the current directory node list is a self-created directory node. If it is created by itself, then it acquires the lock. If not, it calls the exists (String path, boolean watch) method and monitors the changes in the list of directory nodes on the Zookeeper until the node it creates is the lowest numbered directory node in the list, thus obtaining the lock and releasing the lock, as long as it deletes the directory node it created previously.

4. Queue management: Zookeeper can handle two types of queues: when the members of a queue are gathered together, the queue is available, otherwise it is always waiting for all the members to arrive. This is a synchronous queue. Queues perform queue entry and dequeue operations according to FIFO, such as implementing the producer and consumer models.

Use eclipse to connect to zookeeper

In eclipse, we can import the package we need, and then add, delete, modify and check the node. This can be done when connecting, here I use 3 zookeeper to operate. Ubuntu1,2,3 are all hostnames.

ZooKeeper zk = null; @ Before public void init () throws Exception {zk = new ZooKeeper ("ubuntu2:2181,ubuntu1:2181,ubuntu3:2181", 5000, new Watcher () @ Override public void process (WatchedEvent event) {System.out.println (event.getPath ()); System.out.println (event.getType ());}});}

Create a node: here I create a permanent node, which can be divided into a temporary node and a permanent node in the zookeeper node. Create an eclipse node under the directory. The encoding format of the content is utf-8,Ids, which means permission control. Here, I use open ACL permission control. Finally, the stream needs to be turned off.

@ Test public void testZkNode () throws Exception {String path = zk.create ("/ eclipse", "instruction Exchange Technology" .getBytes ("utf-8"), Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT); System.out.println ("created a permanent node:" + path); zk.close ();}

Then there is the registration listener. After all, a very important function of zookeeper is to monitor the status of the entire service.

@ Test public void testGet () throws Exception {/ / listener registration can only take effect once byte [] data = zk.getData ("/ eclipse", true, new Stat ()); System.out.println (new String (data, "utf-8")); Thread.sleep (Long.MAX_VALUE);}

Call execution in the main method.

@ Test public void testSet () throws UnsupportedEncodingException, KeeperException, InterruptedException {zk.setData ("/ eclipse", "who is the hero" .getBytes ("utf-8"),-1); zk.close ();} IV, dynamic server

Since I mainly share how to dynamically monitor the online and offline of the server on the client, let's write a server process first.

First of all, we need to define the node information we need later, run it on the client of zookeeper, and create a grpnode node for our subsequent operations.

Private ZooKeeper zk; private String groupNode = "grpnode"; private String subNode = "sub"; / / register the information public void connectZK (String name) throws KeeperException, InterruptedException, IOException {zk = new ZooKeeper ("ubuntu2:2181,ubuntu1:2181,ubuntu3:2181", 5000, new Watcher () {/ / callback method @ Override public void process (WatchedEvent event) {}} when the event occurs) with zookeeper String path = zk.create ("/" + groupNode + "/" + subNode, name.getBytes (), Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL_SEQUENTIAL); System.out.println ("server comes online and creates a child node:" + path);}

Next comes the processing of zookeeper's default business logic, which is finally called in the main method. Of course, you can also pack this into a jar package and put it on hadoop to run.

/ / Business processing logic public void handle () throws Exception {Thread.sleep (Long.MAX_VALUE);} public static void main (String [] args) throws Exception {if (args.length==0) {System.err.println ("the number of parameters is incorrect, please add the server name as a parameter to start."); System.exit (1);} / register this server information AppServer server = new AppServer () with zookeeper; server.connectZK (args [0]) Server.handle ();}

5. Dynamic client

Once the server is written, we need a client to listen to the server's online and offline operations. Also use a zookeeper listening callback method. Once the server changes, it can be dynamically monitored here. The main purpose is to monitor the changes of parent and child nodes.

Private volatile List servers; private ZooKeeper zk / / use the listener function of zk to trigger the action of server update public void connectZK () throws IOException, KeeperException, InterruptedException {zk = new ZooKeeper ("ubuntu2:2181,ubuntu1:2181,ubuntu3:2181", 5000) New Watcher () {/ / callback method @ Override public void process (WatchedEvent event) {if ("/ grpnode" .equals (event.getPath ()) & & event.getType () = = EventType.NodeChildrenChanged) {/ / trigger the action try {updateServerList () that updates the server list } catch (Exception e) {e.printStackTrace ();}); updateServerList ();}

Get the list of servers dynamically, which is mainly to listen for changes in the parent and child nodes.

/ / dynamically get the list of servers public void updateServerList () throws KeeperException, InterruptedException, UnsupportedEncodingException {ArrayList serverList=new ArrayList (); / / listen to the child node and register the listener List childer=zk.getChildren ("/ grpnode", true) to the parent node; / / traverse the child node for (String child:childer) {byte [] data=zk.getData ("/ grpnode/" + child,false, new Stat ()); String server=new String (data, "utf-8") / / store the obtained server name into list serverList.add (server);} / / put the temporary list into the global list servers=serverList; System.out.println ("the latest online server is:" + serverList);}

Finally, we are most familiar with the main method.

/ / public void handle () throws InterruptedException {Thread.sleep (Long.MAX_VALUE);} public static void main (String [] args) throws IOException, InterruptedException, KeeperException {AppClient client=new AppClient (); client.connectZK (); client.handle () } Zookeeper, as a sub-project of Hadoop project, is an indispensable module of Hadoop cluster management. It is mainly used to control the data in the cluster, such as managing NameNode in Hadoop cluster, and state synchronization between Master Election and Server in Hbase. Zoopkeeper provides a good mechanism for distributed cluster management, which is a data structure based on a hierarchical directory tree, and manages the nodes in the tree effectively, so that you can design a variety of distributed data management models. Will it help you to see the above? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.