Shulou(Shulou.com)06/01 Report--

This article introduces how Zookeeper realizes simple configuration center in big data development. The content is very detailed. Interested friends can refer to it for reference. I hope it can help you.

Big Data Development-Zookeeper Implementation Simple Configuration Center

Why Zookeeper?

Zookeeper is a distributed, open-source distributed coordination service, often used in Ha architecture, such as Hadoop, Hbase, it is a distributed application to provide consistency services software, providing functions such as: configuration maintenance, domain name services, distributed synchronization, registry, etc., as a configuration center, what is needed is a stable global consistent configuration center, and Zk itself provides primitives to meet the requirements.

1. realization idea

1. To create a Web project, the database connection pool information is handed over to Zookeeper Configuration Center for management, that is, the configuration parameters are placed in the node information of Zk.

2. When the project starts, pull Mysql information parameters from Zookeeper.

3. When the database connection pool information in Zk is modified, it is automatically perceived, and the previous connection pool is released correctly to create a new connection pool.

2. realization process

2.1 Start the springboot project and pull configuration from zk

/**

* 1. Start the web container

* 2. Initialize zookeeper

* 3. Configure database connection pools

**/

public static void main(String[] args) {

SpringApplication.run(Application.class, args);

initZk();

configHikariSource();

}

//Initialize zk

private static void initZk() {

zkClient = new ZkClient("centos:8521");

zkClient.setZkSerializer(new ZkStrSerializer());

zkClient.subscribeDataChanges("/jdbc", new IZkDataListener() {

public void handleDataChange(String path, Object data) {

System.out.println(path + " data is changed, new data " + data);

hikariDataSource.close();

configHikariSource();

}

public void handleDataDeleted(String path) {

System.out.println(path + " is deleted!! ");

hikariDataSource.close();

}

});

}

2.2 Get database connection pool information to access mysql

Serialize the data in zk, convert it to strings, use HiKari tools to establish connections, manipulate databases

/**

* Configure database connection pools

*

* 1. Get configuration information from zookeeper

* 2. Update hikari configuration

* 3. Execute test sql

*/

private static void configHikariSource(){

JDBCConfig myConfig = getJDBCConfig();

updateHikariConfig(myConfig);

try {

executeTestSQL();

} catch (SQLException e) {

e.printStackTrace();

}

}

2.3 Listens on zk, changes node data, releases and creates new database connections

//update database information

private static void updateHikariConfig(JDBCConfig myConfig) {

HikariConfig config = new HikariConfig();

config.setJdbcUrl(myConfig.getUrl());

config.setUsername(myConfig.getUsername());

config.setPassword(myConfig.getPassword());

config.addDataSourceProperty( "driverClassName" , myConfig.getDriver());

config.addDataSourceProperty( "cachePrepStmts" , "true" );

config.addDataSourceProperty( "prepStmtCacheSize" , "250" );

config.addDataSourceProperty( "prepStmtCacheSqlLimit" , "2048" );

hikariDataSource = new HikariDataSource(config);

}

How to implement the simple configuration center of Zookeeper in big data development is shared here. I hope the above content can be of some help to everyone and learn more knowledge. If you think the article is good, you can share it so that more people can see it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.