Shulou(Shulou.com)06/03 Report--

Today, compared with tonight's kafka+Spark streaming+Redis real-time data analysis system (https://www.iteblog.com/archives/1378)

The main purpose is to integrate kafka and Spark,Redis.

Redis has never been used, so it is quite difficult. Sure enough, there is no problem in the front, but there is a problem in the redis part of the back:

The class GenericObjectPoolConfig was not found.

Baidu has been unable to find a solution for a long time. Later, I realized that this should be a maven library package, so, in

Http://maven.outofmemory.cn/org.apache.commons/commons-pool2/2.2/

On this page, you find the configuration dependency information for maven:

Org.apache.commons

Commons-pool2

2.2

Add decisively to pom.xml, and after some action, OK!

By the way, I studied this GenericObjectPool gadget and got a preliminary understanding that GenericObjectPool is an object pool provided by apache.

Object pool creation (see GenericObjectPool):

Public GenericObjectPool (PoolableObjectFactory factory, GenericObjectPool.Config config): this method creates an GenericObjectPool instance. The GenericObjectPool class has implemented all the core operations related to object pooling, and developers can use it through inheritance or encapsulation. Through this constructor, we can clearly see that a Pool needs to specify a PoolableObjectFactory instance and the Config information of this object pool. PoolableObjectFactory is mainly used to "create new objects". For example, when there are not enough objects in the object pool, you can use the PoolableObjectFactory.makeObject () method to create objects and deliver them to Pool management.

This constructor instantiates an LinkedList as an object pool container for accessing objects. In addition, it decides whether to start a background thread based on the value of timeBetweenEvictionRunsMillis, which is used to periodically scan the list of objects in the pool to detect whether the "objects in the object pool" idle has reached the threshold, and if so, remove the object.

This article first records several related web pages for a rainy day:

1. GenericObjectPool source code analysis:

Http://www.cnblogs.com/chenjian5/p/5149312.html

2. Javadoc of GenericObjectPool

Http://commons.apache.org/proper/commons-pool/api-1.6/org/apache/commons/pool/impl/GenericObjectPool.html

3. Optimize the use of GenericObjectPool object pool

Http://www.jianshu.com/p/397169e211de

4. The use of apache-common pool

Http://www.open-open.com/lib/view/open1415453575730.html

Of course, what is most worth preserving is maven's central repository: http://maven.outofmemory.cn/.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.