Shulou(Shulou.com)05/31 Report--

This article will explain in detail what are the common problems and solutions of Storm. The content of the article is of high quality, so the editor will share it with you for reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

1. Configuration of JDK environment variables in storm cluster

Problem analysis: after configuring JDK in the linux system, the / etc/ profile value is generally modified to configure the environment variables, but there are problems when installing the storm cluster

Problem solving: you need to add environment variables to the / etc/.bashrc file or the installed JDK cannot be used.

2. Issues related to supervisor

1) the supervisor name is the same after installation and the error report is started.

Problem analysis: there is a line of code in supervisor.cji as follows:

This code looks for the native hostname when starting supervisor.

Problem solving: modify the hostname of the machine every day to correspond to it.

2) java.lang.UnsatisfiedLinkError exception occurs when Supervisor is started

The specific error messages are as follows:

Java.lang.UnsatisfiedLinkError:/usr/local/lib/libjzmq.so.0.0.0: libzmq.so.1: cannot open shared objectfile: No such file or directory exception occurs when Supervisor is started.

Problem analysis: the zmq dynamic link library was not found.

Problem solving 1: configure the environment variable export LD_LIBRARY_PATH=/usr/local/lib

Problem solving 2: edit the / etc/ld.so.conf file, add a line: / usr/local/lib, and execute it again

Restart Supervisor using the sudo ldconfig command

3. Serialize log4j.Logger exception when topologies is released

Problem analysis: the log system fails to handle serialization operations correctly

Problem resolution: using slf4j to replace log4j log jar package processing

4. The following exception occurred while submitting the topology:

Exception in thread "main" java.lang.IllegalArgumentException: xxx host is not set at backtype.storm.utils.NimbusClient. (NimbusClient.java:30) at backtype.storm.utils.NimbusClient.getConfiguredClient (NimbusClient.java:17) at backtype.storm.StormSubmitter.submitJar (StormSubmitter.java:78) at backtype.storm.StormSubmitter.submitJar (StormSubmitter.java:71) at backtype.storm.StormSubmitter.submitTopology (StormSubmitter.java:50)

Problem analysis: there is no problem with starting nimbus. The exception is caused by incorrect conf_dir path setting.

Problem solving: modify the bin/storm script and add the following code:

CONF_DIR = STORM_DIR + "/ conf"

5. When using the following version of storm0.7, outofmenmory will appear for a long time

Problem analysis: not triggering a tuple lower than the storm0.7 version will create a hashmap to store the message tree structure where the tuple is located, which will cause great pressure on the gc for a long time and lead to the emergence of outofmemory

Problem resolution: upgrading the storm version will resolve the problem.

6. An exception was reported when storm connected to mysql database.

Connecting to a remote mysql database Times has the following error:

Message from server: "Host FILTER" is not allowed to connect to this MySQL server

Problem analysis: it may be caused by not giving IP access to the Mysql database.

Problem resolution: log in to the mysql database and execute the following command:

Grant all on *. * to root@'%' identified by "123456"

This operation is to grant access to any IP address (in the command *. * is wildcard with any IP, you can specify IP username: root password: 123456)

About Storm frequently asked questions and solutions what is shared here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.