Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "storm log how to debug". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

How do I use a custom profile?

You may need to use your own configuration file when developing storm applications. Storm.yaml can only be used by storm's own configuration system. Ini4j is recommended for the most widely used ini profile in the background. It is a lightweight ini profile reader with an easy-to-understand api.

Read the configuration file in the prepare method of spout and bolt. The initialization of any resource class needs to be placed in the prepare method (such as database connections, reading file index classes, etc.).

Considerations in program deployment

I package the storm program, please do not type in the dependency, the dependency needs to be deployed separately

II you need an effective mechanism to distribute the program's dependency packages to all storm nodes (can upload to nimbus and do directory synchronization through rsync)

III the collection of dependencies you need to distribute = all the sets of dependencies for your program? The intersection of the storm dependency package and your dependency package (otherwise topology will fail to initialize)

The content and path of the IV custom configuration file need to be exactly the same on all supervisor nodes, the path is inconsistent, topology initialization fails, the content is inconsistent, and the behavior of the program is undefined.

How do I use the logging system? How to debug?

After a lot of hard work, your program was compiled successfully. However, this does not mean that it will run according to the behavior you set. How to debug becomes a problem.

The most basic and primitive idea is log. Log in all the places you think may go wrong (especially in the prepare method, what is usually done here is the initialization of resource classes, and if they fail, topology will fail to initialize, which will help you quickly locate the cause of the error).

Storm integrates with log4j by default. You can find the configuration file for log4j in the $STORM_HOME/log4j/ directory. Control the behavior of log4j by adjusting it.

Don't rush to deploy your program in the real world. If you want to test it, you should first observe it in Local mode mode. If the results of the test under local mode are as expected, and the error in the real world should generally be caused by non-code factors. You need to check the node's resource files, custom configuration files, dependency packages, and so on. In addition, under local mode, all log printing information is output directly to the screen, so that you can see it more clearly.

The structure of the storm log is described below:

All log files are stored in the $STORM_HOME/logs/ path.

On the nimbus node: nimbus.log records the output information during nimbus startup, including startup time and printing information during the initialization of each worker and task, and so on. Ui.log records the output information during the startup process of the storm monitor program, including the startup time, and so on.

On the supervisor node: supervisor.log records the relevant startup information of supervisor. Worker-XXX (a supervisor node usually has multiple worker deployed) records the output of message delivery and task execution (that is, the log print part of your code). One of the design goals of storm is to make task deployment transparent to users. The result is that when you need to look at the log output of a task, you don't know which log file on which node to look for that information. Therefore, it is strongly recommended that you debug your program under local mode, and then do log alarms in the real world.

This is the end of the content of "how to debug storm logs". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.