Shulou(Shulou.com)06/03 Report--

Editor to share with you the example analysis of Hadoop remote debug debugging, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

1. Hadoop remote debug

When hadoop starts the service, it is finally started through the java command, which is essentially a java program. Debug is a very important tool when studying source code, but hadoop is compiled code and runs directly in liunx. It can not be run directly in debug in tools such as eclipse like ordinary programs.

For the above situations, java provides a way of remote debug.

This method requires the following parameters to be added when the java program starts:-agentlib:jdwp=transport=dt_socket,server=y,address=6603,suspend=y

Where transport is the data transmission method for both parties, server indicates whether to listen to the debugging request of debuger, address is the listening port, and suspend indicates whether to wait for startup.

So in order to be able to remotely debug hadoop, you need to modify the startup script of hadoop.

It is recommended to modify the hdfs file in the bin directory.

The modification method is as follows:

As shown in the figure above, the commented-out statement contains the parameters of remote debugging, and what is not commented is its normal startup parameters. Another benefit of the modification here is that remote debug settings can be made for the specified service. The figure above shows remote debug settings for namenode. To remotely debug other services, you only need to modify the parameters corresponding to the service name in the hdfs file of the corresponding node.

After the modification is completed, start it using the script in the sbin directory.

After starting successfully, you can use tools such as eclipse for remote debugging.

Let's take eclipse as an example

First, right-click in the code box of eclipse, select debug as in the pop-up option box, and select debug configurations in its pop-up option box.

Open the configuration page of debug

As shown below:

Then double-click remote java application (or right-click and select new configuration)

The interface is as follows:

Name can be customized, host needs to write the address to start the remote debug server, and prot needs to write the port in the above parameters. If correct, you can click the debug button below to start debug.

The above is all the contents of the article "sample Analysis of Hadoop's remote debug debugging". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.