Shulou(Shulou.com)06/02 Report--

This article mainly explains "what are the methods of troubleshooting common problems online in Java". Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn "what are the methods of troubleshooting common problems online in Java?"

First, where is the bottleneck of system performance

We often mention the running environment of the project, so what does the running environment include? It generally includes your operating system, CPU, memory, hard drive, network bandwidth, JRE environment, various components on which your code depends, and so on. Therefore, the bottleneck of system performance is often IO bottleneck, CPU bottleneck, memory bottleneck or performance bottleneck caused by programs.

Log in to the server and use the TOP command to get a comprehensive view of the system resource usage

How to analyze the bottleneck of CPU

Using the TOP command, enter uppercase P (that is, shift + P) to sort by CPU use descending order. On the third line of the TOP command, you can see the key information% id: percentage of idle CPU time. If this value is too low, it indicates that there is a bottleneck in the system CPU. If it is too low, it is usually caused by your java program, so you need to log in to the docker container and use the jstack command to view the stack information to analyze the reason.

Confirm the target process

View the corresponding process information-> Log in to the container to view the container id-> enter the container-> top command in the container to view the target process whose CPU is too high

# top#ps-ef | grep process number # sudo docker ps-a#sudo docker exec-it container id bash#top

You can use the sar command for CPU usage details; 1 / 3 in the command means sampling once per second for a total of 3 samples

# sar-u 1 3

Print stack information

Since the process is started by the admin user, jstack printing stack information needs to switch admin users and make sure that the jstack command is installed on your machine

Then exit the container and move the file copy to the personal home directory (if the cp command is not available, you can move to the personal home directory through the scp command)

Use the sftp command to copy the file to the local machine to analyze the stack log information; if you install the sz command, or download it through the sz command

# sudo-u admin/ opt/usr/java/bin/jstack-l 76 > / home/admin/test/logs/jstack.log# cp / home/admin/test/logs/jstack.log ~ /

Analyze stack information

Enter the container to see which threads account for a high percentage (the screenshot is only to show how to use it, but the actual cpu is not very high)

# top-HP process id

Convert the high proportion thread PID into hexadecimal, go to the stack log to find the corresponding thread stack information, and analyze the problem.

How to analyze memory bottleneck

In the process of project development, unreasonable use of threads or collections will usually lead to memory oom. For memory bottlenecks, you can check memory usage through top command or free command. For more details, you can check it through vmstat command.

Free command. The actual available memory is free + buff + available.

# free-m

How to analyze the memory usage and find the reason for the excessive memory? you need to log in to the docker container to view the processes with a high proportion of memory, dump through the jmap command, and analyze it through IBM's analysis tool MA.

Determine the target process-> jmap-> copy to the personal home directory through the scp command-> because the dump file is relatively large, compress it with the tar command before downloading it to the local

# sudo-u admin/ opt/usr/java/bin/jmap-dump:live,format=b,file=/home/admin/test/logs/java.heap.bin process number # scp username @ host:/home/admin/test/logs/java.heap.bin ~ / # cd ~ # tar-zcvf java.heap.bin.tar.gz java.heap.bin

In general, the most frequently viewed memory analysis is the Histogram under Actions. Check how many object references are not as good as GC;. Generally, a normal dump file does not look obvious. You need to compare multiple dump files to check the causes of memory leaks.

How to analyze the bottleneck of IO

If there is a performance bottleneck in IO, the% wa in the top tool will be on the high side. Use the iostat command tool to analyze further.

# iostat-d-k-x 1 1

If the value of% iowait is too high, it indicates that the hard disk has an Ipicuro bottleneck. If the% util is close to 100%, it means that too many requests have been generated, the system is fully loaded, and there may be a bottleneck on the disk. If the svctm is close to the await, it means that there is almost no waiting time for the await O; if the await is much larger than the svctm, the queue is too long and the io response is too slow, so necessary optimizations are needed. If the avgqu-sz is large, it also means that there are a lot of io waiting.

The general ways to solve this problem are: using cache, describing the conversion of disk IO; synchronous to asynchronous, random write to sequential write, and replacing hardware.

Network message analysis by calling third-party interface

Projects sometimes encounter service invocations of third-party interfaces, usually through HTTP client requests, and often encounter common problems such as service connection timeout and system jitter; this problem is sometimes troublesome, only through tcpdump to grab network layer messages, through wireshark tools to analyze the reasons; for HTTS protocol, we can only rely on third-party servers to grab packets for analysis.

# tcpdump-I eth0 dst host hostname-C 10240-W 50-w xx.cap

Generally, root user rights are required, and hostname is replaced with the actual host ip or domain name. Eth0 is a network card, and the server generally has multiple network cards, so be sure to specify which network data packet to grab on the network card.

Let's review the process of TCP's three-way handshake and four waves at the transport layer.

The data transmission between the client and the server is generally HTTP or HTTPS protocol, HTTP hypertext transfer protocol is recommended to transfer data over the TCP transport protocol, and the underlying TCP transmission uses the socket Socket for data flow transmission; as for why it is a three-way handshake, it can be understood that the channel is unreliable and the transmission should be reliable, and the three-way handshake is the theoretical minimum.

First handshake: establish a connection. The client sends a connection request message segment, setting the SYN location as 1 and the sequence Number as x; then the client enters the SYN_SEND state and waits for confirmation from the server

Second handshake: the server receives a SYN message segment. When the server receives the SYN message segment from the client, it needs to confirm the SYN message segment and set the Acknowledgment Number to Acknowledgment Number 1 (Sequence Number+1). At the same time, it also sends the SYN request message to itself, setting the SYN location as 1 and the sequence Number as y. The server puts all the above information into a message segment (that is, the SYN+ACK message segment) and sends it to the client, and the server enters the SYN_RECV state.

Third handshake: the client receives the SYN+ACK message segment from the server. Then the Acknowledgment Number is set to yearly 1, and the ACK message segment is sent to the server. After the message segment is sent, both the client and the server enter the ESTABLISHED state and complete the TCP three-way handshake.

After completing the three-way handshake, the client and server can begin to transmit data. This is the general introduction of TCP's three-way handshake.

What about four breakups?

When the client and the server establish a TCP connection through a three-way handshake, when the data transfer is complete, it must be disconnected from the TCP connection. Then for the disconnection of TCP, there is a mysterious "four breakups".

Break up for the first time: host 1 (can make client or server), set Sequence Number and Acknowledgment Number, and send a FIN message segment to host 2; at this point, host 1 enters FIN_WAIT_1 state; this means that host 1 has no data to send to host 2

The second breakup: host 2 received the FIN message segment sent by host 1, returned an ACK message segment to host 1, Acknowledgment Number was Sequence Number plus 1; host 1 entered FIN_WAIT_2 status; host 2 told host 1 that I "agreed" to your shutdown request

Third breakup: host 2 sends a FIN message segment to host 1, requesting to close the connection, while host 2 enters the LAST_ACK state

The fourth breakup: host 1 receives the FIN message segment sent by host 2, sends ACK message segment to host 2, and then host 1 enters the TIME_WAIT state; host 2 closes the connection after receiving the ACK message segment of host 1; at this time, host 1 still does not receive a reply after waiting for 2MSL, which proves that the Server end has been shut down normally, well, host 1 can also close the connection.

Track the dynamic running details of online application java

For online applications, sometimes you need to pay attention to some details of java runtime. You can track https://legacy.gitbook.com/book/json-liu/btrace/details through the Btrace command.

II. Troubleshooting of common problems in project code

Before we talk about this problem, it is necessary to talk about java's class loading mechanism and JVM memory structure. Understanding these will be of great help to our common OOM problems and performance tuning.

Class loading mechanism

There is a complete life cycle from class loading virtual machine memory to final unloading, and its whole life cycle includes seven stages: loading, verification, preparation, parsing, initialization, use and unloading.

In the process of class loading, the classloader adopts the principle of parental delegation, first starting the classloader BootStrap, then extending the classloader, the application loader,

Custom class loader

Verification: convert the full class name of the class file into a binary stream through getResourceAsStream, and then convert the static data structure (constructor, property, method, etc.) into the data structure of the runtime method area

Verification mainly includes class file format verification (whether the class file begins with some illegal characters), metadata information verification (such as whether the java class has a parent class, whether the parent class is modified by final modifiers, etc.), bytecode verification (verifying the methods of the class), and symbol reference verification (whether the corresponding class can be found by fully qualified name)

Preparation: that is, to allocate memory to the class and to initialize the variable assignment public static int = 123. the assignment is not 0 because the putstatic instruction exists in the class constructor method, which is assigned to 123 only in the initialization phase.

Parsing: class, interface, method parsing, mainly replacing symbolic reference with direct reference, symbolic reference java virtual machine memory reference irrelevant, direct reference can be pointer position, offset can be located to the specific location of memory

Initialization: assign variables, putstatic getstatic, invokestatic instructions, "clinit" constructor, assign values

JVM memory structure

When the java virtual machine executes java code, it divides the memory it manages into different areas. The partition structure of JVM memory is as follows:

The method area and stack area are shared by all threads, and the virtual machine stack, local method stack and program counter are thread private.

In these areas, except that the program counter will not cause oom problems, all other areas may produce oom.

Stacking area

Java heap is the largest area of JVM memory, and almost all java instances and array allocations are completed here. According to the memory recovery algorithm, the heap can be divided into the younger generation and the old age, with a ratio of 1:2, in which the younger generation is divided into Eden and two survival regions, which is 8:1:1. Oom is most likely to occur in this area. Generally speaking, there are two possible reasons: the peak traffic exceeds the threshold set by the program or memory leak; for example, the most common memory leak is the collection of local variables, due to improper use, has been unable to GC, will trigger java.lang.OutOfMemoryError

Review the garbage collection algorithms of the younger generation and the old: in the young generation replication algorithm, in the old era with mark removal, tag finishing algorithm, for different java objects, the life cycle is different, some survival age is short, some survival age is long, how does JVM judge that the java object instance can be GC? Java generally refers to the root search algorithm, which starts from the GC ROOT node. If there is no reference chain from the sample to the GC ROOT object node, the object is considered unavailable. Often, some large objects are initialized, and the younger generation cannot put them, and they will store large objects on behalf of the old, so it is easy to produce Full GC. Therefore, large object reads must not be read into memory at one time, but are processed according to row reading. If the setting parameters of the reactor area are unreasonable, you can adjust the size of the reactor area through Xmx.

Method area

In the process of class loading, the class file is read, converted into binary stream information, and finally converted into metadata information for storage, including the fully qualified name, version, method, field and other information of the class, which are allocated to JVM at compile time. This area is the method area, and for some constant pools, it is also allocated here. Garbage collection is also involved in the method area, such as unloading of classes, useless constants and useless classes. Generally, constantly creating dynamic proxies will cause the oom; of the method area to adjust the size of the method area through MaxPermSize.

Virtual machine stack

This area is private to the thread. If the thread wants to run normally in the java virtual machine, it not only needs a counter to record the line number, but also needs its own running space. The virtual machine stack can save the running order of the method, the local variables of the method, and the method needs its own memory space when calculating. We call this area the virtual machine stack. Each stack is divided into local variable table, Operand stack, dynamic linked list, and return address; method execution requires an area to store local variables, and method operation requires memory space, that is, Operand stack. some methods need to load the specified method at run time, and the symbolic reference conversion directly refers to the dynamic linked list; the method will return when it encounters a return instruction or throws an exception, and the address needs to be returned. In this part, there will also be an oom problem. Typically, the thread pool is not set in size, and threads are constantly created in the code, but the creation thread needs memory space, and if there is not enough physical memory, it will oom. When you encounter this problem, you usually adjust the size of the stack and set it through Xss.

Local method stack

Like the virtual machine stack, in the java virtual machine, not only java methods but also local methods are executed, and oom is also generated. In addition, stack overflow exceptions are also generated like the virtual machine stack.

Program counter *

As we all know, when a virtual machine is dealing with multithreading, it takes turns to switch threads to obtain CPU resources. In order to ensure that each thread can execute correctly next time, it is necessary to record the current running location of each thread. The function of the program counter is to record the line number (bytecode) line number (exactly the address of the instruction) executed by each thread next time, so as to ensure that it can be executed correctly next time. The memory is so small that it is almost negligible

After covering these two concepts, let's take a look at some common questions about java

NoSuchMethodException

There are generally two possible reasons for this problem: java ClassLoader mechanism, java two-party package conflict; for ClassLoader problems, you can configure-XX:+TraceClassLoading in JVM to track the class loading process, and the two-party package conflict can directly exclude pom file conflict files.

Third, database mysql slow sql optimization

Presumably when dealing with the database, we often encounter sql query is very slow, a large amount of data, the performance is very low. Students with some development experience in this problem think of using explain execution plan to analyze sql; integrated business scenarios and establish appropriate indexes for optimization. Here I just summarize how to analyze slow sql and how to build indexes.

When it comes to indexing, we have to mention the data structure; mysql is a relational disk-based relational database, and the performance of disk IO is different from that of reading data from memory by several orders of magnitude, so in order to reduce the number of disk IO, the B+ tree is used to store data. The lower the height of the tree, the less the disk IO times. Assuming that the amount of data is N and the amount of data for each disk block is m, then the height of the tree h=log (masking 1) * N, and m = the size of the disk block / the size of the data item for the B + tree, all data have leaf nodes, so the inner node disk block will store more internal nodes, and the index range of each node is larger, and the size of the disk block is fixed to 1 page size, with a default of 16K, so the larger the size of the data item. The smaller the m, the lower the height.

After the principle is clearly explained, take a look at some principles for indexing

The leftmost matching principle, because when building a search tree, it is established in the order from do to right. When you encounter a range query, a fuzzy query or a union query, the index will not take effect. The index field differentiation is higher, that is, the proportion of non-repetition should be large. In this way, it is obvious that the index field can not participate in the calculation, because the data fields stored in the B+ tree are all field names, if there is a function calculation. The cost is quite high.

Sql statement through the execution of plan analysis, the key depends on the size of rows, in general, the smaller the rows, the faster the query, to avoid full-table query, multi-table query as far as possible to use union or union all to query

About the difference of mysql storage engine: after 5.7, myql uses InnoDB storage engine by default. Compared with MyISAM storage engine, InnoDB supports transaction features and also uses B+ tree, but leaf node data domain stores different values. InnoDB stores complete data records and defaults to primary key index order, so InnoDB must have primary key. For ordinary indexes, data domain stores the value of primary key index. So we need to find the primary key index in the ordinary index tree first, and then find the corresponding record in the primary index tree. The MyISAM leaf node stores the address of the data, and the data file and index file are separated.

At this point, I believe you have a deeper understanding of the "Java online common problems troubleshooting methods", might as well come to the actual operation of it! Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.