Shulou(Shulou.com)06/02 Report--

This article mainly introduces Java how to troubleshoot problems, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

Linux command class

The most commonly used tail-f in tail

Tail-300f shopbase.log # countdown 300 lines and enter real-time monitoring file writing mode grep grep forest f.txt # file search grep forest f.txt cpf.txt # multiple files search grep 'log' / home/admin-r-n # directory to find all files that match the keyword cat f.txt | grep-i shopbase grep' shopbase' / home/admin-r-n-include *. {vm Java} # specify the file suffix grep 'shopbase' / home/admin-r-n-exclude *. {vm,java} # Anti-match seq 10 | match seq 10 on grep 5-A 3 # | match seq 10 under grep 5-B 3 # | match up and down grep 5-C 3 # It's all right to use this at ordinary times. Cat f.txt | grep-c 'SHOPBASE' awk 1 basic command

Awk'{print $4 f.txt cpf.txt awk 6} 'f.txt awk' {print NR,$0} 'f.txt cpf.txt awk' {print FNR,$0} 'f.txt cpf.txt awk' {print FNR,FILENAME

"NF" = "$NF} 'f.txt cpf.txt echo 1 f.txt cpf.txt echo 2 f.txt cpf.txt echo 3 awk 4 | awk-F:' {print $1 Magne2 Magnum 3 Magi 4}'2 match

Awk'/ ldb/ {print} 'f.txt # matches ldb awk'! / ldb/ {print} 'f.txt # does not match ldb awk' / ldb/ & & / LISTEN/ {print} 'f.txt # matches ldb and LISTEN awk' $5 ~ / ldb/ {print} 'f.txt # the fifth column matches ldb 3 built-in variables

NR:NR indicates the number of data read according to the record delimiter after execution from awk. The default record delimiter is the newline character, so the default is the number of rows of data read. NR can be understood as the abbreviation of Number of Record.

FNR: when awk processes multiple input files, after processing the first file, NR does not start at 1, but continues to accumulate, so there is FNR. Every time a new file is processed, FNR starts counting from 1, and FNR can be understood as File Number of Record.

NF: NF indicates the number of fields in which the current record is divided, and NF can be understood as Number of Field.

Find sudo-u admin find / home/admin / tmp / usr-name * .log (multiple directories to find) find. -iname * .txt (both uppercase and lowercase) find. -type d (all subdirectories in the current directory) find / usr-type l (all symbolic links in the current directory) find / usr-type l-name "z*"-ls (symbolic link details eg:inode, directory) find / home/admin-size + 250000k (files over 2500k, of course + changed to-that is less than) find / home/admin f-perm 777-exec ls-l {} (query files by permissions) find / home/admin-atime-files accessed within 11 days find / home/admin-ctime-11 files find / home/admin-mtime-11 files modified within 11 days find / home/admin-amin-files accessed within 11 minutes find / home/admin-cmin-11 files find / home/admin-mmin-11 minutes after status change Modified files pgm batch query logs for which vm-shopbase meets the criteria

Pgm-A-f vm-shopbase 'cat / home/admin/shopbase/logs/shopbase.log.2017-01-17 | grep 2069861630' tsar tsar is our own collection tool. It is easy to persist the data collected by history on disk, so we quickly query the historical system data. Of course, real-time applications can also be queried. It is installed on most machines.

Tsar # # you can view the metrics of the most recent day

Cdn.xitu.io/2019/3/7/169572be99c3eecc?imageView2/0/w/1280/h/960/format/webp/ignore-error/1 "> tsar-- mem tsar-- load tsar-- cpu # # of course, this can also be used with the-d parameter to query the situation of a single indicator on a certain day. Top top, in addition to looking at some basic information, the rest is to cooperate to query various questions of vm.

Ps-ef | grep java top-H-p pid gets the thread from decimal to hexadecimal, and jstack goes to catch what the thread is doing

Other netstat-nat | awk'{print $6}'| sort | uniq-c | sort-rn # check the current connection and pay attention to the high close_wait, such as the following

Troubleshooting sharp weapon

The first thing btrace wants to say is btrace. It's really a production environment & a pre-sent troubleshooting problem killer. I won't say anything about the profile. Go straight to the code dry

See who currently calls the add method of ArrayList, and print only the thread call stack whose size of the current ArrayList is greater than 500

@ OnMethod (clazz = "java.util.ArrayList", method= "add", location = @ Location (value = Kind.CALL, clazz = "/. /", method= "/. /") public static void m (@ ProbeClassName String probeClass, @ ProbeMethodName String probeMethod, @ TargetInstance Object instance, @ TargetMethodOrField String method) {

If (getInt ("java.util.ArrayList", "size"), instance) > 479) {println ("check who ArrayList.add method:" + probeClass + "#" + probeMethod + ", method:" + method + ", size:" + getInt (field ("java.util.ArrayList", "size"), instance)); jstack (); println (); println ("= ="); println ();} monitors the value returned when the current service method is called and the requested parameters

OnMethod (clazz = "com.taobao.sellerhome.transfer.biz.impl.C2CApplyerServiceImpl", method= "nav", location = @ Location (value = Kind.RETURN)) public static void mt (long userId, int current, int relation, String check, String redirectUrl, @ Return AnyType result) {println ("parameter# userId:" + userId + ", current:" + current + ", relation:" + relation + ", check:" + check + ", redirectUrl:" + redirectUrl + ", result:" + result)) } some of the tools of other functional groups are more or less available, not to mention. If you are interested, please move. Https://github.com/btraceio/btrace

Note:

After observation, the release output of 1.3.9 is unstable. It takes several more triggers to see the correct result. When the regular expression matches the trace class, the range must be controlled, otherwise it is very likely that the application will get stuck due to the principle of bytecode injection. If you want the application to return to normal, you need to restart the application. Greys Greys is the masterpiece of @ du Kun. Name a few great features (some of which overlap with btrace):

Sc-df xxx: outputs the details of the current class, including source location and classloader structure

Trace class method: I really like this feature! JProfiler could have seen this feature a long time ago. Print out the time-consuming situation of the current method call, subdivided into each method. It's very helpful to check the performance of methods. For example, I used the trace command: http://www.atatech.org/articles/52947 in my previous article.

Other functional parts coincide with btrace, you can choose, if you are interested, please move. Http://www.atatech.org/articles/26247

Also related is arthas, which is based on Greys and is interested in moving to http://mw.alibaba-inc.com/products/arthas/docs/middleware-container/arthas.wiki/home.html?spm=a1z9z.8109794.header.32.1lsoMc.

JavOSize said that a function classes: by changing the bytecode, changed the content of the class, effective immediately. So you can quickly log somewhere to see the output, but the disadvantage is that the code is too intrusive. But if you know what you're doing, it's a good thing.

Other functions Greys and btrace can easily do, not to mention.

You can take a look at an introduction to javOSize that I wrote earlier. Please move to http://www.javosize.com/ on the official website of http://www.atatech.org/articles/38546.

JProfiler used to judge a lot of problems through JProfiler, but now Greys and btrace can basically solve them. In addition, the problem is basically the production environment (network isolation), so basically do not use it, but still need to mark it. Please move to https://www.ej-technologies.com/products/jprofiler/overview.html on the official website.

Mass murderer

EclipseMAT can be used as a plug-in for eclipse or opened as a separate program. For more information, please go to http://www.eclipse.org/mat/

The developers in the zprofiler group should be known to everyone. To put it in a nutshell: why do you need mat when you have zprofiler? for details, please move to zprofiler.alibaba-inc.com.

Java three axes. Oh, no, seven.

Jps, I only use one command:

Sudo-u admin / opt/taobao/java/bin/jps-mlvV

Common usage of jstack:

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jstack 2815

Native+java stack:

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jstack-m 2815

Jinfo can see the parameters of system startup, as follows

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jinfo-flags 2815

Two uses of jmap

1. Check the heap

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jmap-heap 2815

2.dump

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jmap-dump:live,format=b,file=/tmp/heap2.bin 2815 or

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jmap-dump:format=b,file=/tmp/heap3.bin 2815 3. Who's taking over the pile? Together with zprofiler and btrace, troubleshooting problems is like adding power to a tiger.

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jmap-histo 2815 | head-10

There are many jstat jstat parameters, but one is enough.

Sudo-u admin / opt/taobao/install/ajdk-8_1_1_fp1-b52/bin/jstat-gcutil 2815 1000

Jdb today, jdb is also often used. Jdb can be used to pre-send debug, assuming that your pre-sent java_home is / opt/taobao/java/, remote debug port is 8000. Then sudo-u admin / opt/taobao/java/bin/jdb-attach 8000. The above indicates that jdb starts successfully. Breakpoints can be set later for debugging. For more information, please see oracle official description http://docs.oracle.com/javase/7/docs/technotes/tools/windows/jdb.html.

CHLSDB CHLSDB feels that in many cases you can see something more interesting, so I won't elaborate on it. It is said that tools such as jstack and jmap are based on it.

Sudo-u admin / opt/taobao/java/bin/java-classpath / opt/taobao/java/lib/sa-jdi.jar sun.jvm.hotspot.CLHSDB more details can be seen R this post http://rednaxelafx.iteye.com/blog/1847971

Plugin of intellij idea

You can't remember the key promoter shortcut once, but you can remember it a few more times, right?

Maven helper is a good helper for analyzing maven dependencies.

VM options

Which file did your class load from?

-the XX:+TraceClassLoading result looks like [Loaded java.lang.invoke.MethodHandleImpl$Lazy from D:\ programme\ jdk\ jdk8U74\ jre\ lib\ rt.jar] the application hung up the output dump file

-XX:+HeapDumpOnOutOfMemoryError-XX:HeapDumpPath=/home/admin/logs/java.hprof Group basically contains this option in the vm parameter jar package conflict

Isn't it too much to write a separate headline for this? Everyone has dealt with this annoying case more or less. I can't handle you if I don't believe in so many plans?

Mvn dependency:tree > ~ / dependency.txt type out all dependencies

Mvn dependency:tree-Dverbose-Dincludes=groupId:artifactId type only the dependencies of the specified groupId and artifactId

-XX:+TraceClassLoading vm startup script is added. The details of the loading class can be seen in the tomcat startup script

-verbose vm startup script is added. The details of the loading class can be seen in the tomcat startup script

Greys:sc greys's sc command can also clearly see where the current class is loaded.

Tomcat-classloader-locate can know where the current class is loaded curl http://localhost:8006/classloader/locate?class=org.apache.xerces.xs.XSObject through the following url

The surprise brought to us by ALI-TOMCAT (thank you @ business view) list the jar loaded by the container

Curl http://localhost:8006/classloader/jars

Lists the actual jar package locations currently loaded by the current class, which is useful when resolving class conflicts

Curl http://localhost:8006/classloader/locate?class=org.apache.xerces.xs.XSObject

Screenshot.png screenshot.png other

Gpref http://www.atatech.org/articles/33317

If dmesg finds that his java process has disappeared quietly, leaving almost no clues, then dmesg is likely to have what you want.

Sudo dmesg | grep-I kill | less to find the keyword oom_killer. The results found are similar to the following:

[6710782.021013] java invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_scoe_adj=0 [6710782.070639] []? Oom_kill_process+0x68/0x140 [6710782.257588] Task in / LXC011175068174 killed as a result of limit of / LXC011175068174 [6710784.698347] Memory cgroup out of memory: Kill process 215701 (java) score 854 or sacrifice child [6710784.707978] Killed process 215701, UID 679, (java) total-vm:11017300kB, anon-rss:7152432kB, file-rss:1232kB above indicate that the corresponding java process is killed by the OOM Killer of the system, with a score of 854. Explain OOM killer (Out-Of-Memory killer), which monitors the memory resource consumption of the machine. When the machine runs out of memory, the mechanism scans all processes (calculated according to certain rules, memory usage, time, etc.), picks out the process with the highest score, and then kills it to protect the machine.

Dmesg log time conversion formula: log actual time = 1970-01-01 Greenwich mean + (seconds of current time-seconds since the system started + log time printed by dmesg) seconds:

Date-d "1970-01-01 UTC echo" $(date +% s)-$(cat / proc/uptime | cut-f 1-d'') + 12288812.926194 "| bc seconds" the rest is to see why the memory is so large that OOM-Killer is triggered.

Thank you for reading this article carefully. I hope the article "how to troubleshoot Java" shared by the editor will be helpful to everyone. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.