Shulou(Shulou.com)06/01 Report--

This article mainly explains "what are the problems encountered in the implementation of the Java probe". The content of the explanation in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn what are the problems encountered in the implementation of the Java probe.

The Java probe can cut into the application code unknowingly while the Java application is running. it is a tool for listening to the code behavior or changing the code behavior.

There are only two ways to implement distributed call link tracing, code intrusive and non-code intrusive, and the implementation based on Java probe belongs to non-code intrusive.

The code written by programming languages running on the Java virtual machine has a unified intermediate format: class file format. The implementation of dynamic modification of class bytecode insertion of additional behavior of the code, can achieve non-intrusive application call behavior collection.

Thanks to the Instrumentation interface provided by Java SE 6. Based on Instrumentation, a Java Agent application (Java probe) for modifying class bytecode at runtime can be developed, which can replace the bytecode of the class before the class is loaded, or modify the bytecode of the class by reloading the class after the class is loaded.

Simply implementing runtime modification of class bytecode is not enough to call it a "probe". For Java Agent developed based on Instrumentation, only the virtual machine parameter "- javaagent" is added to the Java application startup command to specify the location of the Java Agent application jar package, and the probe can be inserted into every corner of the application code without introducing its jar package into the project. Environment isolation is achieved by using different class loading from the application, giving people the illusion that Java Agent is adsorbed to run on the application.

The reason why Instrumentation is difficult to control is that you need to understand the Java class loading mechanism and bytecode, and you can accidentally encounter all kinds of unfamiliar Exception. The author has stepped on a lot of holes in the implementation of Java probe, one of which is the problems related to class loading, which is also to be shared with you in this article.

A class loaded by a parent class loader cannot refer to a class loaded by a child class loader

A class loaded by the parent class loader cannot reference the class loaded by the child class loader, otherwise a NoClassDefFoundError will be thrown.

How do you understand this sentence? This is actually a face-to-face test.

The java.* classes provided by JDK are loaded by the startup class loader. What happens if we modify the class under the java package in java agent and insert the code that calls logback to print the diary? Since the logback under the java agent package is loaded by AppClassLoader (the application class loader, also known as the system class loader), and the startup class loader that loads the class under the java package is the parent class loader of AppClassLoader, the code calling the logback print diary is inserted into the class under the java package. First, when loading the class under the java package, jvm will check whether the startup class loader has loaded this class, if it has not been loaded. However, the startup class loader cannot load the class of the logback package, and the startup class loader does not ask the subclass loader whether the subclassloader can load the class, even if the subclassloader loads the class. So there will be NoClassDefFoundError.

If we have to modify the classes under the java package, and if we have to access the classes we wrote in the project or the classes provided by the third-party jar package, or the classes under the javaagent package we wrote, how can we avoid NoClassDefFoundError?

The author encountered this problem on the Internet to find a lot of resources, unfortunately did not find. So the author thinks that his computer contains the source code of Arthas, so it is better to learn how to solve Arthas.

Arthas is an open source Java diagnostic tool from Alibaba, which is very suitable for online troubleshooting.

Refer to the solution of Alibaba's open source Arthas:

The class used to receive buried point code reported events (Spy):

Public final class Spy {

Public static void before (String className, String methodName, String descriptor, Object [] params) {

}

Public static void complete (Object returnValueOrThrowable, String className, String methodName, String descriptor) {

}

}

Before: report before method execution

Complete: reported before the method return or before throwing an exception. When the method throws an exception, the first parameter is the exception, otherwise the first parameter is the return value

Place Spy under a separate jar package, call the appendToBootstrapClassLoaderSearch method of Instrumentation in the premain and agentmain methods, and send the jar package where the Spy class is located to be scanned and loaded by the startup class loader, as shown in the following code.

/ / agent-spy.jar

String agentSpyJar = jarPath [1]

File spyJarFile = new File (agentSpyJar)

Instrumentation.appendToBootstrapClassLoaderSearch (new JarFile (spyJarFile))

Print the classloader in the Spy class, and if the printed result is null, the Spy class is loaded by the startup classloader.

Public final class Spy {

Static {

System.out.println ("Spy class loader is" + Spy.class.getClassLoader ())

}

/ /.

}

Finally, the reporting method is injected into Spy and called through reflection in Spy. The code for the complete Spy class is as follows.

Public final class Spy {

Public static Method beforMethod

Public static Method completeMethod

Public static void before (String className, String methodName, String descriptor, Object [] params) {

If (beforMethod! = null) {

Try {

BeforMethod.invoke (null, className, methodName, descriptor, params)

} catch (IllegalAccessException | InvocationTargetException e) {

}

}

}

Public static void complete (Object returnValueOrThrowable, String className, String methodName, String descriptor) {

If (completeMethod! = null) {

Try {

CompleteMethod.invoke (null, returnValueOrThrowable, className, methodName, descriptor)

} catch (IllegalAccessException | InvocationTargetException e) {

}

}

}

}

Calling through reflection has an impact on performance, especially since each method on the calling link requires reflection to call two reporting methods.

It may not be completely understood correctly, but the author has tried that this scheme is really feasible.

Realize the isolation between Agent and application environment

Why quarantine?

Isolation is to avoid Agent contamination of the application itself, so that the developer Java Agent does not have to consider whether the introduced jar package conflicts with the jar package introduced by the target application.

What happens when Java Agent encounters an Spring Boot app?

After the Spring Boot application is packaged, attaching Agent to the application startup may throw an eye-catching NoClassDefFoundError exception, which will not happen in IDEA testing. The reason behind this is that Agent and the packaged Spring Boot application use different class loaders.

We may call the code of the monitored SpringBoot application in Agent, or we may call the API of the third-party jar package that Agent depends on, and these jar packages are also imported in the SpringBoot application, and NoClassDefFoundError may appear.

The jar package for Agent is loaded by the AppClassLoader class loader (the system class loader).

In IDEA, the project's class files and third-party libraries are loaded through AppClassLoader, and the jar specified with-javaagent is also loaded through AppClassLoader, so testing in idea will not encounter this problem.

After the SpringBoot application is packaged, the JVM process startup entry is no longer the main method we wrote, but the startup class generated by SpringBoot. SpringBoot uses a custom class loader (LaunchedClassLoader) to load classes in jar and third-party jar packages, whose parent class loader is AppClassLoader.

That is, after the SpringBoot application is packaged, the class loader used to load the class under the javaagent package is the parent class loader of the class loader used by SpringBoot.

How to achieve isolation?

Let the load agent package be loaded using a custom class loader instead of using the AppClassLoader loader.

Refer to the implementation of Alibaba's open source Arthas, and customize URLClassLoader to load agent packages and third-party jar packages that agent depends on.

Because the class in which the premain or agentmain methods reside is loaded by jvm using AppClassLoader, agent must be split into two jar packages. The core functions are placed under the agent-core package, and the classes where the premain or agentmain methods are located are placed under the agent-boot package. Load agent-core using a custom URLClassLoader class loader in the premain or agentmain method.

Step one:

A custom class loader, OnionClassLoader, inherits URLClassLoader, as shown in the following code:

Public class OnionClassLoader extends URLClassLoader {

Public OnionClassLoader (URL [] urls) {

Super (urls, ClassLoader.getSystemClassLoader (). GetParent ())

}

@ Override

Protected synchronized Class loadClass (String name, boolean resolve) throws ClassNotFoundException {

Final Class loadedClass = findLoadedClass (name)

If (loadedClass! = null) {

Return loadedClass

}

/ / give priority to loading system classes from parent (SystemClassLoader) to avoid throwing ClassNotFoundException

If (name! = null & & (name.startsWith ("sun.") | | name.startsWith ("java.")) {

Return super.loadClass (name, resolve)

}

Try {

Class aClass = findClass (name)

If (resolve) {

ResolveClass (aClass)

}

Return aClass

} catch (Exception e) {

/ / ignore

}

Return super.loadClass (name, resolve)

}

}

At the same time, specify in the constructor that the parent class loader of OnionClassLoader is the parent class loader of AppClassLoader.

ClassLoader.getSystemClassLoader (): get the system class loader (AppClassLoader)

Step 2:

Use the OnionClassLoader classloader to load the agent-core in the premain or agentmain method.

/ / 1

File agentJarFile = new File (agentJar)

Final ClassLoader agentLoader = new OnionClassLoader (new URL [] {agentJarFile.toURI () .toURL ()})

/ / 2

Class transFormer = agentLoader.loadClass ("com.msyc.agent.core.OnionClassFileTransformer")

/ / 3

Constructor constructor = transFormer.getConstructor (String.class)

Object instance = constructor.newInstance (opsParams)

/ / 4

Instrumentation.addTransformer ((ClassFileTransformer) instance)

1. Construct agent-core.jar according to the absolute path of OnionClassLoader.

2. Load ClassFileTransformer under agent-core.jar

3. Create a ClassFileTransformer instance using reflection

4. Add ClassFileTransformer to Instrumentation

Classes under the agent-core package that the OnionClassFileTransformer class depends on are naturally also loaded using the OnionClassLoader class loader, including third-party jar packages that agent-core depends on.

Adapting to webmvc framework

The difficulty of generating distributed call chain diaries lies in the concatenation of method burial points and method call diaries.

There are many ways to connect the diaries of the distributed call chain, and the author adopts the simplest way: manage the time of id+.

For the same thread in the same process, id can concatenate the called methods and sort the method call diary according to the hit time and the value of an accumulator.

For different processes, the management diaries of different applications can be concatenated by passing the management id, and sorted according to the management time.

For example, the purpose of adapting the webmvc framework is to get the dotted ID (transaction ID) passed from the calling source from the request header. For DispatcherServlet#doDispatch method stuffing, get the request header "S-Tid" from the HttpServletRequest parameter. "S-Tid" is a custom request header parameter that is used to pass dotted ID.

The author encountered the same problem when adapting to webmvc and openfeign. For example, when adapting to webmvc, when modifying the doDispatch method of DispatcherServlet, the asm framework throws java.lang.TypeNotPresentException.

Java.lang.TypeNotPresentException: this exception is thrown when an application attempts to access a type using a string that represents a type name, but cannot find a type definition with the specified name.

The reason for this is that when you rewrite a DispatcherServlet class using the asm framework, asm uses the Class.forName method to load the class referenced by the symbol, and throws a TypeNotPresentException if the target class cannot be loaded.

The default asm uses the class loader that loads itself to try to load some of the classes that currently rewrite the class, while the class loader used to load the asm framework uses the same class loader as the agent-core package, and the DispatcherServlet is loaded by SpringBoot's LaunchedClassLoader class loader.

Fortunately, the ClassFileTransformer#transform method passes the class loader used to load the current class:

Public class OnionClassFileTransformer implements ClassFileTransformer {

@ Override

Public byte [] transform (ClassLoader loader, String className, Class classBeingRedefined

ProtectionDomain protectionDomain, byte [] classfileBuffer) {

/ /.

}

}

If the class that currently needs to be rewritten is DispatcherServlet, the first parameter of the transform method is the class loader that will be used to load the DispatcherServlet class

We just need to specify that asm uses the class loader passed in by the ClassFileTransformer#transform method to load the classes that DispatcherServlet depends on.

ClassWriter classWriter = new ClassWriter (ClassWriter.COMPUTE_MAXS | ClassWriter.COMPUTE_FRAMES) {

@ Override

Protected ClassLoader getClassLoader () {

Return loader

}

}

As shown in the code, we override the getClassLoader method of asm's ClassWriter class, and the returned class loader is the class loader passed in by the ClassFileTransformer#transform method.

Thank you for your reading, the above is the content of "what are the problems encountered in the implementation of Java probe". After the study of this article, I believe you have a deeper understanding of the problems encountered in the implementation of Java probe, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.