Shulou(Shulou.com)06/03 Report--

This article mainly introduces Java how to achieve dynamic tracking technology, the article is very detailed, has a certain reference value, interested friends must read it!

Start with JSP.

For most Java programmers, they were exposed to a technology called JSP (Java Server Pages) in the early days. Although this technology is out of date in today's era of front-end code separation, front-end logic separation, and front-end organizational structure separation, there are still some interesting things that are worth talking about.

At that time, when we were just getting started with Java, most of our efforts seemed to be focused on the display of JSP pages:

"this table shows the wrong number of rows."

"it was written in the for loop. Change it, refresh the page and try again."

"well, well, the form shows no problem, but the login person's name is not available. Is there a problem with Sesstion access?"

"it's possible. I'll change it again and try to refresh it later."

……

When we change the code over and over again to refresh the browser page and try again, we may not have noticed a cool thing: after we modified the code, we simply refreshed the browser page, and the change took effect, and the whole process did not restart JVM. According to our common sense, Java programs generally load class files at startup, if they all modify the code like JSP and take effect without restarting, then the problem at the beginning of the article can be solved: add a log print code in the Java file, it will take effect without restarting, it will not destroy the scene, but also can locate the problem. Can't help but give it a try: modify, compile, and replace class files. Well, no, the newly changed code didn't work. Then why can JSP do it? Let's first look at how JSP works.

When we open a browser and request access to a JSP file, the whole process goes like this:

[failed to upload picture... (image-ad91d8-1560497185049)]

JSP file processing process

After the JSP file has been modified, it takes effect in time because the Web container (Tomcat) checks to see if the requested JSP file has been changed. If any changes have occurred, the JSP file is reparsed, translated into a new Sevlet class, and loaded into JVM. All subsequent requests will be processed by this new Servet. There is a problem here. According to Java's class loading mechanism, classes are not allowed to repeat in the same ClassLoader. To get around this limitation, the Web container creates a new ClassLoader instance each time to load the newly compiled Servlet class. All subsequent requests will be processed by the new Servlet, thus switching between the old and the new JSP.

HTTP services are stateless, so JSP scenarios are basically one-time consumption, which makes sense to "replace" class by creating new ClassLoader, but for other applications, such as Spring framework, even if you do so, most objects are singletons. For objects that have already been created in memory, we cannot modify the object behavior by creating new ClassLoader instances.

I just want to print a log without restarting the app. Is that so difficult?

Java object behavior

Since JSP's method doesn't work, let's see if there is another way. If you think about it, we will find that the problem at the beginning of the article is essentially the problem of dynamically changing the behavior of objects that already exist in memory. Therefore, we need to find out where the JVM is related to the behavior of the object and whether it is possible to change it.

As we all know, objects use two things to describe things: behavior and attributes. For example:

Public class Person {private int age;private String name;public void speak (String str) {System.out.println (str);} public Person (int age, String name) {this.age = age;this.name = name;}}

In the above Person class, age and name are properties, and speak is behavior. An object is a case of a class, and the properties of each object belong to the object itself, but the behavior of each object is public. For example, we now create two objects based on the Person class, personA and personB:

Person personA = new Person (43, "lixunhuan"); personA.speak ("I am Li Xunhuan"); Person personB = new Person (23, "afei"); personB.speak ("I am a punk")

PersonA and personB have their own names and ages, but have a common behavior: speak. Imagine, if we were designers of the Java language, how would we store the behavior and properties of objects?

"it's very simple. Attributes follow objects, and each object saves a copy. Behavior is something common. Pull it out and put it in a separate place."

"Huh? take out the public parts, which is like code reuse."

"from the main road to simplicity, many things are supposed to go the same way."

In other words, the first step is to find the common place to store the behavior of the object. After a search, we found the following description:

Method area is created on virtual machine startup, shared among all Java virtual machine threads and it is logically part of heap area. It stores per-class structures such as the run-time constant pool, field and method data, and the code for methods and constructors.

The object behavior (methods, functions) of Java is stored in the method area.

"where does the data in the method area come from?"

"the data in the method area is extracted from the class file when the class is loaded."

"where do the class files come from?"

"compiled from Java or other source code conforming to the JVM specification."

"where does the source code come from?"

"nonsense, handwritten, of course!"

"push backwards, handwriting is fine, compilation is fine, as for loading. Is there a way to load a class that has already been loaded? If so, we can modify the area of the target method in the bytecode, and then reload the class, so that the behavior of the object in the method area is changed, and the properties of the object are not changed. nor does it affect the state of the existing object, then the problem can be solved. However, doesn't this violate JVM's principle of class loading? After all, we don't want to change ClassLoader. "

"Junior, you can go and see the java.lang.instrument.Instrumentation."

Java.lang.instrument.Instrumentation

After reading the document, we found two interfaces: redefineClasses and retransformClasses. One is to redefine class, and the other is to modify class. The two are more or less the same. Take a look at reDefineClasses's instructions:

This method is used to replace the definition of a class without reference to the existing class file bytes, as one might do when recompiling from source for fix-and-continue debugging. Where the existing class file bytes are to be transformed (for example in bytecode instrumentation) retransformClasses should be used.

All replace existing class files, redefineClasses provides bytecode files to replace existing class files, and retransformClasses replaces existing bytecode files after modification.

Of course, it's not safe to replace classes directly at run time. For example, an exception is thrown when a new class file references a class that does not exist, or a field of a class is deleted, and so on. So, as stated in the document, instrument has many limitations:

The redefinition may change method bodies, the constant pool and attributes. The redefinition must not add, remove or rename fields or methods, change the signatures of methods, or change inheritance. These restrictions maybe be lifted in future versions. The class file bytes are not checked, verified and installed until after the transformations have been applied, if the resultant bytes are in error this method will throw an exception.

Basically, all we can do is simply modify some of the behavior in the method, which is enough for our initial question, printing a log. Of course, there are many other very useful things we can do besides printing logs through reTransform, which will be described below.

So how do we get the class file we need? One of the easiest way is to recompile the modified Java file to get the class file, and then call redefineClasses replacement. But what should we do with files that don't have (or can't get, or are inconvenient to modify) source code? In fact, for JVM, no matter it is Java or Scala, the source code of any language that conforms to the JVM specification can be compiled into a class file. JVM operates on class files, not source code. So, in this sense, we can say that "JVM has nothing to do with language". In that case, with or without source code, we only need to modify the class file.

Direct operation bytecode

Java is a language that software developers can understand, class bytecode is a language that JVM can read, and class bytecode will eventually be interpreted as a machine-readable language by JVM. No matter which language it is, it is created by man. Therefore, in theory (and indeed in practice), people can read any of the above languages, and since they can read them, they can naturally modify them. As long as we want, we can skip the Java compiler and write bytecode files directly, but this is not in line with the development of the times. After all, high-level language design is designed to serve us humans, and its development efficiency is much higher than that of machine languages.

Bytecode files are far less readable to humans than Java code. Nevertheless, some outstanding programmers have created frameworks that can be used to edit bytecode directly, providing interfaces that allow us to easily manipulate bytecode files and inject methods to modify classes. dynamically create a new class, and so on. One of the most famous frameworks is ASM, where bytecode operations in cglib, Spring and other frameworks are based on ASM.

As we all know, the AOP of Spring is based on dynamic proxy. Spring dynamically creates a proxy class at run time, references the proxied class in the proxy class, and performs some mysterious operations before and after the proxied method is executed. So how does Spring create proxy classes at run time? The beauty of dynamic proxies is that we do not have to manually write proxy class code for each class that needs to be proxied, Spring will dynamically create a class at run time, the process of creation here is not to write a Java file through a string, then compile it into a class file and then load it. Spring will directly "create" a class file, then load it, and the tool to create the class file is ASM.

At this point, we know that using the ASM framework to directly manipulate the class file, add a piece of code to print the log in the class, and then call retransformClasses.

BTrace

Up to now, we all stay at the level of theoretical description. So how to implement it? First, let's look at a few questions:

1. In our project, who will do this to find the bytecode, modify the bytecode, and then reTransform the action? We are not prophets, and it is impossible to know whether we may encounter such a problem at the beginning of the article in the future. Considering the performance-to-price ratio, it is also impossible for us to develop a piece of code specifically to modify and reload the bytecode in every project.

two。 What if JVM is not local, but remote?

3. What if you can't even use ASM? Can it be more general, more "stupid"?

Fortunately, because of BTrace, we don't have to write a set of such tools ourselves. What is BTrace? BTrace is open source, and the project description is extremely brief:

A safe, dynamic tracing tool for the Java platform.

BTrace is a secure tool based on Java language that can provide dynamic tracking service. BTrace is based on ASM, Java Attach Api, and Instruments development, and provides a lot of annotations for users. With these annotations, we can write BTrace scripts (simple Java code) to achieve the desired effect without getting caught up in ASM's manipulation of bytecode.

Take a look at a simple example provided by the BTrace official: intercept methods that start with read in all classes in all java.io packages, and print class names, method names, and parameter names. When the program IO load is relatively high, you can see which classes are caused from the output information, isn't it convenient?

Package com.sun.btrace.samples;import com.sun.btrace.annotations.*;import com.sun.btrace.AnyType;import static com.sun.btrace.BTraceUtils.*;/*** This sample demonstrates regular expression* probe matching and getting input arguments* as an array-so that any overload variant* can be traced in "one place". This example* traces any "readXX" method on any class in* java.io package. Probed class, method and arg* array is printed in the action.*/@BTrace public class ArgArray {@ OnMethod (clazz= "/ java\\ .io\\.. * /", method= "/ read.*/") public static void anyRead (@ ProbeClassName String pcn, @ ProbeMethodName String pmn, AnyType [] args) {println (pcn); println (pmn); printArray (args);}}

Let's look at another example: print the number of threads created so far every 2 seconds.

Package com.sun.btrace.samples;import com.sun.btrace.annotations.*;import static com.sun.btrace.BTraceUtils.*;import com.sun.btrace.annotations.Export;/*** This sample creates a jvmstat counter and* increments it everytime Thread.start () is* called. This thread count may be accessed* from outside the process. The @ Export annotated* fields are mapped to jvmstat counters. The counter* name is "btrace." + + "." + * / @ BTrace public class ThreadCounter {/ / create a jvmstat counter using @ Export@Export private static long count;@OnMethod (clazz= "java.lang.Thread", method= "start") public static void onnewThread (@ Self Thread t) {/ / updating counter is easy. Just assign to// the static fieldfields countdown;} @ OnTimer (2000) public static void ontimer () {/ / we can access counter as "count" as well// as from jvmstat counter directly.println (count); / / or equivalently... println (Counters.perfLong ("btrace.com.sun.btrace.samples.ThreadCounter.count"));}}

Is it instructive to look at the usage above? I can't help but have a lot of ideas. For example, check when HashMap triggers rehash, and how many elements are in the container at this time, and so on.

With BTrace, the problem at the beginning of the article can be solved perfectly. As for the specific functions of BTrace and how to write the script, there are a large number of explanations and examples in the BTrace project on Git, and the online article introducing the use of BTrace is Heihe Sha number, so I won't repeat it here.

We understand the principle, and have the support of useful tools, and all that is left is to exert our creativity, as long as we use it reasonably in the right situation.

Since BTrace can solve all the problems mentioned above, what is the architecture of BTrace?

BTrace mainly has the following modules:

1.BTrace scripts: with the annotations defined by BTrace, we can easily develop scripts as needed.

2.Compiler: compile the BTrace script into a BTrace class file.

3.Client: sends the class file to Agent.

4.Agent: Java-based Attach Api,Agent can be dynamically attached to a running JVM, and then open a BTrace Server to receive the BTrace script sent by client; parse the script, and then find the class to be modified according to the rules in the script; after modifying the bytecode, call the reTransform API of Java Instrument to modify the behavior of the object and make it effective.

The architecture of the entire BTrace is roughly as follows:

[failed to upload picture... (image-5cb505-1560497185048)]

BTrace workflow

BTrace finally realized the replacement of class through Instruments. As mentioned above, there are many restrictions on the use of Instruments for security reasons, and BTrace is no exception. BTrace is "read-only" to JVM, so the limitations of BTrace scripts are as follows:

Creation of objects is not allowed

Creation of arrays is not allowed

Exception is not allowed

Catch exceptions are not allowed

It is not allowed to call methods of other objects or classes at will, only static methods provided in com.sun.btrace.BTraceUtils (some data processing and information output tools) are allowed.

It is not allowed to change the properties of a class

No member variables and methods are allowed, only static public void methods are allowed

Inner classes and nested classes are not allowed

Synchronization methods and synchronization blocks are not allowed

Loops are not allowed

It is not allowed to inherit other classes at will (except java.lang.Object, of course)

Implementation of interfaces is not allowed

Assert is not allowed

Class objects are not allowed

It is understandable that there are so many restrictions. What BTrace needs to do is that although the bytecode is modified, it has no effect on the normal operation of the whole program except for the required information.

Arthas

BTrace script has a certain learning cost in use, and it would be better if it can encapsulate some commonly used functions and provide simple commands directly. Alibaba's engineers have long thought of this, and just last year (September 2018), Alibaba opened up his own Java diagnostic tool, Arthas. Arthas provides simple command line operation and powerful functions. The technical principle behind it is roughly the same as that mentioned in this article.

The purpose of this article is to explain the context of Java dynamic tracking technology. After mastering the principles behind the technology, readers can develop their own "Frozen Throne" if they like.

The end.

Now, let's try to stand on higher ground and "overlook" these problems.

Java's Instruments leaves hope for runtime dynamic tracking, Attach API provides an "entrance" for runtime dynamic tracking, and ASM greatly facilitates "human" manipulation of Java bytecode.

Based on Instruments and Attach API predecessors created tools such as JProfiler, Jvisualvm, BTrace, Arthas and so on. Based on ASM, cglib and dynamic agent are developed, and then Spring AOP is widely used.

Java is a static language, and it is not allowed to change the data structure at runtime. However, after Java 5 introduced Instruments,Java 6 and Attach API, things began to change. Although there are many limitations, however, with the efforts of our predecessors, only a little bit of reserved space similar to "read-only" has been used to create a variety of brilliant technologies. it greatly improves the efficiency of software developers in locating problems.

The computer should be one of the greatest inventions in the history of mankind, from electromagnetic induction to magnetoelectricity, to high and low voltage simulating bits of 0 and 1, to binary representation of several basic types, to basic types representing infinite objects, and finally infinite combination of objects to interactive simulation of real life and even the whole universe.

These are all the contents of the article "how to implement dynamic tracking Technology in Java". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.