Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of how far Java is away from the Linux kernel, the content is detailed and easy to understand, the operation is simple and fast, and has a certain reference value. I believe you will gain something after reading this article how far Java is away from the Linux kernel. Let's take a look.

How far is Java from the kernel?

Test environment version information:

Ubuntu (lsb_release-a) Distributor ID: UbuntuDescription: Ubuntu 19.10Release: 19.10Linux (uname-a) Linux yahua 5.5.5 # 1 SMP … X86_64 x86_64 x86_64 GNU/LinuxJavaOpenjdk jdk14

How can people who play with the kernel also know Java? This is mainly due to my school's Java course and my graduation experience when I worked as an Android phone in Huawei. Several modules have been scanned from APP/Framework/Service/HAL/Driver, and naturally I have some understanding of Java.

Every time I mention Java, I think of an interesting experience. Just graduated to the department for the first week, the department leader (Huawei is regarded as Manager) arranged for us to be familiar with Android. It took me a few days to write an Android game, something like Lianliankan. At the beginning of the week meeting, when the leader saw my presentation, he looked unhappy and questioned that my direct leader (called PL,Project Leader in Huawei) did not tell us the direction of the department.

Emm, I really didn't understand what it meant to be familiar with Android at that time, and later PL said that it was necessary to be familiar with the xxx module, and APP was only part of it. If I had been affirmed at that time, I might have been a Java engineer now ( manual dog head).

Start with launcher.

The furthest distance in the world is when we sit next door, I am looking at the agreement at the bottom, and you are studying spring. If you want to get closer between us, download openjdk source code (openjdk), then download glibc (glibc), and then download kernel source code (kernel).

Java program to JVM, which everyone must be more familiar with than I am, will not play tricks.

Let's take the entry of JVM as an example to analyze the flow from JVM to the kernel. The entry is the main function (java.base/share/native/launcher/main.c):

JNIEXPORT intmain (int argc, char** argv) {/ / omit ten thousand lines of parameter handling code return JLI_Launch (margc, margv, jargc, (const char**) jargv, 0, NULL, VERSION_STRING, DOT_VERSION, (const_progname! = NULL)? Const_progname: * margv, (const_launcher! = NULL)? Const_launcher: * margv, jargc > 0, const_cpwildcard, const_javaw, 0);}

JLI_Launch did three things we care about.

First, call CreateExecutionEnvironment to find the setting environment variables, such as the path to JVM (the variable jvmpath below). In my case, the / usr/lib/jvm/java-14-openjdk-amd64/lib/server/libjvm.so,window platform may be libjvm.dll.

Second, call LoadJavaVM to load the JVM, which is the libjvm.so file, and then find the corresponding field assigned to the InvocationFunctions by the function that created the JVM:

Jboolean LoadJavaVM (const char * jvmpath, InvocationFunctions * ifn) {void * libjvm;// omit error handling libjvm = dlopen (jvmpath, RTLD_NOW + RTLD_GLOBAL); ifn- > CreateJavaVM = (CreateJavaVM_t) dlsym (libjvm, "JNI_CreateJavaVM"); ifn- > GetDefaultJavaVMInitArgs = (GetDefaultJavaVMInitArgs_t) dlsym (libjvm, "JNI_GetDefaultJavaVMInitArgs"); ifn- > GetCreatedJavaVMs = (GetCreatedJavaVMs_t) dlsym (libjvm, "JNI_GetCreatedJavaVMs"); return JNI_TRUE;}

Dlopen and dlsym involve dynamic linking, which is simply understood that libjvm.so contains the definitions of JNI_CreateJavaVM, JNI_GetDefaultJavaVMInitArgs, and JNI_GetCreatedJavaVMs. After dynamic linking is completed, ifn- > CreateJavaVM, ifn- > GetDefaultJavaVMInitArgs, and ifn- > GetCreatedJavaVMs are the addresses of these functions.

Make sure that libjvm.so has these three functions.

Objdump-D / usr/lib/jvm/java-14-openjdk-amd64/lib/server/libjvm.so | grep-E "CreateJavaVM | GetDefaultJavaVMInitArgs | GetCreatedJavaVMs" | grep ": $" 00000000008fa9d0: 00000000008faa20: 00000000009098e0:

There are these implementations in the openjdk source code (under hotspot/share/prims/), interested students can continue to study.

Finally, call JVMInit to initialize the JVM,load Java program.

JVMInit calls ContinueInNewThread, which calls CallJavaMainInNewThread. To put it another way, I really don't like to talk about the problem in the way of function calls. A calls b and c, which is a waste of space, but some places span too much and are afraid of causing misunderstandings (especially for beginners). Believe me, water injection, there is no, I do not need experience + 3 .

The main logic of CallJavaMainInNewThread is as follows:

Int CallJavaMainInNewThread (jlong stack_size, void* args) {int rslt; pthread_t tid; pthread_attr_t attr; pthread_attr_init (& attr); pthread_attr_setdetachstate (& attr, PTHREAD_CREATE_JOINABLE); if (stack_size > 0) {pthread_attr_setstacksize (& attr, stack_size);} pthread_attr_setguardsize (& attr, 0) / no pthread guard page on java threads if (pthread_create (& tid, & attr, ThreadJavaMain, args) = = 0) {void* tmp; pthread_join (tid, & tmp); rslt = (int) (intptr_t) tmp;} else {rslt = JavaMain (args);} pthread_attr_destroy (& attr); return rslt;}

See pthread_create, the case is solved, the thread of Java is realized through pthread. You can get into the kernel here, but let's move on to JVM. ThreadJavaMain calls JavaMain directly, so the logic here is that if the thread is created successfully, the new thread executes the JavaMain, otherwise you know that the JavaMain is executed in the current process.

JavaMain is the focus of our attention. The core logic is as follows:

Int JavaMain (void* _ args) {JavaMainArgs * args = (JavaMainArgs *) _ args; int argc = args- > argc; char * * argv = args- > argv; int mode = args- > mode; char * what = args- > what; InvocationFunctions ifn = args- > ifn; JavaVM * vm = 0; JNIEnv * env = 0; jclass mainClass = NULL; jclass appClass = NULL; / / actual application class being launched jmethodID mainID; jobjectArray mainArgs; int ret = 0; jlong start, end / * Initialize the virtual machine * / if (! InitializeJVM (& vm, & env, & ifn)) {/ / 1 JLI_ReportErrorMessage (JVM_ERROR1); exit (1);} mainClass = LoadMainClass (env, mode, what); / / 2 CHECK_EXCEPTION_NULL_LEAVE (mainClass); mainArgs = CreateApplicationArgs (env, argv, argc); CHECK_EXCEPTION_NULL_LEAVE (mainArgs) MainID = (* env)-> GetStaticMethodID (env, mainClass, "main", "[Ljava/lang/String;) V"); / / 3 CHECK_EXCEPTION_NULL_LEAVE (mainID); / * Invoke main method. * / (* env)-> CallStaticVoidMethod (env, mainClass, mainID, mainArgs); / / 4 ret = (* env)-> ExceptionOccurred (env) = = NULL? 0: 1; LEAVE ();}

Step 1, call InitializeJVM to initialize the JVM. InitializeJVM calls ifn- > CreateJavaVM, which is JNI_CreateJavaVM in libjvm.so.

Step 2, LoadMainClass, ends up calling JVM_FindClassFromBootLoader, which also finds the function (defined under hotspot/share/prims/) through dynamic linking, and then calls it.

Steps 3 and 4, Java students should know that this is to call the main function.

It's a little beside the point. Let's continue to take pthread_create as an example and look at the kernel.

In fact, pthread_create is still a short distance from the kernel, which is glibc (nptl/pthread_create.c). The creation thread is ultimately implemented through the clone system call, and we don't care about the details of the glibc (otherwise we miss again), let's see how it is different from the direct clone.

(recommended micro-class: Java micro-class)

The following discussion about threads is excerpted from the book.

Const int clone_flags = (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SYSVSEM | CLONE_SIGHAND | CLONE_THREAD | CLONE_SETTLS | CLONE_PARENT_SETTID | CLONE_CHILD_CLEARTID | 0); _ _ clone (& start_thread, stackaddr, clone_flags, pd, & pd- > tid, tp, & pd- > tid)

The description of each logo is as follows (this sentence is not excerpted. ).

The flag describes that CLONE_VM shares with the current process VMCLONE_FS shares file system information CLONE_FILES shares open files CLONE_PARENT shares the same parent process as the current process CLONE_THREAD belongs to the same thread group as the current process, which also means that the thread CLONE_SYSVSEM shares sem_undo_list.

Share VM, file system information, and open files with the current process. When we see these, we understand that the so-called thread is like this.

Linux does not essentially separate the process from the thread, which is also known as the lightweight process (Low Weight Process, LWP). The difference is that the thread shares memory, files and other resources with the process (thread) that created it.

The complete paragraphs are as follows (several paragraphs enlarged by double quotation marks), which can be read in detail by interested students:

"the clone_flags parameter passed from fork to _ do_fork is fixed, so it can only be used to create processes. The kernel provides another system call clone,clone and finally calls _ do_fork implementation. Unlike fork, users can determine clone_flags according to their needs, and we can use it to create threads, as follows (the parameters of clone may be different on different platforms):

SYSCALL_DEFINE5 (clone, unsigned long, clone_flags, unsigned long, newsp, int _ _ user *, parent_tidptr, int, tls_val, int _ user *, child_tidptr) {return _ do_fork (clone_flags, newsp, 0, parent_tidptr, child_tidptr);}

Linux treats threads as lightweight processes, but the characteristics of threads are not arbitrarily determined by Linux and should be compatible with other operating systems as far as possible, so it follows the requirements of threads in POSIX standards. Therefore, to create a thread, the parameters passed to the clone system call should also be basically fixed.

The parameters for creating a thread are more complex, but fortunately, pthread (POSIX thread) provides us with a function, which can be called pthread_create. The function prototype (user space) is as follows.

Int pthread_create (pthread_t * thread, const pthread_attr_t * attr, void * (* start_routine) (void *), void * arg)

The first parameter, thread, is an output parameter in which the thread's id is stored after the thread is created, and the second parameter is used to customize the properties of the new thread. The successful creation of the new thread executes the function pointed to by start_routine, and the argument passed to that function is arg.

How exactly does pthread_create call clone, roughly as follows:

/ / Source: glibcconst int clone_flags = (CLONE_VM | CLONE_FS | CLONE_FILES | CLONE_SYSVSEM | CLONE_SIGHAND | CLONE_THREAD | CLONE_SETTLS | CLONE_PARENT_SETTID | CLONE_CHILD_CLEARTID | 0); _ _ clone (& start_thread, stackaddr, clone_flags, pd, & pd- > tid, tp, & pd- > tid)

There are many flags for the clone_flags setting. The first few flags indicate that the thread shares resources with the current process (and possibly the thread), and CLONE_THREAD means that the new thread and the current process are not parent-child.

The clone system call is also ultimately implemented through _ do_fork, so the difference between it and the fork that creates the process is limited to differences caused by different parameters, and there are two questions that need to be explained.

First, vfork sets the CLONE_VM flag, causing changes to local variables by the new process to affect the current process. So does clone, which is also set up with CLONE_VM, also have this hidden danger? The answer is no, because the new thread specifies its own user stack, which is specified by stackaddr. The sp parameter of the copy_thread function is stackaddr,childregs- > sp = sp, which modifies the pt_regs of the new thread, so when the new thread executes in user space, it uses a different stack from the current process and will not cause interference. Then why doesn't vfork do this? please refer to the design intent of vfork.

Second, fork returns twice, and so does clone, but they all return to the system call to start execution. How does pthread_create get the new thread to execute start_routine? Start_routine is executed indirectly by the start_thread function, so we just need to know how start_thread is called. Start_thread is not passed to the clone system call, so its call has nothing to do with the kernel, and the answer lies in the _ _ clone function.

(recommended tutorial: Linux tutorial)

In order to fully understand how the new process uses its user stack and start_thread calling procedures, it is necessary to analyze the _ _ clone function, even if it is platform-dependent and written in assembly language.

/ * i386*/ENTRY (_ clone) movl $- EINVAL,%eaxmovl FUNC (% esp),% ecx / * no NULL function pointers * / testl% ecx,%ecxjz SYSCALL_ERROR_LABELmovl STACK (% esp),% ecx / * no NULL stack pointers * / 1testl% ecx,%ecxjz SYSCALL_ERROR_LABELandl $0xfffffff0,% ecx / * alignment * / / 2subl $28% ecxmovl ARG (% esp),% eax / * no negative argument counts * / movl% eax 12 (% ecx) movl FUNC (% esp),% eaxmovl% eax,8 (% ecx) movl $0esipushl 4 (% ecx) pushl% ebx / / 3pushl% esipushl% edimovl TLS+12 (% esp),% esi / / 4movl PTID+12 (% esp),% edxmovl FLAGS+12 (% esp),% ebxmovl CTID+12 (% esp),% edimovl $SYS_ify (clone),% eaxmovl% ebx (% ecx) / / 5int $0x80 / / 6popl% edi / / 7popl% esipopl% ebxtest% eax,%eax / / 8jl SYSCALL_ERROR_LABELjz L (thread_start) ret / / 9L (thread_start): / / 10movl% esi,%ebp / * terminate the stack frame * / testl $CLONE_VM,% edije L (newpid) L (haspid): call *% ebx/* … , /

Take _ _ clone (& start_thread, stackaddr, clone_flags, pd, & pd- > tid, tp, & pd- > tid) as an example

FUNC (% esp) corresponds to & start_thread

STACK (% esp) corresponds to stackaddr

ARG (% esp) corresponds to pd (parameters passed to start_thread by the new process).

Step 1, assign the stack stackaddr of the new process to ecx to ensure that its value is not 0.

Step 2, put pd, & start_thread, and 0 on the stack of the new thread, which has no effect on the stack of the current process.

In step 3, the values of the three registers of the current process are put on the stack, and the value of the esp register is reduced by 12.

Step 4, prepare the system call, where FLAGS+12 (% esp) is stored in ebx, corresponding to clone_flags, and the system call number of clone is stored in eax.

Step 5, put the clone_flags on the stack of the new process.

Step 6, initiate a system call using the int instruction and give it to the kernel to create a new thread. As of this point, all the code is executed by the current process, and the new thread is not executed.

The code starting from step 7 will be executed by both the current process and the new thread. For the current process, the program takes the registers that step 3 into the stack out of the stack. But for the new thread, it is executed from the ret_from_fork of the kernel, and after switching to user mode, its stack has become stackaddr, so its edi equals clone_flags,esi equals zero and EBX equals & start_thread.

The result of the system call is returned by eax. Step 8 determines the result of the clone system call. For the current process, the clone system call successfully returns the id of the new thread in its pid namespace, which is greater than 0, so it executes the ret exit _ _ clone function. For the new thread, the return value of the clone system call is equal to 0, so it executes the code at L (thread_start). When the CLONE_VM flag of clone_flags is set, call *% ebx,ebx equals & start_thread is executed, and start_thread is executed, which calls the start_routine provided to pthread_create, ending. "

From this point of view, the Java → JVM → glibc → kernel doesn't seem to be far away.

This is the end of the article on "how far is Java from the Linux kernel?" Thank you for reading! I believe that everyone has a certain understanding of the knowledge of "how far is Java from the Linux kernel". If you want to learn more knowledge, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.