Shulou(Shulou.com)06/01 Report--

This article introduces the knowledge of "how to build Docker image tuning in Java SpringBoot project". Many people will encounter this dilemma in the operation of actual cases, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Foreword:

In the previous era of SpringCloud micro-services, services were based on "Jar packages", and each service was Jar for interconnection and invocation between services. Now, with the popularity of Kubernetes, it has changed to one image and one service, relying on the unified arrangement of images by Kubernetes to manage the services. In the practice of Kubernetes microservices, there must be no more contact than the Docker image. Because the programming language I use is Java, I have more contact with Java SpringBoot projects, so I am more concerned about how to better compile Docker images through Dockerfile.

To put it simply, Kubernetes micro-service is the relationship between the arrangement, combination and mutual adjustment of a group of mirrors, so how to compile images will make the service performance better, make the construction, push and pull of images faster, and make them occupy less network resources. Optimization and easier to use has become a top priority, and it is also a problem worth pondering. Here I will briefly describe the exploration of how the Docker image of the SpringBoot project is packaged to write Dockerfile.

System environment:

Docker version: 18.09.3

Open JDK basic image version: openjdk:8u212-b04-jre-slim

Image repository for testing: Aliyun Docker Hub

Project Github: https://github.com/my-dlq/blog-example/tree/master/springboot-dockerfile

First, explore how conventional Springboot compiles Docker images

Here we will use the Dockerfile writing of the regular SpringBoot compiled Docker image to see how the compiled image works.

1. Prepare to compile the SpringBoot project of the image

Here, we prepare an ordinary springboot project compiled by Maven to build the Docker image. The project content is shown in the following figure. You can see that what you want to use is the Jar file of the application inside. Store it in the image to complete the image construction task.

Jar file size: 70.86mb

2. Prepare the Dockerfile file

To build a Docker image, you need to prepare the Dockerfile file in advance, and the contents of this Dockerfile file are instructions for building the Docker image. Here is a Dockerfile of a commonly used SpringBoot build Docker image, put it in the Java source directory (the parent directory of target), and make sure that the path set in the Dockerfile script set below corresponds to the target path.

FROM openjdk:8u212-b04-jre-slimVOLUME / tmpADD target/*.jar app.jarRUN sh-c 'touch / app.jar'ENV JAVA_OPTS= "- Duser.timezone=Asia/Shanghai" ENV APP_OPTS= "ENTRYPOINT [" sh ","-c "," java $JAVA_OPTS-Djava.security.egd=file:/dev/./urandom-jar / app.jar $APP_OPTS "] 3. Build Docker image

Use the Docker build command to build the Docker image and observe the compilation time.

Because the image needs to be pushed to the Aliyun Docker repository later, the image prefix is Aliyun.

Time: this parameter shows the elapsed time of the execution process

$time docker build-t registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1.

Construction process

Sending build context to Docker daemon 148.7MBStep 1: 7: FROM openjdk:8u212-b04-jre-slim8u212-b04-jre-slim: Pulling from library/openjdk743f2d6c1f65: Already exists b83e581826a6: Pull complete 04305660f45e: Pull complete bbe7020b5561: Pull complete Digest: sha256:a5bcd678408a5fe94d13e486d500983ee6fa594940cbbe137670fbb90030456cStatus: Downloaded newer image for openjdk:8u212-b04-jre-slim->; 7c6b62cf60eeStep 2: VOLUME / tmp-- >; Running in 13a67ab65d2bRemoving intermediate container 13a67ab65d2b->; 52011f49ddefStep 3: ADD target/*.jar app.jar-> 26aa41a404fdStep 4 touch 7: RUN sh-c 'touch / app.jar'->; Running in 722e7e44e04dRemoving intermediate container 722e7e44e04d->; 7baedb10ec62Step 5-Duser.timezone=Asia/Shanghai 7: ENV JAVA_OPTS= "- Duser.timezone=Asia/Shanghai"->; Running in 2681d0c5edacRemoving intermediate container 2681d0c5edac->; 5ef4a794b992Step 6 Duser.timezone=Asia/Shanghai 7: ENV APP_OPTS= "- >; Running in 5c8924a2a49dRemoving intermediate container 5c8924a2a49d-> Fba87c19053aStep 7ax 7: ENTRYPOINT ["sh", "- c", "java $JAVA_OPTS-Djava.security.egd=file:/dev/./urandom-jar / app.jar $APP_OPTS"]->; Running in c4cf97009b3cRemoving intermediate container c4cf97009b3c-- > d5f30cdfeb81Successfully built d5f30cdfeb81Successfully tagged registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1real 0m13.778suser 0m0.078ssys 0m0.153s

See that the compilation is completed in 14s.

4. Push the image to the image repository

Push the image to the Aliyun repository, then view and record the push time

$time docker push registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1

Execution process

The push refers to repository [registry.cn-beijing.aliyuncs.com/mydlq/springboot] cc1a2376d7c0: Pushed 2b940d07e9e7: Pushed 9544e87fb8dc: Pushed feb5d0e1e192: Pushed 8fd22162ddab: Pushed 6270adb5794c: Pushed 0.0.1: digest: sha256:dc60d304383b1441941ca4e9abc08db775d7be57ccb7c534c929b34ff064a62f size: 1583real 0m24.335suser 0m0.052ssys 0m0.059s

See that this time is completed in 25 seconds.

5. Pull the image

Here, switch to another server to perform the image pull operation and observe the image pull time.

$time docker pull registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1

Pull process

0.0.1: Pulling from mydlq/springboot743f2d6c1f65: Already exists b83e581826a6: Pull complete 04305660f45e: Pull complete bbe7020b5561: Pull complete 4847672cbfa5: Pull complete b60476972fc4: Pull complete Digest: sha256:dc60d304383b1441941ca4e9abc08db775d7be57ccb7c534c929b34ff064a62fStatus: Downloaded newer image for registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1real 0m27.528suser 0m0.033ssys 0m0.192s

It took a total of 28s to complete the pull.

6. Try again after modifying the Java source code and repackaging the Jar

Here, modify the contents of the JAVA file of the source code, and then repackage the Jar package to try to compile, push, and pull the process again. Because Docker uses hierarchical caching when performing the build, this is a faster process.

(1), compilation

$time docker build-t registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2 .Sending build context to Docker daemon 148.7MBStep 1 c67160dd2a23Step 7: FROM openjdk:8u212-b04-jre-slim->; 7c6b62cf60eeStep 2 ADD target/*.jar app.jar 7: VOLUME / tmp-- >; Using cache-- >; 52011f49ddefStep 3 ADD target/*.jar app.jar 7: ADD target/*.jar app.jar-- >; c67160dd2a23Step 4 touch 7: RUN sh-c 'touch / app.jar'-- >; Running in 474900d843a2Removing intermediate container 474900d843a2-- > 3ce9a8bb2600Step 5Djava.security.egd=file:/dev/./urandom 7: ENV JAVA_OPTS= "- Duser.timezone=Asia/Shanghai"->; Running in f48620b1ad36Removing intermediate container f48620b1ad36->; 0478f8f14e5bStep 6-- >: ENV APP_OPTS= "- >; Running in 98485fb15fc8Removing intermediate container 98485fb15fc8->; 0b567c848027Step 7-hand 7: ENTRYPOINT [" sh ","-c "," java $JAVA_OPTS-Djava.security.egd=file:/dev/./urandom-jar / app.jar $APP_OPTS "]->; Running in e32242fc6efeRemoving intermediate container e32242fc6efe-- > 7b223b23ebfdSuccessfully built 7b223b23ebfdSuccessfully tagged registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2real 0m3.190suser 0m0.039ssys 0m0.403s

You can see the cache used by the first 1 and 2 layers in the process of compiling the image, so it is very fast. The editor-in-chief translation process takes 4 seconds to complete.

(2), push

$time docker push registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2The push refers to repository [registry.cn-beijing.aliyuncs.com/mydlq/springboot] d66a2fec30b5: Pushed f4da2c7581aa: Pushed 9544e87fb8dc: Layer already exists feb5d0e1e192: Layer already exists 8fd22162ddab: Layer already exists 6270adb5794c: Layer already exists real 0m20.816suser 0m0.024ssys 0m0.081s

You can see that only the first two layers are pushed, and the other four times there is no push because the remote warehouse has not changed. The whole push process takes 21 seconds to complete.

(3), pull

$time docker pull registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.20.0.2: Pulling from mydlq/springboot743f2d6c1f65: Already exists b83e581826a6: Already exists 04305660f45e: Already exists bbe7020b5561: Already exists d7e364f0d94a: Pull complete 8d688ada35b1: Pull complete Digest: sha256:7c13c40fa92ec2fdc3a8dfdd3232be1be9c1a1a99bf123743ff2a43907ee03dcStatus: Downloaded newer image for registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2real 0m23.214suser 0m0.053ssys 0m0.097s

Local and the first four layers of the cache, pull only the last two layers with changes. This process takes 24 seconds to complete.

7. The feeling in the process of using mirror image

Using this way to build a Docker image for the SpringBoot project, it gives me the impression that as long as there is a little change in the source code, the SpringBoot project needs to compile the project by Maven and then build it by the Docker image. Every time, a 70m + application Jar file is stored in Docker, and sometimes the letter is changed obviously, and the whole program Jar may have to be re-stored in the Docker image, and then in the push and pull process. It is very inconvenient to push a large image or pull a large image for transmission every time.

2. Understand Docker layering and caching mechanism 1. Introduction to Docker hierarchical caching

In order to save storage space, Docker adopts the concept of hierarchical storage. Shared data layers the image and the container, different images can share the same data, and a RW layer is assigned to the container on the image to speed up the startup order of the container.

During the process of building the image, Docker progressively executes the instructions in Dockerfile in the order specified in Dockerfile. As each instruction is checked, Docker will look for reusable existing images in its cache instead of creating a new (duplicate) image.

Each line of command in Dockerfile creates a new layer that contains changes to the file system before and after the execution of this line of command. To optimize this process, Docker uses a caching mechanism: as long as this line of commands remains the same, the result is the same as the previous one, using the previous result directly.

To take full advantage of hierarchical caching, we must understand how the command line in Dockerfile works, especially the commands RUN,ADD and COPY.

Refer to the Docker documentation for Docker image cache: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

2. Layering of SpringBoot Docker images

After SpringBoot is compiled into an image, the underlying layer will be a system, such as Ubantu. The upper layer is the dependent JDK layer, and then the SpringBoot layer. We cannot operate the bottom two layers, so optimization can only be considered by the SpringBoot layer.

Third, what causes the bloated Jar package

From the above experiments, we know that the reason why each compilation, push, pull process is relatively slow, the reason is the huge image file. After learning the concept of Docker caching, there is an idea that if files that do not change often are cached, files that change frequently will not be cached. Since SpringBoot projects are constantly changing, how can caching be used to implement them? If you force the use of the cache, then each mirror is not the old program content in the cache.

So consider what files are contained in the application Jar package, which files in Java are often changed, and which are not often changed. In this regard, the following will analyze the application Jar package hit by SpringBoot.

1. Unpack the Jar package to view the contents

Display the unzipped list and view each folder size

$tree-L 3-si-du. ├── [74m] BOOT-INF │ ├── [2.1k] classes │ └── [74m] lib ├── [649] META-INF │ ├── [552] MANIFEST.MF │ └── [59] maven └── [67] org └── [38] springframework

You can see that the largest file is the lib folder, open this folder, inside is a bunch of related dependent Jar, one of which Jar is not large, but a pile of Jar combination is very large, generally SpringBoot projects rely on Jar size to maintain at 40MB ~ 160MB.

Take a look at the org folder, where the code adds up to only a few hundred KB. Therefore, the SpringBoot program Jar package is composed of these Classes files and dependent Jar, these dependencies Jar a total of 74 MB, accounting for almost all the size of the application Jar package.

2. New ideas to solve the problem of bloated.

If a Jar package contains only class files, the size of the Jar package may be a few hundred KB. Now to explore, if you separate the Jar and class that lib depends on, set the Jar package of the application to contain only the class file, and put the Jar file in the lib folder outside the SpringBoot Jar.

When we write a program, the Jar we often rely on doesn't change very often. What changes a lot is the source code program. The Jar package we rely on is very large and the source code is very small. Think about it carefully. If you set a separate cache for the application dependent Jar package when packaged into a Docker image, and the application Jar package contains only Class files, so that in the process of Docker compilation, push and pull, except for all of them for the first time, the Jar file that only contains Class will be changed in the subsequent process of compilation, push and pull, just a few hundred KB. It can be said that this process can be completed in an instant. So think about how to separate the dependent Jar package under the lib folder from the application Jar package.

3. How to solve the separation of lib and class files

After searching for a lot of relevant information, it is found that the Maven plug-in of SpringBoot does a lot of things when performing Maven compilation and playing Jar package. If the packaging logic of some plug-ins is changed, all the Jar packages under the lib folder will be copied to the outside of the application Jar when playing the application Jar, leaving only the compiled bytecode file.

Introduce these Maven tools into the project pom.xml

Org.apache.maven.plugins maven-jar-plugin true lib/ org.springframework.boot spring- Boot-maven-plugin nothing nothing org.apache.maven.plugins maven-dependency-plugin Copy-dependencies prepare-package copy-dependencies ${project.build.directory} / lib

Execute the Maven command to package Jar

$mvn clean install

When the Maven command is finished, look at the target directory as shown below:

Then test whether the Jar file is working properly.

$java-jar springboot-helloworld-0.0.1.jar

Then see the run log, OK! The Dockerfile transformation work will continue next.

4. Talk about how to modify Springboot compiled Docker image

> Project Github address: https://github.com/my-dlq/blog-example/tree/master/springboot-dockerfile

1. Modify the Dockerfile file

To modify the above Dockerfile file, you need to add a layer of instructions to copy the dependent Jar in the lib directory to the image, and keep the rest consistent with the above Dockerfile.

FROM openjdk:8u212-b04-jre-slimVOLUME / tmpCOPY target/lib/. / lib/ADD target/*.jar app.jarRUN sh-c 'touch / app.jar'ENV JAVA_OPTS= "- Duser.timezone=Asia/Shanghai" ENV APP_OPTS= "" ENTRYPOINT ["sh", "- c", "java $JAVA_OPTS-Djava.security.egd=file:/dev/./urandom-jar / app.jar $APP_OPTS"]

> A layer of instruction has been added to copy the lib folder to the image. Due to the Docker cache mechanism, this layer must be used before replication and application of Jar. In this way, as long as the dependency Jar in the lib/ folder remains unchanged each time, a new layer will not be created, but the cache will be reused.

2. Compile, push and pull the Docker image for the first time after modification

Before performing compilation, push, and pull, all resources related to the last image of the server are cleared before execution.

(1), compilation

$time docker build-t registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1 .Sending build context to Docker daemon 223.2MBStep 1 take 8: FROM openjdk:8u212-b04-jre-slim8u212-b04-jre-slim: Pulling from library/openjdk743f2d6c1f65: Already exists b83e581826a6: Pull complete 04305660f45e: Pull complete bbe7020b5561: Pull complete Digest: sha256:a5bcd678408a5fe94d13e486d500983ee6fa594940cbbe137670fbb90030456cStatus: Downloaded newer image for openjdk:8u212-b04-jre-slim->; 7c6b62cf60eeStep 2-- 8: VOLUME / tmp-- >; Running in 529369acab24Removing intermediate container 529369acab24-> Ad689d937118Step 3 touch 8: COPY target/lib/. / lib/->; 029a64c15853Step 4-touch: ADD target/*.jar app.jar->; 6265a83a1b90Step 5-hand 8: RUN sh-c 'touch / app.jar'->; Running in 839032a58e6bRemoving intermediate container 839032a58e6b->; 5d877dc35b2bStep 6-hand 8: ENV JAVA_OPTS= "- Duser.timezone=Asia/Shanghai"->; Running in 4043994c5fedRemoving intermediate container 4043994c5fed->; 7cf32beb571fStep 7 hand 8: ENV APP_OPTS= "- > Running in b7dcfa10458aRemoving intermediate container b7dcfa10458a->; b6b332bcf0e6Step 8amp 8: ENTRYPOINT ["sh", "- c", "java $JAVA_OPTS-Djava.security.egd=file:/dev/./urandom-jar / app.jar $APP_OPTS"]->; Running in 539093461b59Removing intermediate container 539093461b59-- >; d4c095c4ffecSuccessfully built d4c095c4ffecSuccessfully tagged registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1real 0m22.983suser 0m0.051ssys 0m0.540s

(2), push

$time docker push registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1The push refers to repository [registry.cn-beijing.aliyuncs.com/mydlq/springboot] c16749205e05: Pushed 7fef1a146748: Pushed a3bae74bbdf2: Pushed 9544e87fb8dc: Pushedfeb5d0e1e192: Pushed8fd22162ddab: Pushed6270adb5794c: Pushed 0.0.1: digest: sha256:e2f4db740880dbe5338b823112ba9467fedf8b27cd75572611d0d3837c80f157 size: 1789real 0m30.335suser 0m0.052ssys 0m0.059s

(3), pull

$time docker pull registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.10.0.1: Pulling from mydlq/springboot743f2d6c1f65: Already exists b83e581826a6: Pull complete 04305660f45e: Pull complete bbe7020b5561: Pull complete de6c4f15d75b: Pull complete 7066947b7d89: Pull complete e0742de67c75: Pull complete Digest: sha256:e2f4db740880dbe5338b823112ba9467fedf8b27cd75572611d0d3837c80f157Status: Downloaded newer image for registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.1real 0m36.585suser 0m0.024ssys 0m0.092s3, compile again, push, pull

(1), compilation

$time docker build-t registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2 .Sending build context to Docker daemon 223.2MBStep 1 COPY target/lib/ 8: FROM openjdk:8u212-b04-jre-slim-- >; 7c6b62cf60eeStep 2 COPY target/lib/ 8: VOLUME / tmp-- >; Using cache-- >; ad689d937118Step 3 COPY target/lib/ 8: COPY target/lib/. / lib/-- >; Using cache-- >; 029a64c15853Step 4 amp 8: ADD target/*.jar app.jar-- > 563773953844Step 5ENV JAVA_OPTS= 8: RUN sh-c 'touch / app.jar'->; Running in 3b9df57802bdRemoving intermediate container 3b9df57802bd->; 706a0d47317fStep 6-- > 8: ENV JAVA_OPTS= "- Duser.timezone=Asia/Shanghai"->; Running in defda61452bfRemoving intermediate container defda61452bf->; 742c7c926374Step 7-- 8: ENV APP_OPTS= "- >; Running in f09b81d054ddRemoving intermediate container f09b81d054dd-> 929ed5f8b12aStep 8ax 8: ENTRYPOINT ["sh", "- c", "java $JAVA_OPTS-Djava.security.egd=file:/dev/./urandom-jar / app.jar $APP_OPTS"]->; Running in 5dc66a8fc1e6Removing intermediate container 5dc66a8fc1e6-- >; c4942b10992cSuccessfully built c4942b10992cSuccessfully tagged registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2real 0m2.524suser 0m0.051ssys 0m0.493s

As you can see, this cache is used directly at layer 3, and the whole compilation process takes only 2.5 seconds.

(2), push

$time docker push registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2The push refers to repository [registry.cn-beijing.aliyuncs.com/mydlq/springboot] d719b9540809: Pushed d45bf4c5fb92: Pushed a3bae74bbdf2: Layer already exists 9544e87fb8dc: Layer already exists feb5d0e1e192: Layer already exists 8fd22162ddab: Layer already exists 6270adb5794c: Layer already exists 0.0.2: digest: sha256:b46d81b153ec64321caaae7ab28da0e362ed7d720a7f0775ea8d1f7bef310d00 size: 1789real 0m0.168suser 0m0.016ssys 0m0.032s

You can see that the image push is completed within 0.2s.

(3), pull

$time docker pull registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.20.0.2: Pulling from mydlq/springboot743f2d6c1f65: Already exists b83e581826a6: Already exists 04305660f45e: Already exists bbe7020b5561: Already exists de6c4f15d75b: Already exists 1c77cc70cc41: Pull complete aa5b8cbca568: Pull complete Digest: sha256:b46d81b153ec64321caaae7ab28da0e362ed7d720a7f0775ea8d1f7bef310d00Status: Downloaded newer image for registry.cn-beijing.aliyuncs.com/mydlq/springboot:0.0.2real 0m1.947suser 0m0.017ssys 0m0.042s

You can see that the image pull is completed within 2 seconds.

5. Final summary

Due to network fluctuations and system changes, time can only be used as a reference, but the process of compiling, pushing and pulling is indeed a lot faster. Most files are cached, and only a few hundred KB of traffic is naturally much faster than dozens of MB or even hundreds of MB.

Finally, this approach only provides a reference. Since the mirroring of microservice Docker, the entire image is maintained rather than a service program, so the concern is whether the Docker image can work properly, and how to build the image will make the constructed image more useful.

In the production environment, because the version changes slowly, it is not easy to update, so in the production environment, it is best to apply the original SpringBoot image compilation method to ensure installation (unless a large number of instances have verified the construction method).

This is the end of "how to build Docker Image tuning for the Java SpringBoot Project". Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.