Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about the construction of BuildKit optimized Dockerfile, which may not be well understood by many people. in order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.

Docker automatically builds the image by reading the instructions in Dockerfile, and Dockerfile is a text file that in turn contains all the commands needed to build a given image.

The above explanation is extracted from the official documentation of Docker and summarizes the use of Dockerfile. The use of Dockerfile is important because it is our blueprint and a record of the layers we added to the Docker image.

In this article, we will learn how to take advantage of BuildKit capabilities, a set of enhancements introduced in Docker v18.09. Integrated BuildKit will provide us with better performance, storage management and security.

The goal of this article

Reduce build time

Reduce the mirror size

Get maintainability

Get repeatability

Understanding Multi-stage Dockerfile

Learn about BuildKit features.

precondition

Docker concept knowledge

Docker is installed (currently using v19.03)

A Java application (in this article, I used a Jenkins Maven sample application)

Let's get started!

A simple example of Dockerfile

The following is an example of an unoptimized Dockerfile that contains a Java application. We will make some optimizations step by step.

FROM debian COPY. / app RUN apt-get update RUN apt-get-y install openjdk-11-jdk ssh emacs CMD ["java", "- jar", "/ app/target/my-app-1.0-SNAPSHOT.jar"]

Here, we might ask ourselves: how long will it take to build? To answer this question, let's create the Dockerfile on the local development environment and have Docker build the image.

# enter your Java app folder cd simple-java-maven-app-master # create a Dockerfile vim Dockerfile # write content, save and exit docker pull debian:latest # pull the source image time docker build-- no-cache-t docker-class. # overwrite previous layers # notice the build time 0mcm21s user 0je 23s system 0% cpu 1pure 5je 17 total

At this point, we need 1m55s for our build.

Would it make any difference if we only enabled BuildKit with no other changes?

Enable BuildKit

BuildKit can be enabled in two ways:

Set the DOCKER_BUILDKIT = 1 environment variable when invoking the Docker build command, for example:

Time DOCKER_BUILDKIT=1 docker build-- no-cache-t docker-class

To set Docker BuildKit to enable by default, you need to make the following settings in / etc/docker/daemon.json, and then restart:

{"features": {"buildkit": true}}

The initial effect of BuildKit

DOCKER_BUILDKIT=1 docker build-no-cache-t docker-class. 0jue 54s user 0je 93s system 1 cpu 1 cpu 43jue 00 total

At this point, we need 1m43s for our build. On the same hardware, it takes about 12 seconds to build. This means that building can save about 10% of your time with almost no effort.

Now let's see if some additional steps can be taken to further improve.

The order of change from the smallest to the most frequent

Because order is important for caching, we move the COPY command closer to the end of the Dockerfile.

FROM debian RUN apt-get update RUN apt-get-y install openjdk-11-jdk ssh emacs RUN COPY. / app CMD ["java", "- jar", "/ app/target/my-app-1.0-SNAPSHOT.jar"] avoid using "COPY."

Select a more specific COPY parameter to avoid cache interruptions. Copy only what you want.

FROM debian RUN apt-get update RUN apt-get-y install openjdk-11-jdk ssh vim COPY target/my-app-1.0-SNAPSHOT.jar / app CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] apt-get update is used with the install command

This prevents the use of outdated package caching.

FROM debian RUN apt-get update & &\ apt-get-y install openjdk-11-jdk ssh vim COPY target/my-app-1.0-SNAPSHOT.jar / app CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] remove unnecessary dependencies

Do not install debugging and editing tools at the beginning, but you can install them later when needed.

FROM debian RUN apt-get update & &\ apt-get-y install--no-install-recommends\ openjdk-11-jdk COPY target/my-app-1.0-SNAPSHOT.jar / app CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] remove the package manager cache

Your mirror does not need this cached data. Take this opportunity to free up some space.

FROM debian RUN apt-get update & &\ apt-get-y install--no-install-recommends\ openjdk-11-jdk & &\ rm-rf / var/lib/apt/lists/* COPY target/my-app-1.0-SNAPSHOT.jar / app CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] use official images whenever possible

There are many reasons to use official images, such as reducing image maintenance time and image size, and pre-configuring images for use by containers.

FROM openjdk COPY target/my-app-1.0-SNAPSHOT.jar / app CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] uses specific tags

Do not use latest tags.

FROM openjdk:8 COPY target/my-app-1.0-SNAPSHOT.jar / app CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] looking for the smallest image

The following is a list of openjdk images. Choose the lightest mirror image that suits you best.

REPOSITORY TAG tag SIZE size openjdk 8 634MB openjdk 8-jre 443MB openjdk 8-jre-slim 204MB openjdk 8-jre-alpine 83MB is built from source in a consistent environment

If you don't need the entire JDK, you can use the Maven Docker image as the basis for building.

FROM maven:3.6-jdk-8-alpine WORKDIR / app COPY pom.xml. COPY src. / src RUN mvn-e-B package CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] gets the dependency in a separate step

Can be cached-the Dockerfile command used to get dependencies. Caching this step will speed up the build.

FROM maven:3.6-jdk-8-alpine WORKDIR / app COPY pom.xml. RUN mvn-e-B dependency:resolve COPY src. / src RUN mvn-e-B package CMD ["java", "- jar", "/ app/my-app-1.0-SNAPSHOT.jar"] Multi-phase build: delete build dependencies

Why use a multi-phase build?

Separate the build from the runtime environment

DRY mode

With different details of development, testing, and other environments

Linearized dependency relation

Has a platform-specific phase

FROM maven:3.6-jdk-8-alpine AS builder WORKDIR / app COPY pom.xml. RUN mvn-e-B dependency:resolve COPY src. / src RUN mvn-e-B package FROM openjdk:8-jre-alpine COPY-- from=builder / app/target/my-app-1.0-SNAPSHOT.jar / CMD ["java", "- jar", "/ my-app-1.0-SNAPSHOT.jar"]

If you build our application at this time

Time DOCKER_BUILDKIT=1 docker build-no-cache-t docker-class. 0 user 41s cpu 54s system 2% cpu 35656 total

You will notice that it takes about 35.66 seconds to build our application. This is a pleasant progress.

Next, we will introduce the functionality of other scenarios.

Multi-stage Construction: different Mirror Styles

The following Dockerfile shows the different stages of Debian-based and Alpine-based mirroring.

FROM maven:3.6-jdk-8-alpine AS builder... FROM openjdk:8-jre-jessie AS release-jessie COPY-- from=builder / app/target/my-app-1.0-SNAPSHOT.jar / CMD ["java", "- jar", "/ my-app-1.0-SNAPSHOT.jar"] FROM openjdk:8-jre-alpine AS release-alpine COPY-- from=builder / app/target/my-app-1.0-SNAPSHOT.jar / CMD ["java", "- jar", "/ my-app-1.0-SNAPSHOT.jar"]

To build a specific image, we can use the-target parameter:

Time docker build-no-cache-target release-jessie. Different mirror styles (DRY / global ARG) ARG flavor=alpine FROM maven:3.6-jdk-8-alpine AS builder... FROM openjdk:8-jre-$flavor AS release COPY-- from=builder / app/target/my-app-1.0-SNAPSHOT.jar / CMD ["java", "- jar", "/ my-app-1.0-SNAPSHOT.jar"]

The ARG command specifies the image to build. In the above example, we specify alpine as the default image, but we can also specify the image with the-build-arg flavor= parameter in the docker build command.

Time docker build--no-cache-target release-build-arg flavor=jessie. Concurrence

Concurrency is important when building Docker images because it takes full advantage of available CPU threads. In a linear Dockerfile, all phases are performed sequentially. With multi-phase builds, we can get smaller dependency phases ready for the master phase to use them.

BuildKit even brings another performance benefit. If this phase is not used in future builds, these phases will be skipped at the end, rather than being processed and discarded.

Here is an example Dockerfile, where the site's assets are built in an assets phase:

FROM maven:3.6-jdk-8-alpine AS builder... FROM tiborvass/whalesay AS assets RUN whalesay "Hello DockerCon!" > out/assets.html FROM openjdk:8-jre-alpine AS release COPY-- from=builder / app/my-app-1.0-SNAPSHOT.jar / COPY-- from=assets / out/assets CMD ["java", "- jar", "/ my-app-1.0-SNAPSHOT.jar"]

This is another Dockerfile where the C and C + + libraries are compiled and used later in builder.

FROM maven:3.6-jdk-8-alpine AS builder-base... FROM gcc:8-alpine AS builder-someClib... RUN git clone... . / configure-- prefix=/out & & make & & make install FROM g++:8-alpine AS builder-some CPPlib... RUN git clone... & & cmake... FROM builder-base AS builder COPY-- from=builder-someClib / out / COPY-- from=builder-someCpplib / out / BuildKit application cache

BuildKit has a special feature of package manager caching. Here are some examples of cache folder locations:

Package Manager path

Apt/ var/lib/apt/lists go ~ / .cache/go-build go-modules $GOPATH/pkg/mod npm ~ / .npm pip ~ / .cache/pip

We can compare this Dockerfile with the Dockerfile described above from source code builds in a consistent environment. This earlier Dockerfile has no special caching processing. We can use-mount=type=cache to do this.

FROM maven:3.6-jdk-8-alpine AS builder WORKDIR / app RUN-- mount=target=. -- mount=type=cache,target / root/.m2\ & & mvn package-DoutputDirectory=/ FROM openjdk:8-jre-alpine COPY-- from=builder / app/target/my-app-1.0-SNAPSHOT.jar / CMD ["java", "- jar", "/ my-app-1.0-SNAPSHOT.jar"] BuildKit security features

BuildKit has security features, and in the following example, we use-mount=type=secret to hide some confidential files, such as ~ / .aws / credentials.

FROM RUN... RUN-mount=type=secret,id=aws,target=/root/.aws/credentials,required\. / fetch-assets-from-s3.sh RUN. / build-scripts.sh

To build this Dockerfile, you need to use the-secret parameter:

Docker build-secret id=aws,src=~/.aws/credentials

And to improve security and avoid using commands such as COPY. / keys/private.pem / root .ssh / private.pem, we can use ssh in BuildKit to solve this problem:

FROM alpine RUN apk add-- no-cache openssh-client RUN mkdir-p-m 0700 ~ / .ssh & & ssh-keyscan github.com > > ~ / .ssh/known_hosts ARG REPO_REF=19ba7bcd9976ef8a9bd086187df19ba7bcd997f2 RUN-- mount=type=ssh,required git clone git@github.com:org/repo / work & & cd / work & & git checkout-b $REPO_REF

To build this Dockerfile, you need to load your SSH private key in ssh-agent.

Eval $(ssh-agent) ssh-add ~ / .ssh/id_rsa # this is the SSH key default location docker build-- ssh=default. After reading the above, do you have any further understanding of how BuildKit optimizes the construction of Dockerfile? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.