Shulou(Shulou.com)06/01 Report--

How to build a multi-platform Docker image in buildx, many novices are not very clear about this. In order to help you solve this problem, the following editor will explain it in detail. People with this need can come and learn. I hope you can get something.

Method 1: compile directly on the target hardware

If you have access to the system of the target CPU architecture, and the operating system supports the various tools needed to run the build, you can compile the program directly on the target system.

To build a Docker image, for example, you can install Docker on a raspberry pie, and then build an image of the arm platform directly through Dockerfile on the raspberry pie.

What if I cannot access the system of the target CPU architecture? Is there a way to build programs that target CPU architecture directly on the current system in some way? Please see below.

Method 2: simulate the target hardware

Remember the arcade games we played in all kinds of Internet cafes, billiard rooms and other occasions when we were kids? Let me show you a picture to recall:

What if we want to relive the arcade game we played before? At this point, you need to use the Emulator. With the help of the simulator, we can turn back time and experience the fun of classic games.

In addition to playing games, the simulator can also be used to build programs across CPU architectures. The most commonly used simulator is the open source QEMU,QEMU that supports many common CPU architectures, including ARM, Power-PC, and RISC-V. By simulating a complete operating system, a general ARM virtual machine can be created, which can boot Linux, set up the development environment, or compile programs within the virtual machine.

However, it is a bit wasteful to simulate the entire operating system, because in this mode, QEMU will simulate the entire system, including timers, memory controllers, bus controllers, and other hardware. But the compiler doesn't need to care about this at all and can be more concise.

Method 3: simulate the user space of the target hardware through binfmt_misc

On Linux, in addition to simulating the complete operating system, QEMU has another mode called user-mode (User mod). In this mode, QEMU registers a binary conversion handler in the Linux kernel through binfmt_misc, and dynamically translates the binary file when the program is running, and converts the system call from the target CPU architecture to the current system CPU architecture as needed. The end result looks like running the binaries of the target CPU schema locally.

With QEMU's user-mode mode, we can create a lightweight virtual machine (chroot or container) and then compile the program in the virtual machine system, which is as easy as local compilation. As we will see later, this is the approach used to build Docker images across platforms.

Method 4: use a cross compiler

Finally, a method commonly used in embedded system community: cross-compilation (cross-compilation) is introduced.

A cross-compiler is a compiler specially designed to run on a given system platform, but can compile executable files for another system platform. For example, the C++ cross-compiler on Linux systems with amd64 architecture can compile executable files running on embedded devices running on aarch74 (64-bit ARM) architecture. To take another real example, the APP of Android devices is basically compiled in this way.

From a performance point of view, this method is no different from method one, because there is almost no performance loss without the participation of the simulator. But cross-compilation is not universal, and its complexity depends on the language the program uses, which is super easy if you use Golang.

In the national container era, we discussed building not only a single executable, but also a container image. And building a container image is more complex than the method mentioned above, coupled with the complexity of Docker itself, which is almost a long-standing problem.

However, with the introduction of a new experimental plug-in, it is much easier to build a Docker image of a multi-platform architecture, and what this plug-in is will be described in more detail below.

two。 Build a multi-platform Docker image

With the plug-in buildx introduced by Docker 19.03, you can easily build multi-platform Docker images. Buildx is docker build. The next-generation alternative to the command, which extends the functionality of BuildKit with all the features of docker build.

Here's how to use buildx to build Docker images for different platforms in just a few minutes. The steps are as follows:

Enable the buildx plug-in

To use buildx, first make sure that the Docker version is no less than 19.03, and also enable it by setting the environment variable DOCKER_CLI_EXPERIMENTAL. You can enable the buildx plug-in for the current terminal with the following command:

? → export DOCKER_CLI_EXPERIMENTAL=enabled

Verify that it is on:

? → docker buildx versiongithub.com/docker/buildx v0.3.1-tp-docker 6db68d029599c6710a32aa7adcba8e5a344795a7

If setting the environment variable DOCKER_CLI_EXPERIMENTAL does not take effect on some systems (such as Arch Linux), you can choose to compile from source code:

→ export DOCKER_BUILDKIT=1???? → docker build-- platform=local-o. Git://github.com/docker/buildx???? → mkdir-p ~ / .docker/cli-plugins & & mv buildx ~ / .docker/cli-plugins/docker-buildx enable binfmt_misc

> if you are using Docker desktop version (MacOS and Windows), you can skip this step if binfmt_misc is enabled by default.

If you are using Linux, you need to enable binfmt_misc manually. Most Linux distributions are easy to enable, but there is an easier way to run a privileged container with a setup script:

→ docker run-- rm-- privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d

> it is recommended to upgrade the Linux kernel version above 4.x, especially for CentOS users. You may encounter errors.

Verify whether binfmt_misc is enabled:

→ ls-al / proc/sys/fs/binfmt_misc/ Total dosage 0 Total dosage 0 root root 0 November 18 00:12 qemu-aarch74-rw-r--r-- 1 root root 0 November 18 00:12 qemu-arm-rw-r--r-- 1 root root 0 November 18 00:12 qemu-ppc64le-rw-r--r-- 1 root root 0 November 18 00:12 qemu-s390x--w- -1 root root 0 November 18 00:09 register-rw-r--r-- 1 root root 0 November 18 00:12 status

Verify that the appropriate processor is enabled:

→ cat / proc/sys/fs/binfmt_misc/qemu-aarch74enabledinterpreter / usr/bin/qemu-aarch74flags: OCFoffset 0magic 7f454c460201010000000000000000000200b7mask ffffffffffffff00fffffffffffffffffeffff switches from default builder to multi-platform builder

Docker uses builders that do not support multi-CPU architectures by default, and we need to switch them manually.

First create a new builder:

→ docker buildx create-- use-- name mybuilder

Start the builder:

→ docker buildx inspect mybuilder-- bootstrap [+] Building 5.0s (1 to 1) FINISHED = > [internal] booting buildkit 5.0s = > = > pulling image moby/buildkit:buildx-stable-1 4.4s = > = > creating container buildx_buildkit_mybuilder0 0.6sName: mybuilderDriver: docker-containerNodes:Name: mybuilder0Endpoint: Unix:///var/run/docker.sockStatus: runningPlatforms: linux/amd64 Linux/arm64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

Looking at the currently used builders and the CPU schemas supported by the builders, you can see that many CPU schemas are supported:

→ docker buildx lsNAME/NODE DRIVER/ENDPOINT STATUS PLATFORMSmybuilder * docker-container mybuilder0 unix:///var/run/docker.sock running linux/amd64, linux/arm64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6default docker default default running linux/amd64, linux/386 to build multi-platform images

Now we can build an image that supports a multi-CPU architecture! Suppose you have a simple golang program source code:

→ cat hello.gopackage mainimport ("fmt"runtime") func main () {fmt.Printf ("Hello,% s!\ n", runtime.GOARCH)}

Create a Dockerfile to containerize the application:

? → cat DockerfileFROM golang:alpine AS builderRUN mkdir / appADD. / app/WORKDIR / appRUN go build-o hello .from alpineRUN mkdir / appWORKDIR / appCOPY-- from=builder / app/hello .CMD [". / hello"]

This is a multi-stage build Dockerfile that uses the Go compiler to build the application and copies the built binaries into the alpine image.

Now you can use buildx to build a Docker image that supports arm, arm64, and amd64 multiple architectures, and push it to Docker Hub:

→ docker buildx build-t yangchuansheng/hello-arch-- platform=linux/arm,linux/arm64,linux/amd64. -- push

> you need to log in to the authentication Docker Hub through the docker login command in advance.

Now you can pull the image you just created through docker pull mirailabs/hello-arch, and Docker will pull the matching image according to your CPU architecture.

The principle behind it is also simple. As mentioned earlier, buildx builds three different images of three different CPU architectures (arm,arm64 and amd64) through QEMU and binfmt_misc. After the build is complete, a manifest list is created with pointers to the three mirrors.

If you want to save the built image locally, you can specify type as docker, but you must build different images for different CPU architectures, and cannot merge them into one image, that is:

→ docker buildx build-t yangchuansheng/hello-arch-- platform=linux/arm-o type=docker. → docker buildx build-t yangchuansheng/hello-arch-- platform=linux/arm64-o type=docker. → docker buildx build-t yangchuansheng/hello-arch-- platform=linux/amd64-o type=docker. Test multi-platform image

Since binfmt_misc was previously enabled, we can now run any Docker image of the CPU architecture, so we can test whether there is a problem with the three images generated earlier on the local system.

First, list the digests of each mirror:

? → docker buildx imagetools inspect yangchuansheng/hello-archName: docker.io/yangchuansheng/hello-arch:latestMediaType: application/vnd.docker.distribution.manifest.list.v2+jsonDigest: sha256:ec55f5ece9a12db0c6c367acda8fd1214f50ee502902f97b72f7bff268ebc35aManifests: Name: docker.io/yangchuansheng/hello-arch:latest@sha256:38e083870044cfde7f23a2eec91e307ec645282e76fd0356a29b32122b11c639 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/arm/v7 Name: docker.io/yangchuansheng/hello-arch:latest@sha256:de273a2a3ce92a5dc1e6f2d796bb85a81fe1a61f82c4caaf08efed9cf05af66d MediaType: application/vnd.docker .accountion.artic.v2 + json Platform: linux/arm64 Name: docker.io/yangchuansheng/hello-arch:latest@sha256:8b735708d7d30e9cd6eb993449b1047b7229e53fbcebe940217cb36194e9e3a2 MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/amd64

Run each image and observe the output:

→ docker run-- rm docker.io/yangchuansheng/hello-arch:latest@sha256:38e083870044cfde7f23a2eec91e307ec645282e76fd0356a29b32122b11c639Hello, armchair docker run? → docker run-- rm docker.io/yangchuansheng/hello-arch:latest@sha256:de273a2a3ce92a5dc1e6f2d796bb85a81fe1a61f82c4caaf08efed9cf05af66dHello, Arm64 baby baby? → docker run-- rm docker.io/yangchuansheng/hello-arch:latest@sha256:8b735708d7d30e9cd6eb993449b1047b7229e53fbcebe940217cb36194e9e3a2Hello, amd64! Is it helpful for you to read the above content? If you want to know more about the relevant knowledge or read more related articles, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.