Shulou(Shulou.com)06/02 Report--

This article mainly introduces "how to build a Docker image and run everywhere". In the daily operation, I believe that many people have doubts about how to build a Docker image and run it everywhere. The editor consulted all kinds of materials and sorted out simple and easy-to-use operation methods. I hope it will be helpful to answer the questions of "how to build a Docker image and run everywhere". Next, please follow the editor to study!

Method 1: build directly on the target hardware

If we can access the target architecture hardware and have all the build data we need on the operating system, we can compile the application directly on the hardware.

For example, to build a multi-architecture Docker image for our specific scenario, we can install the Docker runtime environment on the raspberry pie and then build the image directly above through the application's Dockerfile, as on the development machine. This method is feasible because Raspbian, the official operating system of raspberry pie, supports local installation of Docker.

But what if we don't have easy access to the target hardware? Can we build applications with non-native architecture directly on the development machine?

Method 2: simulate the target hardware

Do you remember the happy time with 16 Nintendo consoles? I was just a child at the time, but when I got older, I found that I missed classic games such as Super Mary and Wheel of time and Space. I didn't get a chance to own a super Nintendo console, but thanks to simulators like ZSNES, I was able to go back and experience the fun of these classic games on a 32-bit PC.

Through the simulator, we can not only play video games, but also build non-local binaries. Of course, instead of using ZSNES, we use a more powerful and flexible simulator: QEMU. QEMU is a free and open source simulator that supports many common architectures, including ARM, Power-PC, and RISC-V. By running a full-function simulator, we can start a general-purpose ARM virtual machine that can run the Linux operating system, and then set up the development environment in the virtual machine and compile the application.

However, if you think about it carefully, a fully functional virtual machine is a waste of resources. In this mode, QEMU simulates the entire system, including hardware such as timers, memory controllers, SPI, and I2C bus controllers. But in most cases, we don't care about the hardware features mentioned above when we compile the application. Could it be better?

Method 3: simulate the user space of the target architecture through binfmt_misc

On Linux systems, QEMU has another mode of operation, which allows you to run non-native architecture binaries through a user-mode emulator. In this mode, QEMU skips the simulation of the entire target system hardware described in method 2 and instead registers a binary format handler in the Linux kernel through binfmt_misc to intercept and convert the unfamiliar binary code and then execute it, while converting the system call from the target system to the current system as needed. Eventually, for users, they will find that they can run these heterogeneous binary programs locally.

With user-mode simulators and QEMU, we can install other Linux distributions through lightweight virtualization (chroot or containers) and compile the heterogeneous binaries we need as if we were local.

We will see below that this will be an option for building multi-architecture Docker images.

Method 4: use a cross compiler

Finally, we also have a standard practice in the embedded system community: cross-compilation.

A cross-compiler is a special compiler that runs on the host architecture, but can generate binaries for different target architectures. For example, we could have a C++ cross-compiler with amd64 architecture, and the target architecture is an aarch74 (64-bit ARM) embedded device (such as a smartphone or something). A real-life example of this approach is that billions of Android devices around the world use it to build software.

In terms of performance, this approach is as efficient as building directly on the target hardware (method 1), because it does not run on the simulator. But the variables of cross-compilation depend on the programming language used, which is very convenient if it is Go.

Are you confused? It will be more complicated for Docker images.

Note that all the compilation methods mentioned above only generate a single application binary. For modern containers, when we introduce Docker images, it's not just about building separate binaries, it's about building an entire heterogeneous container image! This is more troublesome than what I said before.

If all this sounds painful, don't be sad, because building non-native platform binaries is already painful. On top of this, increasing the complexity of Docker seems to be left to experts.

Thanks to the experimental extensions brought about by the latest version of the Docker runtime environment, building multi-architecture images is now much more convenient than before.

Build a multi-architecture Docker image

To make it easier to build multi-architecture Docker images, we can use the recently released Docker extension: buildx. Buildx is the front end of the next generation of standard docker build commands, the familiar commands for building Docker images. With all the functions of BuildKit, buildx extends the function of the docker build command in the table and becomes the new back end of the Docker building system.

Let's take a few minutes to look at how to use buildx to build multi-architecture images.

Step 1: turn on buildx

To use buildx, first make sure that our Docker runtime environment is the latest version 19.03. In the new version, buildx is actually bundled with Docker by default, but it needs to be enabled by setting the environment variable DOCKER_CLI_EXPERIMENTAL. Let's open it in the current command line session:

$export DOCKER_CLI_EXPERIMENTAL=enabled

Verify that we can now use buildx by checking the version:

$docker buildx version github.com/docker/buildx v0.3.1-tp-docker 6db68d029599c6710a32aa7adcba8e5a344795a7

Optional step: build from source code

If you want to use the latest version of buildx, or if setting the DOCKER_CLI_EXPERIMENTAL environment variable does not take effect in the current environment (for example, I found that the setting is not valid in the Arch Linux system), we can build buildx from the source code:

$export DOCKER_BUILDKIT=1$ docker build-platform=local-o. Git://github.com/docker/buildx $mkdir-p ~ / .docker/cli-plugins & & mv buildx ~ / .docker/cli-plugins/docker-buildx

Step 2: enable binfmt_misc to run non-local architecture Docker images

If you are using the Docker desktop version of Mac or Windows, you can skip this step because binfmt_misc is enabled by default.

If you are using a Linux system, you need to set up binfmt_misc. In most distributions, this is simple, but you can now set it up more easily by running a privileged Docker container:

$docker run-rm-privileged docker/binfmt:66f9012c56a8316f9244ffd7622d7c21c1f6f28d

Verify that the binfmt_misc settings are correct by checking the QEMU handler:

$ls-al / proc/sys/fs/binfmt_misc/ total 0 drwxr-xr-x 2 root root 0 Nov 12 09:19. Dr-xr-xr-x 1 root root 0 Nov 12 09:16.. -rw-r--r-- 1 root root 0 Nov 12 09:25 qemu-aarch74-rw-r--r-- 1 root root 0 Nov 12 09:25 qemu-arm-rw-r--r-- 1 root root 0 Nov 12 09:25 qemu-ppc64le-rw-r--r-- 1 root root 0 Nov 12 09:25 qemu-s390x-w-1 root root 0 Nov 12 09:19 register-rw-r--r-- 1 root root 0 Nov 12 09:19 status

Then, verify that the specified schema handler is enabled, for example:

$cat / proc/sys/fs/binfmt_misc/qemu-aarch74 enabled interpreter / usr/bin/qemu-aarch74 flags: OCF offset 0 magic 7f454c460201010000000000000000000200b7 mask ffffffffffffff00fffffffffffffffffeffff

Step 3: switch the default Docker image builder to a multi-architecture builder

By default, Docker uses the old builder and does not support multi-schema builds.

To create a new builder that supports multiple architectures, run:

$docker buildx create-use-name mybuilder

Verify that the new builder is in effect:

$docker buildx ls NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS mybuilder * docker-container mybuilder0 unix:///var/run/docker.sock inactive default docker default default running linux/amd64, linux/arm64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6

Got it. Now Docker will use a new builder to support the construction of multi-architecture images.

Step 4: build a multi-architecture image

Well, now we can finally start building a multi-architecture image. To demonstrate this functionality, we need a sample application.

Let's create a simple Go application that outputs the architecture information for the current runtime environment:

$cat hello.go package main import ("fmt"runtime") func main () {fmt.Printf ("Hello,% s!\ n", runtime.GOARCH)}

Let's create a Dockerfile to containment the application

$cat Dockerfile FROM golang:alpine AS builder RUN mkdir / app ADD. / app/ WORKDIR / app RUN go build-o hello. FROM alpine RUN mkdir / app WORKDIR / app COPY-- from=builder / app/hello. CMD [". / hello"]

This is a multi-stage Dockerfile that builds our application through the Go compiler, and then creates a minimum image of the built binaries using the Alpine Linux image.

Now, let's use buildx to build a multi-architecture image that supports arm, arm64, and amd64 architectures, and push it to Docker Hub at once:

Docker buildx build-t mirailabs/hello-arch-- platform=linux/arm,linux/arm64,linux/amd64. -- push

Yeah, that's it. Now we have a multi-architecture Docker image on Docker Hub that supports arm, arm64, and amd64 architectures. When we run docker pull mirailabs/hello-arch, Docker gets the matching image based on the architecture of the machine.

If the reader asks, how does buildx achieve this magic? Well, behind the command, buildx creates three Docker images using QEMU and binfmt_misc (one each for arm, arm64, and amd64 schemas). When the build is complete, Docker creates a list containing the three images and their corresponding schemas. In other words, a "multi-architecture image" is actually a list of images corresponding to each architecture.

Step 5: test the multi-architecture image

Let's quickly test the multi-architecture images to make sure they all work. Because we have set up binfmt_misc, we are able to perform mirroring of any architecture on the development machine.

First, list the hash values for each mirror:

$docker buildx imagetools inspect mirailabs/hello-arch Name: docker.io/mirailabs/hello-arch:latest MediaType: application/vnd.docker.distribution.manifest.list.v2+json Digest: sha256:bbb246e520a23e41b0c6d38b933eece68a8407eede054994cff43c9575edce96 Manifests: Name: docker.io/mirailabs/hello-arch:latest@sha256:5fb57946152d26e64c8303aa4626fe503cd5742dc13a3fabc1a890adfc2683df MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/arm/v7 Name: docker.io/mirailabs/hello-arch:latest@sha256:cc6e91101828fa4e464f7eddec3fa7cdc73089560cfcfe4af16ccc61743ac02b MediaType: application/ Vnd.docker.distribution.manifest.v2+json Platform: linux/arm64 Name: docker.io/mirailabs/hello-arch:latest@sha256:cd0b32276cdd5af510fb1df5c410f766e273fe63afe3cec5ff7da3f80f27985d MediaType: application/vnd.docker.distribution.manifest.v2+json Platform: linux/amd64

With the help of these hash values, we can run the images one by one and observe their output:

$docker run-rm docker.io/mirailabs/hello-arch:latest@sha256:5fb57946152d26e64c8303aa4626fe503cd5742dc13a3fabc1a890adfc2683df Hello, arm! $docker run-rm docker.io/mirailabs/hello-arch:latest@sha256:cc6e91101828fa4e464f7eddec3fa7cdc73089560cfcfe4af16ccc61743ac02b Hello, arm64! $docker run-rm docker.io/mirailabs/hello-arch:latest@sha256:cd0b32276cdd5af510fb1df5c410f766e273fe63afe3cec5ff7da3f80f27985d Hello, amd64! At this point, the study on "how to build a Docker image and run it everywhere" is over. I hope it can solve everyone's doubts. The collocation of theory and practice can better help you learn, go and try it! If you want to continue to learn more related knowledge, please continue to follow the website, the editor will continue to work hard to bring you more practical articles!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.