Shulou(Shulou.com)06/01 Report--

This article mainly introduces how to install containerd and openfaas on tinycorelinux. It is very detailed and has a certain reference value. Interested friends must read it!

Why containerization is so important is because containers are now the most popular native virtual cloud appliance (app deployment-level convergence represents "creating a package structure for cloud APP". K8s are called cloud native so you can simply understand them as cloud native software packages, cloud appliance to be different from internal cloud appstack fusion for app development) as a unified deployment solution. It mainly focuses on solving the problem of "isolation of software distribution resource quotas" on clusters and clouds. Software resource quotas have always been a complex related problem. Coincidentally, these problems also exist in local and native package management software (packages mainly focus on solving dependency problems). A unified set of kernel-level support solutions (such as liblxc,libcgroups) are provided on linux. Based on them + brtfs, you can completely invent a simple docker runtime with shell, which, of course, cannot be compared with the final complete container and container management system containerd,openfaas). On the other hand, "resource isolation" goes a little further, it is easy to be associated with the problems such as "how to start the software" and become a problem to be solved by software such as systemd (systemd-nspawn can create the lightest containers) Then it becomes a problem to be solved by software such as k8s. Therefore, the integration and correlation of these three major problems occur in all aspects, and it is easy to become the central problem to be solved by some kind of "hybrid package management and container management" fusion system, and then need such a kind of software. Core os's https://github.com/rkt/rkt is the representative of this kind of software and builds an OS based on container as package management (although they are ready for dreprecate in mid-2020).

That is to say, containerization can be simple or complex, and different OS also have solutions to integrate different complex container management. In addition to core os's rkt, tc's tce pkg itself is a sandboxie environment, but it has nothing to do with the real containerization of lxc,ovz,containerd mentioned above. The previous articles related to openfaas are examples of installing openfaas in a popular linux distribution, and the following article will introduce the installation practice of trying to install real containers in tinycorelinux11, namely containerd and openfaas.

The main problem here is that tc itself is a raw linux distribution that pursues small and simple. Generally speaking, like alpine, tc is often used as a container guest os such as boot2docker, and containerd is rarely installed on tc as a container server environment. Therefore, installing containers in tc may be cumbersome because there is no ready-made reference solution. For example, tc does not use a complex init startup management system such as systemd, but instead a simple sysv init (although systemd proposes a huge init pid 1, it only focuses on "startup", which is in line with kiss). Generally speaking, linux distributions rely on systemd to deal with some crucial basic issues of container settings. For example, we will talk about systemd automatically managing cgroups, later, which are not available in tc and may not be solved manually.

Anyway, let's try it out. Our test environment is tc11.

1, create a containerd.tcz and faasd.tcz

Prepare a basic ezremasterd tc11 iso that integrates openssh,sudo passwd tc and bootcode tce=sda1,echo / opt/tcemirror,/etc/passwd,/etc/shadow > / mnt/sda1/opt/.filetool.lst,tar into restore=sda1 mydata.gz, that is, "A rootbuild packer script for fully retryable, build matecloudos (2) from 0" like the first section of iso. We start two virtual machines that boot this iso, and both use parted to sda1. These two virtual machines prepare one to generate containerd.tcz and faasd.tcz (python-m SimpleHTTPServer 80 in its directory for later download, and ifconfig is optimistic about ip in advance), another test generated tcz (/ opt/tcemirror is set to the correct structure of the first address + 11.x/x86_64/tcz), and arrange these files in the / mnt/sda1 root directory of the first virtual machine:

The preparation folder structure and binaries:docker are combined using the version of the previous article (do two folders and a containerd-root, in which cni is placed in / opt/cin/bin,runc / usr/local/sbin,containerd in / usr/local/bin, a faasd-root in / usr/local/bin/faasd,faas-cli, and finally, several offline docker image mentioned above are also integrated in faasd-root/tmp/*.tar). These exe set chmod + x, because these exe are go, are statically linked, and can be run directly on tc11 (lib64 must ln-s to lib, otherwise ctr will not work). Prepare several configuration files: some dynamic files for containerd and faasd startup, without creating them. Otherwise, it will cause the read-only file system to fail to write error / etc/cni/net.d/10-openfaas.conflist/var/lib/faasd/secrets/*/var/lib/faasd/resolv.conf/var/lib/faasd-provider/resolv.conf these files must be / var/lib/faasd/docker-compose.yaml/var/lib/faasd/prometheus.yml prepared overlay module: tce-load-iw bc compiletc perl5 recompile kernel in the first set, open config_overy_fs to m in config Get overlay module because it is closed in tc11. Download the required kernel compilation file http://mirrors.163.com/tinycorelinux/11.x/x86_64/release/src/kernel/ in tc11 to / mnt/sda1 to extract and cp config-5.4.3-tinycore64 linux-5.4.3/.configsudo make oldconfig, prompting several interactive items to directly enter sudo make installsudo make modules_install to put the resulting overlay module file (in / lib/modules/fs) into the containerd.tcz folder structure that is ready to be packaged. Prepare two service files and chmod + x:containerd-root/usr/local/init.d/start-containerd:/sbin/modprobe overlay/usr/local/bin/containerdcontainerd-root/usr/local/init.d/start-faasd:for I in 1 23; do [[!-z "$(ctr image list | grep basic-auth-plugin)"]] & & break;ctr-- address=/run/containerd/containerd.sock image import / tmp/faasd-containers/basic-auth-plugin-0.18.18.tar Echo "checking basic-auth ($I), if failed at 3jit may require a reboot"; sleep 3done for i in 1 23 address=/run/containerd/containerd.sock image import do [!-z "$(ctr image list | grep nats)"] & & break;ctr-- address=/run/containerd/containerd.sock image import / tmp/faasd-containers/nats-streaming-0.11.2.tar;echo "checking nats ($I), if failed at 3JIT may require a reboot"; sleep 3 Done for i in 1 23 sleep do [!-z "$(ctr image list | grep prometheus)"] & & break;ctr-- address=/run/containerd/containerd.sock image import / tmp/faasd-containers/prometheus-v2.14.0.tar;echo "checking prometheus ($I), if failed at 3 done it may require a reboot"; sleep 3 done for i in 1 2 3 Do [[!-z "$(ctr image list | grep gateway)]] & & break;ctr-- address=/run/containerd/containerd.sock image import / tmp/faasd-containers/gateway-0.18.18.tar;echo" checking gateway ($I), failed at 3 Magi it may require a reboot "; sleep 3 done for i in 1 23 X do [!-z" $(ctr image list | grep queue-worker) "] & & break Ctr-address=/run/containerd/containerd.sock image import / tmp/faasd-containers/queue-worker-0.11.2.tar;echo "checking queueworker ($I), if failed at 3 if failed at it may require a reboot"; sleep 3 Donecd / var/lib/faasd/usr/local/bin/faasd provider/usr/local/bin/faasd up prepares mkall.sh to put chmod + x under the root of / mnt/sda1: the contents of mkall.sh (note that the unity is stored in tcz as tc:staff): rm-rf containerd.tcz containerd.tcz.md5.txt faasd.tcz faasd.tcz.md5.txt faasd.tczmksquashfs containerd-root containerd.tcz-noappend-no-fragments-force-uid tcmd5sum containerd.tcz > containerd.tcz.md5. Txtmksquashfs faasd-root faasd.tcz-noappend-no-fragments-force-uid tc-force-gid staffmd5sum faasd.tcz > faasd.tcz.md5.txtecho containerd.tcz > faasd.tcz.dep

The packaged tcz of sudo. / mkall.sh is more than 80 m each. When ready, we can test it in the second set. After tce-load-iw faasd, if you are not satisfied with the test, you can delete / restart the tcz under mnt/sda1/tce/optional. We need to constantly mkall and test the two tcz generated.

2, test

The first time the test starts start-containerd,start-faasd, it appears: no cgroup mount found in mountinfo: unknown this is the logic mentioned above that tc11 does not have the logic to automatically handle cgroups. And containerd depends on them. The basics of linux's kernel support for containers are provided in kernel config in tc11, but not further.

CGroup provides a CGroup virtual file system as a user interface for grouping management and subsystem settings. To use CGroup, the CGroup file system must be mounted. At this point, you specify which subsystem to use through the mount option. It should be noted that in operating systems using the systemd system, the / sys/fs/cgroup directory is mounted by systemd during system startup, and the mount is read-only. In other words, it is not recommended that we create a new directory under the / sys/fs/cgroup directory and mount other subsystems. This is quite different from the previous operating system.

In view of this, we are lucky to find https://gitee.com/binave/tiny4containerd/blob/master/src/rootfs/usr/local/etc/init.d/cgroupfs.sh, which uses old docker and is based on https://github.com/tianon/cgroupfs-mount/ (the lvm dynamic extension partition script and docker service here in this project https://gitee.com/binave/tiny4containerd/src/rootfs/usr/local/, cert processing and other functions are also good, which can be used in the future) There are a few sentences in it.

Mount-t tmpfs-o uid=0,gid=0,mode=0755 cgroup / sys/fs/cgroup if you don't have the above sentence mount, then you will mkdir: can't create directory 'cpu': No such file or directory, because / sys/fs/cgroup is just a fake fscd / sys/fs/cgroup;# get/mount list of enabled cgroup controllersfor sys in $(awk'! / ^ # / {if ($4 = = 1) print $1}'/ proc/cgroups) given by the kernel; do mkdir-p $sys if! _ mountpoint-Q $sys Then if! Mount-n-t cgroup-o $sys cgroup $sys; then rmdir $sys | | true fi fidone

We put cgroupfs side by side with containerd-root/usr/local/etc/init.d/containerd and add the phrase / usr/local/etc/init.d/cgroupfs.sh mount before starting containerd in the containerd script. Packaging and retesting: jailing process inside rootfs caused: pivot_root invalid argument: unknown (I have never tested whether the docker in https://gitee.com/binave/tiny4containerd/ will have this error, but I heard that I encountered https://forums.docker.com/t/tinycore-8-0-x86-pivot-root-invalid-argument/32633),

In a system running entirely in memory, after an upgrade from 17.09.1-ce to 17.12.0-ce, docker stopped creating containers, failing with message like docker: Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:398: container init caused" rootfs_linux.go:107: jailing process inside rootfs caused\ "pivot_root invalid argument\": unknown..

Check the Internet said to use DOCKER_RAMFS=true ring change, I tried to no avail.

Similar container products have been encountered on other non-tc: https://engineeringjobs4u.co.uk/how-we-use-hashicorp-nomad, for which they have made a kernel patch: https://lore.kernel.org/linux-fsdevel/20200305193511.28621-1-ignat@cloudflare.com/

The main need for this is to support container runtimes on stateless Linux system (pivot_root system call from initramfs). Normally, the task of initramfs is to mount and switch to a "real" root filesystem. However, on stateless systems (booting over the network) it is just convenient to have your "real" filesystem as initramfs from the start.

Add before the manual patch,mnt_init () definition of linux543/fs/namespace.c, and add the new code in the middle. But it didn't work. Therefore, the method of "using hashicorp packer to export dbcolinux to virtual machine and docker format (3)" is converted to the traditional hard disk installation method. Problem solved.

Then came the following: Error creating CNI for basic-auth-plugin: Failed to setup network for task "basic-auth-plugin-1210": failed to create bridge "openfaas0": could not add "openfaas0": operation not supported: failed to create bridge "openfaas0": could not add "openfaas0": operation not supported

This problem is actually expected, because from the experience of the previous article, we have always been concerned that the cni in containerd must work, but after faasd up produced 10openfaas.conflist, I have been trying ifconfig, but did not see the third network card.

Someone on the Internet suggested that it was CONFIG_BRIDGE_VLAN_FILTERING. Look at the kernel,config_bridge of tc11 as a module, and its file should be bridge.ko and so on. But modprobe bridge didn't work, tce-load-iw original-modules-5.4.3-tinycore64, and it worked. (after installing this package, the console shows that many devices have recognized it.)

Retest: Error: Failed to setup network for task "basic-auth-plugin-3894": failed to locate iptables: exec: "iptables": executable file not found in $PATH: failed to locate iptables: exec: "iptables": executable file not found in $PATH, tce-load-iw iptables is required

At this point, faad up started successfully. The containerd console shows warning,memory cgroup not supported, which should be kernel config, but it is not set up.

The above is all the contents of the article "how to install containerd and openfaas on tinycorelinux". Thank you for reading! Hope to share the content to help you, more related knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.