Shulou(Shulou.com)05/31 Report--

This article focuses on "how to build a CI-CD system based on drone". Interested friends may wish to have a look. The method introduced in this paper is simple, fast and practical. Let's let the editor take you to learn how to build a CI-CD system based on drone.

Overview of kubernetes Cluster three-step installation CI defines the entire workflow with a descriptive configuration

Programmers are lazy animals, so think of various ways to solve the problem of repetitive work. If you are still repeating something in your workflow, you may have to think about how to optimize it.

Continuous integration can help us solve repetitive code building, automated testing, release and other repetitive work, through a simple action to submit code, solve a lot of things to be done next.

Container technology makes all this more perfect.

A typical scenario:

We write a front-end project, assuming that it is based on the framework of vue.js. After submitting the code, we want to run the test case, and then build compresses one to the dist directory, and then use nginx to proxy the static files in this directory. Finally, make a docker image and put it in the image warehouse. You can even add a process that runs online.

Now I'm telling you, all you need is a git push action, and the CI tool will help you with everything else! If you haven't used such a system, what are you waiting for? Next, we will systematically introduce all this to you.

Code warehouse management

First of all, SVN this kind of slag software should be eliminated as soon as possible, there is nothing to say, there is really no need for git to exist SVN.

So we choose a git warehouse, more git warehouses, I can choose gogs,gitea gitlab here, according to the needs of their own choice

Docker run-d-- name gogs-time-v / etc/localtime:/etc/localtime-e TZ=Asia/Shanghai-- publish 8022 TZ=Asia/Shanghai 22\-- publish 3000 TZ=Asia/Shanghai 3000-- volume / data/gogs:/data gogs:latest

Access port 3000, and then it's not as powerful as gogs, but it takes up less resources and is fast, and we've been running steadily for several years. The disadvantage is that the API is not complete.

CI tool

When you have used drone.

Install:

Version: '2'services: drone-server: image: drone/drone:0.7 ports:-80 image 8000 volumes:-/ var/lib/drone:/var/lib/drone/ restart: always environment:-DRONE_OPEN=true-DOCKER_API_VERSION=1.24-DRONE_HOST=10.1.86.206-DRONE_GOGS=true-DRONE_GOGS_URL= http://10.1.86. 207drone/drone:0.7 command 3000 / # Code Repository address-DRONE_SECRET=ok drone-agent: image: drone/drone:0.7 command: agent restart: always depends_on:-drone-server volumes:-/ var/run/docker.sock:/var/run/docker.sock environment:-DOCKER_API_VERSION=1.24-DRONE_SERVER=ws://drone-server:8000/ws/broker-DRONE_SECRET=ok

Docker-compose up-d and then, you know, there is no such thing.

Log in to drone with your gogs account

Each step is a container, each plug-in is also a container, a variety of combinations, it is simply movable type printing

I won't dwell on how to use this elementary superficial content, but there are many pitfalls:

Machines with drone can use aufs as much as possible, but some plug-ins in device mapper can not run away, such as some plug-ins in docker in docker. This is not a fault of drone. It can only be said that docker is not good enough for docker in docker support.

Centos's support for aufs is not good enough. If you want to support aufs with centos, you'll have to struggle. Here's the community solution: https://github.com/sealyun/kernel-ml-aufs.

The most recommended is that the machine kernel of drone is upgraded to above 4.9, and then docker uses the overlay2 storage driver. The author has also practiced the high version kernel running container for a long time, which is much more stable than the low kernel.

Installation mode 2, install on k8s:

Helm install stable/drone usage articles

First, create three new files in the main directory of your code repository:

.drone.yml: describes the process of building and deploying (narrow sense), process configuration file (broad sense) CI/CD has no essential difference

Dockerfile: tells you how to package your application into an image. Of course, you don't need it if it's not containerized delivery.

K8s yaml configuration file or docker-compose file or chart package: tells your application how to deploy

Others: such as kube-config etc.

Log in to the drone page with the gogs account password and synchronize the project to see the project list. Turn on the switch to connect to the git warehouse. It is relatively simple to explore on your own.

Official example pipeline: backend: # name of a step You can use any full name image: golang # the essence of each step is to start a container based on this image commands: # execute some commands in this container-go get-go build-go test frontend: image: node:6 commands:-npm install-npm test publish: image: plugins/docker repo: octocat/hello-world tags: [1,1.1] Latest] registry: index.docker.io notify: image: plugins/slack channel: developers username: drone

The container started by each step shares the workdir volume, so that the resulting product of the build step can be used in the publish container.

Combined with Dockerfile, we can see:

# docker build-- rm-t drone/drone. From drone/ca-certsEXPOSE 8000 9000 80 443ENV DATABASE_DRIVER=sqlite3ENV DATABASE_CONFIG=/var/lib/drone/drone.sqliteENV GODEBUG=netdns=goENV XDG_CACHE_HOME / var/lib/droneADD release/drone-server / bin/ # because the working directory is shared, the product of build can be used in publish, so build and release can separate ENTRYPOINT ["/ bin/drone-server"]

The above mentioned separation of build and release is very useful, for example, when building golang code, we need a go environment, but only one executable is needed online or at run time

So you don't have to FROM a basic image of golang in Dockerfile, making your image smaller. For example, java needs maven to build, but not to run online.

So it can also be separated.

When using drone, you should use your imagination. Don't use it to death. Every sentence above needs to be read and understood carefully. Summarize the key points again:

Drone itself is no matter what the function of each step, only a fool to help you pick up the container, run normal and then perform the next step, failure will be terminated.

The functions of compiling, submitting to the image repository, deployment, notification and so on are all determined by the function of the image. The function of the container is called the plug-in in drone. The plug-in is essentially an image. There is a small difference between them.

This means that you can create whatever image you want. For example, if you need maven when compiling, then do a maven image. When you deploy, you need to interface with K8s, so get an image with kubectl client; if you want to deploy a physical machine, get an image of ansible, and so on. Use your imagination and use flexibly.

Drone environment variable

Sometimes we want the docker image tag from CI to be the same as git's tag. The advantage is that it is convenient to know which version of code to run, upgrade, etc., but it is obviously annoying to modify the pipeline file every time, so drone can have many environment variables to help us solve this problem:

Pipeline: build: image: golang:1.9.2 commands:-go build-o test-- ldflags'- linkmode external-extldflags "- static" 'when: event: [push, tag Deployment] publish: image: plugins/docker repo: fanux/test tags: ${DRONE_TAG=latest} dockerfile: Dockerfile insecure: true when: event: [push, tag, deployment]

This example ${DRONE_TAG=latest} if the git tag event triggers pipeline, then use git tag as a mirror tag, otherwise use latest, so that we can always iterate with latest during our research and development process, feel that the version is almost ready, type tag, and generate an image that can be tested by testers. It is very elegant, there is no need to change anything, and it is not easy to make mistakes.

Similarly, there are many other environment variables available, such as the commitID branch information of git, which can be found here.

Docking k8s practice

First of all, there must be a K8s cluster, then the first choice: kubernetes cluster three-step installation advertisement, just ignore it.

With the above groundwork, it is quite simple to interface with k8s: create a kubectl image and embed it in the process:

Put the K8s yaml file of the project into the code, and then apply directly in pipelie

Publish: image: plugins/docker # Image Warehouse, execute Dockerfile plug-in tags:-${DRONE_TAG=latest} insecure: true # copy deploy: image: kubectl:test # this image can be typed by yourself commands:-cat test.yaml-ls-rm-rf / root/.kube & & cp-r. Kube / root # k8s kubeconfig file, there can be multiple Copy the kubeconfig file to which cluster you deploy to-kubectl delete-f test.yaml | | true-kubectl apply-f test.yaml

But there are a few more details about best practices:

The kubeconfig file of K8s is also placed in the code directory (this is not very secure, but the warehouse is private, and you can use drone's secret to expand it in detail)

How to configure the image in the yaml file deployed in K8s? How uncomfortable it is to modify test.yaml every time.

What if there is a difference in the yaml configuration of multiple clusters? Write multiple yaml? Cause chaos, very inelegant

So we introduced chart and deployed it with helm:

ApiVersion: extensions/v1beta1kind: Deploymentmetadata: name: test namespace: {{.Values.namespace}} spec: replicas: {{.Values.replicaCount}} template: metadata: labels: name: test spec: serviceAccountName: test containers:-name: test image: "{{.Values.image.repository}: {{.Values.image.tag}" # the yaml file of deployment is a template Pass parameters to render imagePullPolicy: {{.Values.image.pullPolicy}} when you create it.

Note that with the template, we do not need to change the yaml file when we deploy the v1 and v2 versions, which reduces the risk of error. The environment variables are passed in when pipeline is executed, which is solved perfectly.

In this way, the git tag image tag and the image configuration in yaml are completely unified:

Deploy_dev: # deploy to development environment image: helm:v2.8.1 commands:-mkdir-p / root/.kube & & cp-r. Kube/config-test101.194 / root/.kube-helm delete test-- purge | | true-helm install-- name test-- set image.tag=$ {DRONE_TAG=latest} Chart when: event: Deployment environment: deploy_dev deploy_test: # deploy to the test environment image: helm:v2.8.1 commands:-mkdir-p / root/.kube & & cp-r. Kube/config-test101.84 / root/.kube # two environments use different kubeconfig-helm delete test-- purge | | true-helm install-- name test-- Set image.tag=$ {DRONE_TAG=latest} Chart # passes git tag to helm The image running in this way is the image built during publish, and the tag is consistent with when: event: deployment environment: deploy_test

Above, the above problems are solved elegantly.

Details: event can be a git event or a manual penalty event. It is triggered manually when the type is deployment. Drone supports command line triggering.

We have done secondary development so that drone can trigger the corresponding event on the page.

Principle chapter

When a warehouse is opened on drone, a webhook will be set for the warehouse, which can be seen in the project settings, so that the event of git can be notified to drone,drone to pull the replacement code according to the event.

Basic principles of pipeline

Understanding the principle is very important for using this system, otherwise something will be used to death.

Pipeline is only in charge of the container, and the system of the container does not care. The user decides that this sentence has been emphasized more than once in this article, and it is very important to read it several times.

Plug-in principle

Images are plug-ins, that is, many existing images can be embedded in the drone process directly as plug-ins.

A small difference is that you will find that some plug-ins in drone also have some parameters, which means that they have done more disgraceful things than ordinary images, such as playing the image of docker when publish:

Publish: image: plugins/docker repo: octocat/hello-world tags: [1, 1.1, latest] registry: index.docker.io

You will find that it has parameters such as repo tags, in fact, drone processing is very simple, is to convert these parameters into environment variables to the container, and then the container to process these parameters. The essence is to do this:

Docker run-- rm\-e PLUGIN_TAG=latest\-e PLUGIN_REPO=octocat/hello-world\-e DRONE_COMMIT_SHA=d8dbe4d94f15fe89232e0402c6e8a0ddf21af3ab\-v $(pwd): $(pwd)\-w $(pwd)\-- privileged\ plugins/docker-- dry-run

It's easy to customize a plug-in, as long as you write a script that can handle specific environment variables, such as a plug-in for curl:

Pipeline: webhook: image: foo/webhook url: http://foo.com method: post body: | hello world

Write a script.

#! / bin/shcurl\-X ${PLUGIN_METHOD}\ # handles several environment variables-d ${PLUGIN_BODY}\ ${PLUGIN_URL} FROM alpineADD script.sh / bin/RUN chmod + x / bin/script.shRUN apk-Uuv add curl ca-certificatesENTRYPOINT / bin/script.shdocker build-t foo/webhook .docker push foo/webhook

Turn it into a docker image, and you're done.

So in most cases, we will be lazy to write nothing, just execute the command in the container, which is also a requirement of curl, if we don't write plug-ins.

Pipeline: webhook: image: busybox # finish with busybox command:-curl-X POST-d 123 http://foo.com directly, and don't even bother to develop plug-ins

It is worth noting that some complex functions still need to be developed plug-ins, such as plug-ins used in publish mirroring. With regard to this plug-in, I would like to add that there is a docker engine in docker, which is mirrored by docker engine in docker, so the devicemapper storage driver cannot be supported. Please upgrade the kernel with overlay2 or ubuntu with aufs

At this point, I believe you have a deeper understanding of "how to build a CI-CD system based on drone". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.