Shulou(Shulou.com)06/02 Report--

First, background introduction

Under the background of rapid update and iteration of Internet applications, traditional manual or simple scripts can no longer adapt to this change. At this time, Devops provides us with a good solution. Good application of CI/CD can greatly facilitate our daily work. Automatic and rapid continuous integration / continuous delivery brings us faster application opening speed, better stability and stronger reliability.

II. Topological environment

2.1 Architecture Topology

As shown in the example above, it simply takes the following process topology:

When the push native code is developed to gitlab-server, webhook automatically triggers the jenkins build application to deploy the application on docker host, git clone from the gitlabserver source code, and starts the application front end to place lb to make a high available database connection to the cloud database. Log storage can be delivered to elk in the later stage of log to achieve log visualization construction to complete email notification to relevant personnel (test or open)

2.2 system software version

Name version Linux system CentOS7.3 64-bit Docker1.13Django2.0

III. Installation and deployment

3.1 Jenkins installation and deployment

For Jenkins installation and deployment, please refer to: jenkins Notes

Add Docker target server after installation is complete

Configure the mail sending server

3.2 Docker installation and deployment

For Docker installation, deployment and Dockerfile preparation, please refer to: container Docker details

3.3 Gitlab installation and deployment

GitLab is installed on the public network Linux server and runs some commands. If there is no public network, you need to manually modify the external_url 'http:// of the / etc/gitlab/gitlab.rb file to your own private network IP'.

Yum install-y libsemanage-static libsemanage-devel policycoreutils openssh-server openssh-clients postfixsystemctl enable postfix & & systemctl start postfixwget https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el7/gitlab-ce-8.0.0-ce.0.el7.x86_64.rpmrpm-I gitlab-ce-8.0.0-ce.0.el7.x86_64.rpm# obtains the public network IPPUBLICIP=$ (curl http://ipv4.icanhazip.com)# modifies sed-I "s / Gitlab-server/$ {PUBLICIP} / g "/ etc/gitlab/gitlab.rbgitlab-ctl reconfiguregitlab-ctl restartecho" Username:root "echo" Password etc/gitlab/gitlab.rbgitlab-ctl reconfiguregitlab-ctl restartecho 5iveLefe "

3.4 configure the release process

Jenkins newly built a free-style software project

Using parameterization to build source ports and release of Docker incoming mapping to facilitate subsequent deployment

Source code from gitlab's django project

Using webhook to associate gitlab and jenkins

The jenkins installation plug-in:

Generate random token values

Configure the GitLab webhook URL generated by jenkins to gitlab

When the developer automatically triggers the jenkins build project after the local push code, there is the git pull code written in the Dockerfile, so there is no need to distribute the code from the jenkins to the docker host. Jenkins is used to trigger the docker build.

Configure the mail after the build is completed

Message template, message type selection:

Content type selection: HTML

Fill in the email subject: build notice: ${BUILD_STATUS}-${PROJECT_NAME}-Build # ${BUILD_NUMBER}!

Build a notification template:

${ENV, var= "JOB_NAME"}-the ${BUILD_NUMBER} build log (this email is automatically sent by the program, please do not reply!) Build result-${BUILD_STATUS} build Information Project name: ${PROJECT_NAME} build number: ${BUILD_NUMBER} build SVN version: ${SVN_REVISION} trigger cause: ${CAUSE} build log: ${BUILD_URL} console build Url: ${BUILD_URL} working directory: ${PROJECT_URL} ws Project Url: ${PROJECT_URL} Changes Since Last Successful Build: historical change record: ${PROJECT_URL} changes ${CHANGES_SINCE_LAST_SUCCESS Reverse=true, format= "Changes for Build #% nvv% c", showPaths=true,changesFormat= "[% a]% m", pathFormat= "% p"} Failed Test Results $FAILED_TESTS build log (last 100 lines): ${BUILD_LOG, maxLines=100}

The trigger type can be entered according to your own needs. Enter always here to send email regardless of success or failure.

View files in a remote Docker server

The django deployment has packaged the project's Python3.6 environment package with conda and made a Docker image.

In the past, the pure Python3.6 system was used, and the pip was used to install the requirements.txt module each time, but in the long run, due to little change in the environment, each pip installation was time-consuming, so the packaged Python environment was self-made into a customized environment using conda to reduce the environment deployment time. You can also mount the local disk in the environment through the docker image creation is-v parameter, and you can build the local conda each time. Complete rapid environment deployment.

View Dockerfile

FROM 87a69025db6aMAINTAINER kaliarch# definition working directory ENV WORK_DIR / work/# in docker create working directory in docker RUN mkdir $WORK_DIR# definition mapping port EXPOSE 80WORKDIR $WORK_DIR RUN git clone http://123.xxxx.xxxxx.245/Devops/go2cloud.git# add startup service script ADD * .sh ${WORK_DIR} CMD `which ba sh` / work/start_all.sh & tail-f / work/logs/server-$ (date +% F). Log

View the Django startup script

#! / bin/bashBASEPATH=$ (cd `dirname $0` Pwd) PY_CMD=/python3/bin/python# service entry file # MAIN_APP=$ {BASEPATH} / go2cloud/manage.py # migration script entry file SCRIPTS_APP=$ {BASEPATH} / go2cloud/scripts/migrate_task_schdule.py# delete script entry file DELETE_APP=$ {BASEPATH} / go2cloud/scripts/delete_transfer_server.py# log directory LOG_DIR=$ {BASEPATH} / logs/ [!-d ${LOG_DIR}] & & Mkdir ${LOG_DIR} # Startup Service # nohup ${PY_CMD}-u ${MAIN_APP} runserver 0.0.0.0MAIN_APP 80 > > ${LOG_DIR} server-$ (date +% F). Log 2 > & 1 & # startup script migration scheduling script echo "- $0 $(date) excute-" > > ${LOG_DIR} task-script-$ (date +% F) .lognohup ${PY_CMD}-u ${SCRIPTS_APP} > > ${LOG_DIR} script-$ (date +% F). Lognohup 2 > & 1 & # start the migration deletion script echo "- $0 $(date) excute-" > > ${LOG_DIR} delete-script-$ (date +% F). Lognohup ${PY_CMD}-u ${DELETE_APP} > > ${LOG_DIR} delete -script-$ (date +% F). Log 2 > & 1 &

View the jenkins deployment script

#! / bin/bashrelease=$1port=$2BASEPATH=$ (cd `dirname $0`; pwd) # build the go2cloud-platform image cd / dockerworkdocker build-t go2cloud-platform-mini:$release. IMGNAME = $(docker images | awk-v release=$release'{if ($1pm = "go2cloud-platform-mini" & & $2==release) print $3}') echo $IMGNAME# launch container docker run-Dmurp ${port}: 80-v / testlog/:/work/logs ${IMGNAME}

Use the-v parameter to store logs persistently on the docker host

Fourth, test and display

4.1 Test build

Build the test manually

4.2 View log

4.3 View docker Container

4.4 Test app

5. Reflection and improvement

At present, the database connected to the database is built for the cloud server, and the later database also uses docker. Several groups use docker-compose for unified deployment and management. In the later stage, you can use the public cloud K8s cluster for convenient testing. At present, the logs generated by the docker container are on the docker host and can be stored on the cos later. Then deliver it to elk cluster log visualization processing, make image unified management, make local image repository, add gitlab, add code review, and combine with automatic test.

The above is the whole content of this article, I hope it will be helpful to your study, and I also hope that you will support it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.