Shulou(Shulou.com)06/03 Report--

The main content of this article is to explain the method course of complete GitOps design and implementation of micro-service CI/CD practice. Interested friends may wish to take a look. The method introduced in this paper is simple, fast and practical. Now let the editor to take you to learn the "micro-service CI/CD practice of GitOps complete design and implementation of the method tutorial"!

Single application and environment

Multi-application and environment

CI persistent set

Cheng first prepares a code base: https://github.com/DevOpsCICDCourse/microservicescicd/blob/main/microservice-demo-service-master.zip

Let's take a look at the steps of the CI pipeline:

Because the type of code warehouse implemented this time is a single repository, that is, a storage inventory puts multiple service module codes, and each subdirectory is a service module.

First of all, our continuous integration pipeline needs to be able to correctly obtain the code of which service the current commit is.

Determine the service, then download the code for the service, compile and package, unit test, scan the code, and build the image.

How to get the service information of commit? Here we use the GitLab WebHook function to dock with Jenkins's job build trigger.

The workflow is as follows: when I submit the code in Gitlab, the Jenkins Scheduler job will be triggered through GitLab webhook, and the hook data data information generated by the submitted code will be transmitted to Jenkins Job in the form of POST. At this point, Jenkins job can write and use the Generic Hook plug-in to obtain the Body information of the request body transmitted by this POST request. Is a piece of JSON data, after the job runs, write Pipeline to parse the data in JSON to get the changed service module information. Finally, the CI job of the corresponding service is triggered to build.

CI-Scheduler job

For this job, you only need to enable webhook and configure to trigger token (uniqueness). Generate hookurl: http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CI

Jenkinsfile

Pipeline {agent any stages {stage ("GetData") {steps {script {echo "${webHookData}" data = readJSON text: "${webHookData}" println (data) env.branchName = data.ref-"refs/heads/" env.commitId = data.checkout_sha env.projectId = data.project_id commits = data ["commits"] println ("${env.branchName}) ") println (" ${env.commitID} ") println (" ${env.projectId} ") / / env.moduleName =" service01 "changeServices = [] for (commit in commits) {println (commit.id) / / added for (add in commit. Added) {s = add.split ("/") as List if (s.size () > 1) {if (changeServices.indexOf (s [0]) = =-1) {changeServices.add (s [0]) } / / modified for (m in commit.modified) {s = m.split ("/") as List / / println s / / println s.size () / / println s [0] if (s.size () > 1) {/ / println changeServices.indexOf (s [0]) if ( ChangeServices.indexOf (s [0]) =-1) {changeServices.add (s [0])}} / / removed for (r In commit.removed) {s = r.split ("/") as List println s if (s.size () > 1) {if (changeServices.indexOf (s [0]) = =-1) { ChangeServices.add (s [0])}} println (changeServices) / / currentBuild.description = "Trigger by ${eventType} ${changeServices}} }} stage ('DefineService') {steps {script {println (changeServices) / / Service build order control services = [' service02' 'service01'] for (service in services) {if (changeServices.indexOf (service)! =-1) {jobName =' microservicecicd-'+service+'-service-CI' build job: jobName, wait: false, parameters: [string (name: 'branchName', value: "${env.branchName}") String (name: 'commitId', value: "${env.commitId}"), string (name:' projectId' Value: "${env.projectId}")]}

GitLab configuration WebHook

Enable webhook and configure hookurl: http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CI

CI pipelining-CI job

Each microservice creates a CI job with three string parameters: branch name, commitID, and project ID.

Jenkinsfile

String branchName = "${env.branchName}" String moduleName = "${JOB_NAME}" .split ("/") [1] .split ("-") [1] String srcUrl = "http://gitlab.idevops.site/microservicecicd/microservicecicd-demo-service.git" String commitId =" ${env.commitId} "String projectId =" ${env.projectId} "pipeline {agent {node {label" build "} stages {stage ('GetCode') {steps {script {checkout ([$class: 'GitSCM' Branches: [[name: "${branchName}"]], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: "${moduleName}"], [path:' Dockerfile'] SubmoduleCfg: [], userRemoteConfigs: [[credentialsId: 'gitlab-admin-user'] Url: "${srcUrl}"])}} stage ("Build&Test") {steps {script {echo "Build." Sh "" cd ${moduleName} mvn clean package "} post {always {junit" ${moduleName} / target/surefire-reports/*.xml "} }} stage ("SonarScan") {steps {script {def sonarDate = sh returnStdout: true Script: 'date +% Y% m% d% H% M% S' sonarDate = sonarDate-"\ n" withCredentials ([string (credentialsId:' sonar-admin-user', variable: 'sonartoken'), string (credentialsId:' gitlab-user-token' Variable: 'gitlabtoken')] {/ / some block sh "" cd ${moduleName} sonar-scanner\-Dsonar.projectKey=$ {JOB_NAME}\-Dsonar.projectName=$ { JOB_NAME}\-Dsonar.projectVersion=$ {sonarDate}\-Dsonar.ws.timeout=30\-Dsonar.projectDescription= "xxxxxxx"\-Dsonar.links.homepage= http://www.baidu.com\-Dsonar.sources=src\ -Dsonar.sourceEncoding=UTF-8\-Dsonar.java.binaries=target/classes\-Dsonar.java.test.binaries=target/test-classes\-Dsonar.java.surefire.report=target/surefire-reports\-Dsonar.host.url= "http://sonar.idevops.site" \-Dsonar.login=$ {sonartoken}\-Dsonar.gitlab.commit_sha=$ {commitId}\-Dsonar.gitlab.ref_name=$ {branchName}\-Dsonar.gitlab.project_id=$ {projectId}\-Dsonar.dynamicAnalysis=reuseReports\ -Dsonar.gitlab.failure_notification_mode=commit-status\-Dsonar.gitlab.url= http://gitlab.idevops.site\-Dsonar.gitlab.user_token=$ {gitlabtoken}\-Dsonar.gitlab.api_version=v4 "" } stage ("BuildImage") {steps {script {withCredentials ([usernamePassword (credentialsId: 'aliyun-registry-admin') PasswordVariable: 'password', usernameVariable:' username')]) {env.nowDate = sh returnStdout: true Script: 'date +% Y% m% d% H% M% S' env.nowDate = env.nowDate-"\ n" env.releaseVersion = "${env.branchName}" env.imageTag = "${releaseVersion}-${nowDate}-${commitId}" env.dockerImage = "registry.cn-beijing.aliyuncs.com/microservicecicd/microservicecicd-$ {moduleName}-service:$ {env.imageTag}" env.jarName = "${moduleName}-${branchName}-${commitId}" sh "" docker login-u ${username}-p ${password} registry.cn-beijing.aliyuncs.com Cd ${moduleName} & & docker build- t ${dockerImage}-f.. / Dockerfile-- build-arg SERVICE_NAME=$ {jarName}. Sleep 1 docker push ${dockerImage} sleep 1 docker rmi ${dockerImage} ""}}

GitOps-CI extension

Based on the steps of the original CI job, an additional step is added to update the environment. The GitOps practice stores the current underlying environment deployment files in a Git repository. Our CI job updates the image tag information in the environment deployment file after the image is uploaded. (so we need to get the environment file and update the upload first)

Stage ("PushFile") {/ / when {/ / expression {"${env.branchName}" .requests ("RELEASE-")} / /} steps {script {if ("${env.branchName}" .requests ("RELEASE-")) {println ("branchName = branchName") Env.branchName = "master"} else {env.branchName = "feature"} for (I = 0 I

< 3; i++) { //下载版本库文件 response = GetRepoFile(40,"${moduleName}%2fvalues.yaml", "${env.branchName}") //println(response) //替换文件中内容 yamlData = readYaml text: """${response}""" println(yamlData.image.version) println(yamlData.image.commit) yamlData.image.version = "${releaseVersion}-${env.nowDate}" yamlData.image.commit = "${commitId}" println(yamlData.toString()) sh "rm -fr test.yaml" writeYaml charset: 'UTF-8', data: yamlData, file: 'test.yaml' newYaml = sh returnStdout: true, script: 'cat test.yaml' println(newYaml) //更新gitlab文件内容 base64Content = newYaml.bytes.encodeBase64().toString() // 会有并行问题，同时更新报错 try { UpdateRepoFile(40,"${moduleName}%2fvalues.yaml",base64Content, "${env.branchName}") break; } catch(e){ sh "sleep 2" continue; } } } } } //封装HTTP请求 def HttpReq(reqType,reqUrl,reqBody){ def gitServer = "http://gitlab.idevops.site/api/v4" withCredentials([string(credentialsId: 'gitlab-token', variable: 'gitlabToken')]) { result = httpRequest customHeaders: [[maskValue: true, name: 'PRIVATE-TOKEN', value: "${gitlabToken}"]], httpMode: reqType, contentType: "APPLICATION_JSON", consoleLogResponseBody: true, ignoreSslErrors: true, requestBody: reqBody, url: "${gitServer}/${reqUrl}" //quiet: true } return result } //获取文件内容 def GetRepoFile(projectId,filePath,branchName){ apiUrl = "projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}" response = HttpReq('GET',apiUrl,'') return response.content } //更新文件内容 def UpdateRepoFile(projectId,filePath,fileContent, branchName){ apiUrl = "projects/${projectId}/repository/files/${filePath}" reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}""" response = HttpReq('PUT',apiUrl,reqBody) println(response) } images GitOps-CD部分 CD-Scheduler作业 此作业其实也是接收GitLab的webhook请求， 与CI-scheduler作业类似。不同的是这个CD-scheduler作业是用来接收环境仓库的代码变更。开启webhook， 配置触发token。生成hookurl：http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CD Jenkinsfile pipeline { agent any stages { stage('GetCommitService') { steps { script{ echo 'Hello World' echo "${WebHookData}" // Git Info webhookdata = readJSON text: """${WebHookData}""" eventType = webhookdata["object_kind"] commits = webhookdata["commits"] branchName = webhookdata["ref"] - "refs/heads/" projectID = webhookdata["project_id"] commitID = webhookdata["checkout_sha"] changeServices = [] for(commit in commits) { println(commit.id) //added for (add in commit.added) { s = add.split("/") as List if (s.size() >

1) {if (changeServices.indexOf (s [0])) =-1) {changeServices.add (s [0])} / / modified for (m in commit.modified) {s = m.split ("/") as List / / println s / / println s.size () / / println s [0] If (s.size () > 1) {/ / println changeServices.indexOf (s [0]) if (changeServices.indexOf (s [0]) = =-1) {changeServices.add (s [0]) } / / removed for (rin commit.removed) {s = r.split ("/") as List println s If (s.size () > 1) {if (changeServices.indexOf (s [0]) = =-1) {changeServices.add (s [0])}} }} println (changeServices) currentBuild.description = "Trigger by ${eventType} ${changeServices}"} stage ('DefineService') {steps {script { Println (changeServices) / / Service build order control services = ['service02' 'service01'] for (service in services) {if (changeServices.indexOf (service)! =-1) {jobName =' microservicecicd-'+service+'-service-CD' build job: jobName, wait: false, parameters: [string (name: 'branchName') Value: "${branchName}")]}

Environment library configuration webhook

Enable webhook and configure hookurl: http://jenkins.idevops.site/generic-webhook-trigger/invoke?token=microservicecicd-scheduler-CD

CD pipelining-CD job

Jenkinsfile

String serviceName = "${JOB_NAME}" .split ("-") [1] String nameSpace = "${JOB_NAME}" .split ("-") [0] .split ("/") [- 1] / / pipeline pipeline {agent {node {label "k8s"}} stages {stage ("GetCode") {steps {script {println ( "${branchName}") println ("${env.branchName}" .codes ("RELEASE-") println "get Code" checkout ([$class: 'GitSCM' Branches: [[name: "${env.branchName}"]], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: "${serviceName}"] SubmoduleCfg: [], userRemoteConfigs: [[credentialsId: 'gitlab-admin-user'] Url: "http://gitlab.idevops.site/microservicecicd/microservicecicd-env.git"]]])} stage (" HelmDeploy ") {steps {script {sh"kubectl create ns" ${nameSpace}-uat "| echo false | Helm install "${serviceName}"-- namespace "${nameSpace}-uat". / "${serviceName}" | | helm upgrade "${serviceName}"-- namespace "${nameSpace}-uat". / "${serviceName}" helm list-namespace "${nameSpace}-uat" helm history "${serviceName}"-- namespace "${ NameSpace}-uat "}

At this point, I believe you have a deeper understanding of the "complete GitOps Design and implementation method course of Micro Services CI/CD practice". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.