Shulou(Shulou.com)06/02 Report--

Based on how Jenkins builds a continuous integration pipeline that conforms to the DevOps capability maturity level 3 standard, I believe many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

The core of DevOps is automation, and the core of automation is standardization. The most important part of DevOps is continuous delivery, and the focus of continuous delivery construction is pipeline, so how to build a standard continuous delivery pipeline is the most important link in DevOps construction, and it is also an important scoring point to evaluate DevOps capabilities.

Based on jenkins, this paper makes a technical response to some key points of continuous integration assembly line construction, and leads everyone to land the methodology to specific technical points.

Several nouns involved in the article explain:

1, pipelining: pipeline, an application that automates the process of building, deploying, testing, and releasing

2, artifacts: the outputs of the construction process, including software packages, test reports, application configuration files, etc.

3. Product library: a warehouse for storing full-language products, providing dependent parsing and file storage capabilities.

4, metadata: the whole process data of software life cycle, such as requirements id, code submission information, construction environment, static scan results, test pass rate, security scan results, etc.

Some of the technologies involved in the article are explained in detail: see the article "18 questions about Building Enterprise pipeline Services"

Next, let's take a look at how the three-level standards of the seven dimensions of configuration management, build and continuous integration, test management, deployment and release management, environmental management, data management, measurement and feedback in the construction of the continuous integration pipeline are defined, and which indicators need to be reflected in the jenkins pipeline, and how to use the jenkins pipeline to achieve this standard.

First, configuration management

Third-level standard

Proposal for landing of Jenkins assembly line

Version control

Version control system

1) incorporate automated scripts such as configuration files, build and deployment into version control system management. 2) there is a sound version control system management mechanism, including: code base naming standard backup and availability guarantee mechanism permission model dedicated post management.

Pipeline content (Jenkinsfile) needs to be included in version management pipeline naming needs to have clear specification, pipeline should have clear permissions, developers should only have readable permissions, templates should be prepared by a special team to write technical points: the Share library feature of jenkins can be used, and the pipeline can be managed by a special team in the source code repository.

Branch management

Short-period branches frequently merge to the trunk

Non-pipeline content

Product management

1) bring dependent components into product library management 2) bring all deliverables into product library management, such as test reports 3) there is a clear authority control system for reading and writing in product library

Build a unified product library, such as Artifactory. Set full permissions. Collect the result data of all the tools in the process of building the pipeline, define such data as metadata, and bind it to the artifact. Such as requirements, code submission information, construction environment information, dependency information, static scanning information, unit test information, security scanning information and so on. Technical points: commercial product libraries can be used, such as Artifactory. You can also develop your own metadata management system to collect process data during construction.

Single trusted data source

As a single trusted data source, version control system and product library cover the process of production and deployment.

Establish a unified product library, specify the product library address in jenkinsfile, build without using the dependent parsing address in the pom file, but modify the dependent parsing warehouse to the only trusted warehouse in other ways: use Artifactory to manage the product library uniformly to ensure the only trusted source

Change management

Change process

1) all configuration item changes are triggered by the change management system. 2) Review each change and use automated means.

Do not involve pipelining, pay attention to the separation of configuration and application, and configuration center management

Change tracing

Automatic association between version control system and change management system, two-way synchronization and real-time traceability of information

Does not involve assembly line

Change rollback

1) realize the synchronous rollback of the change management system and the version control system to ensure the consistency of the state. 2) automate the rollback operation.

Does not involve assembly line

Second, construction and continuous integration

Third-level standard

Proposal for landing of Jenkins assembly line

Construction practice

Construction mode

1) define structured build scripts to achieve module-level sharing and reuse. 2) build scripts are uniformly maintained by dedicated personnel.

Technical points: use Jenkins ShareLibrary to build modular management and achieve global sharing

Build the environment

1) standardize the configuration of the construction environment. 2) there is an independent construction resource pool.

Create a small number of fixed standardized construction nodes as independent construction resource pools, and use K8s cluster to create dynamic construction nodes as dynamic resource pools. Technical points: jenkins master-slave architecture, jenkins on k8s

Build a plan

1) implement regular automated build and code submission to trigger build 2) clearly define build plans and rules and share them within the R & D team

Technical point: jenkins trigger, which can realize timing build, polling source code build or webhook trigger build

Construction responsibility

The build tools and environment are maintained by a dedicated team and subdivided into team responsibilities

The jenkins master-slave node or build image is maintained by the unified team. The business unit can only use it and cannot modify it.

Continuous integration

Integrated service

Set up a special continuous integration team to optimize continuous integration systems and services

The unified team builds pipelined templates and continuous integration environments for developers to choose technical points: multiple build environment images can be created through jenkins on K8s, and developers can define the required environment when submitting build tasks.

Integrated frequency

Developers integrate with the code trunk at least once a day

Does not involve assembly line

Integration mode

Every time the code is submitted, the automatic build is triggered, and the build problem is automatically analyzed and accurately pushed to the relevant personnel to deal with.

Each time the code is submitted, jenkins is triggered to build, and complete static scanning, unit testing and other steps are performed during the build process. Technical points: jenkins trigger function, you can set webhook trigger for rotation scm or git.

Feedback period

Integrated problem feedback and resolution can be completed in a few hours

Notify the completion or failure status of the construction work in jenkins pipeline and send email or webhook to the operation and maintenance team and business team.

Third, test management

Third-level standard

Proposal for landing of Jenkins assembly line

Test layering strategy

Stratification method

1) using code-level testing to cover a comprehensive unit test of the function or class method of the module; 2) the system comprehensively carries out non-functional tests such as performance, capacity, stability, reliability, ease of use, compatibility, security and so on.

Unit testing is carried out in the pipeline, and the unit test pass rate is collected as metadata to bind to the product.

Layering strategy

1) Test design is mainly based on interface / service level testing, while user / business level testing is supplemented by a small amount of code level testing. 2) Comprehensive and systematic design of non-functional testing

Interface testing can be integrated in the pipeline, and the pass rate of interface tests can be collected as metadata to bind to artifacts.

Test timing

1) the intervention time of the test in the continuous delivery process is advanced to the coding phase of the development. 2) the code-level testing is carried out after the development of the function or class method of the module.

Improve unit test coverage.

Code quality management

Quality specification

1) establish an organization-level code quality specification 2) establish a complete quality specification, including security vulnerability check and compliance check into the specification 3) establish an enforced quality access control system 4) establish a fixed update mechanism for the specification

It is necessary to add security scanning steps to the jenkins pipeline and set quality levels for the scan test results. Technical points: Xray is integrated in the assembly line as a security scanning tool, and the product metadata is used as the quality access control to determine whether the constructed product is up to the standard.

Inspection mode

Code quality checking is fully automated and does not require manual intervention

The pipeline integrates the sonar scanning tool, which automatically triggers the construction and automatically scans the source code every time the code is submitted, and writes back the result data such as the number of bad smells and the code repetition rate to the product library in the form of metadata. Technical points: static scanning of sonarqube code

Feedback processing

According to the code quality inspection results feedback timely processing, according to the quality specification to maintain a certain technical debt

The static scan result of the code is bound to the product and written back to the product library. The quality of the product is judged by whether the metadata carried by the product passes the quality access control.

Automated testing

Automatic design

1) Automation design of interface / service level test 2) Automation design of code level test

Jenkins pipelining adds interface testing and service testing

Automatic development

1) establish a unified automated testing framework and uniformly manage automated test cases 2) data-driven and keyword-driven methods are adopted in the development of automated test scripts.

Does not involve assembly line

Automatic execution

1) Automated testing for interface / service-level and code-level testing 2) automated testing is triggered by pipeline automation

Perform the required tests in the pipeline

Automatic analysis

Strong automatic judgment ability for automated test results with less false positives

The test results are collected in the pipeline and associated with the deliverables as metadata to ensure that each product can obtain complete test results.

Fourth, deployment and release management

Third-level standard

Proposal for landing of Jenkins assembly line

Deployment and release model

Deployment mode

Fully automated deployment and release

Deployment process as a necessary step in the pipeline technical points: docking tools such as saltstack and ansible are deployed in the pipeline

Deployment process

1) use the same process and tools to deploy all environments. 2) use the same build product during one deployment.

To ensure that the release is tested, you need to build multiple deployments at a time. The product maturity is identified by metadata and warehouse name. In the pipeline, the products should be moved in different maturity warehouses, and the result data in each environment should be collected and stored as metadata. Technical points: application configuration separation, Artifactor metadata and promotion functions

Deployment strategy

1) adopt a regular deployment strategy and have the ability to deploy on a daily basis. 2) the application and the environment as a whole are deployed as the smallest unit of deployment. 3) Separation of application and configuration

Does not involve assembly line

Deployment quality

1) low deployment failure rate 2) deployment activities integrate automated testing functions, and use test results as deployment prerequisites 3) provide change scope reports and test reports for each deployment activity

After deployment, simple verification is performed in the pipeline and the verification result data is collected. The test results are collected into the metadata, and the metadata is verified during deployment to determine whether the deployment is achieved through quality access control. Technical point: Artifactor metadata

Continuous deployment pipeline

Cooperation mode

Ensure orderly delivery between teams by defining a complete software delivery process and clear delivery specifications

Standardize the tool chain and continuous integration pipeline, collect stage result data as metadata, and use metadata to identify product quality standards for use among teams.

Pipeline process

Establish automation capabilities in all aspects of the software delivery process to improve processing efficiency

Does not involve assembly line

Process visualization

1) the delivery process is visible within the team, and the information is shared between teams. 2) the delivery status is traceable.

The result data of the entire construction process is collected in the pipeline and bound to the artifact for all teams to view. Technical point: Artifactor metadata

Fifth, environmental management

Third-level standard

Proposal for landing of Jenkins assembly line

Environment management

Environmental type

Establish a standard R & D environment

Does not involve assembly line

Environment construction

1) the construction of the environment is completed through the self-service resource delivery platform. 2) the environment preparation time is hourly.

The required environment can be created automatically in the pipeline. Technical point: use helm of K8s to automatically pull up the whole environment, and helm is the best way to implement it.

Environment depends on configuration management

Application-centric, service-level dependent configuration management capabilities, such as dependent association services, database services, caching services, related application services, etc.

Does not involve assembly line

Sixth, data management

Third-level standard

Proposal for landing of Jenkins assembly line

Test data management

Data source

A benchmark test data set is formed after exporting part of the production environment data and cleaning sensitive information.

Does not involve assembly line

Data overlay

Establish systematic test data, manage data dependencies, and cover all test types required by the test layering strategy.

Does not involve assembly line

Data independence

1) each test case has its own test data and a clear test initial state. 2) the execution of test cases does not depend on the data generated by other test case execution.

Does not involve assembly line

Data change management

Change process

Bring data changes into the continuous deployment pipeline and complete them automatically after manual confirmation.

The pipeline is integrated with the examination and approval system.

Compatible rollback

Provide a clear rollback mechanism for each data change, and implement change testing, such as providing two automated scripts for upgrade and rollback

Does not involve assembly line

Data monitoring

Establish a hierarchical monitoring mechanism for data changes according to different environments and risk levels.

Does not involve assembly line

Seventh, measurement and feedback

Third-level standard

Proposal for landing of Jenkins assembly line

Measurement index

Metric definition

Establish cross-organizational measurement indicators to measure cross-domain comprehensive dimensions

Does not involve assembly line

Metrics type

Measurement indicators cover process indicators and objectively reflect the current situation of organizational research and development.

Metadata needs to be collected in the pipeline as a subsequent metric.

Measurement data management

There are clear management rules for continuous collection of measurement data, historical measurement data.

Metadata needs to be collected in the pipeline as a subsequent metric.

Metrics update

1) the metrics can be updated regularly according to the requirements. 2) the priorities of the metrics are agreed within the team.

Does not involve assembly line

Measurement-driven improvement

Content and generation method

Metrics reports are classified and graded and content is generated as needed

Metadata needs to be collected in the pipeline as a subsequent metric. Clear the metadata twice and generate a report

Data timeliness

Real-time display of data through visual Kanban

Kanban needs to show the status of the pipeline, such as construction time, pass rate, failure rate, etc.

Coverage area

All team members can view the report

Does not involve assembly line

Feedback improvement

Measurement feedback issues are included in the to-do list of R & D iterations as part of continuous improvement

Does not involve assembly line

Through the above data and analysis, it can be seen that creating a standard assembly line service can match 60% of the third-level standards. Then we can invest a great deal of strength in the construction of the whole DevOps to build an assembly line. A set of standard assembly line services and a stable tool chain will be a cornerstone of DevOps construction and will run through your entire construction cycle.

After reading the above, have you mastered the method of how to build a continuous integration pipeline that meets the level 3 standard of DevOps capability maturity based on Jenkins? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.