Shulou(Shulou.com)06/03 Report--

This article mainly introduces how to use Jenkins + Git Submodule to achieve automatic compilation, has a certain reference value, interested friends can refer to, I hope you can learn a lot after reading this article, the following let the editor take you to understand it.

1. A true short story

It happened in the era of functional phones, our project team developed a mobile phone, and there were about 20 software developers. As a result, a week after the release of the phone, another small factory launched a phone with almost exactly the same software interface, except for the boot interface.

Because at that time, almost everyone used the solutions provided by MTK and Qualcomm, and they all had the same menu function, so you couldn't come up with strong evidence that someone else had stolen your code.

Later, it was found out internally that there was indeed a developer who leaked the code, so all the USB ports on all computers were banned.

This is the true story of my personal experience, when everyone was responsible for a module, such as: an is responsible for call management and phone book, B is responsible for system setup, C is responsible for text messages and MMS. When compiling, it is necessary to put all the code together and compile uniformly, which means that all software personnel can get all the source code, which lays a hidden danger for code leakage, and this serious incident emerges. After all, people die for money and birds die for food!

So, is there any code control to solve this permission problem?

Now in the project, it is emphasized to be layered and divided into modules, which is considered from the perspective of software engineering. If you go a step further, divide these modules into a small subsystem, and each developer is only responsible for his own module and can only have permission to pull his own code, so he can't get the code of all the modules in a project.

Only the project integrator (administrator) has full permission to pull all the source code to build the entire system, so that you can have better control over the security issues of the code.

To achieve such code control, you can use submodule in the git tool. In this article, we will explain the use of git submodule in detail.

This article is instrumental and may be long. In order to provide one-stop service, I will record the relevant resources, steps, error messages and other details for future reference.

In any case, after this article, you can learn and understand the following knowledge points:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Basic usage of Jenkins

Basic instruction usage of git submodule

Through three demo projects, step by step operation to achieve code security control

Using Jenkins + git submodule to realize automatic compilation

The difference between git subtree and submodule

If you need the software and code resources mentioned in this article, you can find a download method at the end of the article.

Second, the basic use of Jenkins 1. What is Jenkins?

Jenkins is an open source continuous integration tool written by Java, which means it helps us automate the construction of various projects. Jenkins runs in a Servlet container (such as Apache Tomcat) and can be installed with one click using apt-get on Ubuntu systems.

Jenkins has the following characteristics:

Embedded in the Web server, through the browser to operate, very convenient

You can execute Apache Ant and Apache Maven-based projects, as well as arbitrary Shell scripts and Windows batch commands

The build can be triggered by a variety of means. For example, it is triggered when submitted to the version control system, scheduled through a mechanism similar to Cron, and can be requested through a specific URL when other builds have been completed.

Jenkins's powerful plug-in allows Jenkins to integrate a lot of software, which may help us to continuously integrate our engineering projects.

Give users a lot of permissions and flexibility to automatically publish, deploy, and so on.

I won't blow the rest. I think it's very useful. If you have a chance, you can also try it. In addition, the virtual machine I tested is the newly installed Ubuntu16.04-64, which follows the process below to ensure that it can run smoothly.

JDK and Jenkins installation methods, a lot of information on the network, some of the process is problematic, or some key steps are not clearly written. In order to make it easier for you to operate successfully at one time, I will record it here.

If you are already familiar with the installation process, you can skate directly to the next topic.

two。 Install JDK8

(1) download and decompress

Download jdk-8u221-linux-x64.tar.gz, (download address is provided at the end of the article), extract to directory / home/sewain/OpenSource, and extract instructions:

Sudo tar-zxvf jdk-8u221-linux-x64.tar.gz-C / opt

(2) set environment variables

Execute the instruction: vim ~ / .bashrc, adding the following at the end:

Export JAVA_HOME=/opt/jdk1.8.0_221 export PATH=$JAVA_HOME/bin:$PATH export CLASSPATH=./$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jart

I suggest you also use such environment variables. If you upgrade the JDK version in the future, you only need to modify the JAVA_HOME.

(3) reload environment variables

Execute the instruction: source ~ / .bashrc, and the environment variable takes effect.

Verify this: java-version. The following message indicates OK:

Java version "1.8.0mm 221" Java (TM) SE Runtime Environment (build 1.8.0_221-b11) Java HotSpot (TM) 64-Bit Server VM (build 25.221-b11, mixed mode) 3. Install Jenkins

(1) Import the GPG key of Jenkins repository

Sudo wget-Q-O-https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add-

(2) add Jenkins repository to the system

Sudo sh-c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > / etc/apt/sources.list.d/jenkins.list'

(3) install using apt

Sudo apt update sudo apt install jenkins

The Jenkins service will start automatically after the installation process is complete, which can be verified by the instruction systemctl status jenkins.

Systemctl status jenkins

(4) configure the port

Jenkins is embedded in a tomcat server and uses port 8080 by default, so it is easy to conflict with other services, so it needs to be modified. Two files are involved:

File one: / etc/init.d/jenkins

In the PATH variable on the first line, add your own JDK address:

PATH=/bin:/usr/bin:/sbin:/usr/sbin:/opt/jdk1.8.0_221/bin

File 2: / etc/default/jenkins

Change the value of HTTP_PORT to the new port number, for example: HTTP_PORT=9090.

(5) start and stop instructions for Jenkins

Sudo service jenkins start sudo service jenkins stop

If you unfortunately encounter an error, you can use these two instructions repeatedly to troubleshoot the error.

4. Configure Jenkins in the browser

Type: htpp://localhost:9090 in the browser, wait a moment, and the interface appears:

Follow the interface prompts to copy the security password from the / var/lib/jenkins/secrets/initialAdminPassword file (root permission is required) and fill it in the browser window.

The install plug-in window appears. Generally, you only need to install the recommended plug-in:

After the plug-in is installed, go to the administrator configuration interface:

At this point, the installation of Jenkins is successfully completed!

5. Configure a small project in Jenkins

(1) prepare a test code Test1

Jenkins is only a component framework, and the specific compilation process is determined by the user. Jenkins first acquires the code in the remote repository through the git tool, and then executes the compilation instructions specified by the user.

Therefore, we need to prepare a test code in advance and put it in a remote repository that Jenkins can access. Of course, it is also possible for you to deploy a git repository in your local Ubuntu system. For convenience, the code I tested, Test1, is in gitee.

(2) create a new project

(3) enter the project name and select the first free style (Free project)

(4) under the first tag General, enter the description information of the project (Description)

The content can be written at will.

(5) Source Code Management source code management

During the build (compilation) process of Jenkins, you need to get the source code, so you need to configure the address and account information (username and password) of the git repository.

First, under the Add drop-down button, select Jenkins:

Enter Username and Password:

After the account is added, in the drop-down box of Credentials, select the gitee account you just added, and you can see that the red permission error message disappears, indicating that you can pull the source code from the remote repository.

(6) Build Triggers selective compilation trigger

You can choose different ways to trigger as needed, such as timing triggering, triggering when another project is built successfully, and so on.

We do not select any items here, we will manually click the button to trigger it in the main interface.

(7) Build Environment compilation environment

I use this part less, using other tools to assist the function of Jenkins.

(8) Build compilation

It tells Jenkins how to build the system, that is to say, Jenkins is just an automated build system, and the specific compilation process can be decided by the user. You can choose the following projects:

We choose to execute the script directly (Execute Shell) here, and enter the following instructions:

(9) Post-compilation operation of Post-build Actions

Tell Jenkins: what do you need to do after compiling a project? For example: send an email, trigger the automatic compilation of the next project, etc., and you can add multiple actions. The options are as follows:

After the above steps are configured, Save is saved, and the full picture of the project can be seen in the main screen, as shown below:

In the Jenkins background, the save path for this project is: / var/lib/jenkins/jobs/Test1.

6. Manually trigger compilation once

Since we did not select any trigger conditions in step (6) above, we need to manually click the Build Now button on the left in the main interface of the project Test1 to trigger.

At this point, in the Build History on the left, you can see the compilation history, and click on the compilation record number to see the details of this compilation.

In the compilation details, click the Console Output button on the left to see the compiled output: the executable was successfully compiled.

As we can see in the Jenkins background, the source code is pulled to the / var/lib/jenkins/jobs/Test1/workspace directory:

At this point, you have learned the most basic operation of Jenkins!

Let's continue to talk about the use of git submodule, this part is the core content!

3. Git submodule basically uses 1. What is git submodule?

Git submodule is a tool for multi-module management that allows one project as a repository and other projects as child modules in the parent project.

The submission of the parent project and the child project is separate, that is, the information submitted by the parent project contains only the information of the child project, not the code of the child project; the child project has its own independent commit,push,pull operation.

Git submodule is generally used in large projects, in order to facilitate reuse, or for the security of the code, often need to be divided into several sub-projects for code management.

Common instructions include:

Add sub-module: git submodule add

Update sub-module: git submodule update

Initialization submodule: git submodule init recursively clone the entire project: git clone--recursive

Pull all sub-modules: git submodule foreach git pull

two。 Use three small projects to test the use of submodule

For demonstration purposes, we create three projects and push them all to the remote repository, using gitee.

Test1: compile to get a dynamic library: libtest1.so

Test2: compile to get a dynamic attitude: libtest2.so

Test3: compiles an executable program that loads and calls functions from the above two dynamic libraries.

Why do you design the module like this: security!

Developer A: responsible for Test1, do not have permission to get the code of other modules

Developer B: responsible for Test2, do not have permission to get the code of other modules

Project manager: responsible for Test3 and code integration, can get all the code

The project manager needs to add Test1 and Test2 as sub module to Test3 and execute the following instructions:

1.git submodule add https://gitee.com/[ your account] / test1.git test1

2.git submodule add https://gitee.com/[ your account] / test2.git test2

After adding Test1 and Test2 as submodules to Test3, take a look at the changes in the file:

You can also take a look at the .gitmodules file, and you can see that this is the configuration file through which the git tool manages the submodules.

The administrator needs to integrate and compile all the modules, so we add a script build.sh under the Test3 directory, and all the compilation instructions are written in this script, as follows:

The content is the most basic, just call the make instruction directly, execute it, and output:

At this point, we have completed the add function of the sub-module.

3. Compile and verify the feasibility in an empty directory

When we clone the Test3 project in another empty directory, we can find that the cloned test1 and test2 folders are empty, as shown below:

You need to manually get all the sub-modules and execute the instructions:

Git submodule update-init-recursive

At this point, using the tree command to take a look at the file changes, you can see that both test1 and test2 files have been pulled down. There is a problem to note here: although the code of the sub-module has been pulled down, its head does not point to the master branch, which needs to be dealt with manually, as shown in the figure:

At this point, we execute the script build.sh again in the test3 directory, and all the submodules can be compiled smoothly.

In the above steps, we manually test the compilation process of submodule in a local temporary directory.

In the next section, we will deploy this process to the Jenkins system, so the instructions just executed need to be written in the build.sh script. The content of build.sh is as follows:

4. Use git module to compile all modules in Jenkins

The following operations are done in the Jenkins panel of the browser.

1. Reconfigure the project

Because we are in Test3 to compile the entire project (Test1 and Test2 are included as submodules), we first delete the previously added Test1 project, as shown in the figure:

Then add the project Test3 again and review the steps:

Hongmeng official Strategic Cooperation to build HarmonyOS Technology Community

Enter description information and select a free-style project

Enter the git warehouse address and account information and select the master branch

Trigger is not set

The compilation environment is not set

Compile: select Execute shell to execute the script and enter the compilation instruction:. / build.sh. (as I just said, Jenkins is an automated build framework, and the specific compilation process is determined by the user, so our compilation process here is to execute the build.sh script under Test3.)

Actions are not set after compilation

Of course, you can also make changes directly based on the previous Test1 project.

At this point, we directly click the Build Now button in Jenkins, and if nothing happens, we will prompt for a compilation error (a red error circle appears under the Build History on the left).

Click in and take a look at the output message (Console Output button) to indicate an error:

As mentioned earlier, after the submodule is obtained, head does not point to the master branch, which needs to be manually modified when we compile for the first time (I did not find any other way, if you know, please do not hesitate to comment!)

Manual solution: in the command line window, go to the Test3 directory / var/lib/jenkins/jobs/Test3/workspace of the Jenkins system and execute the following instructions:

Git submodule update-init-recursive cd test1/ git checkout master cd-cd test2/ git checkout master cd

At this point, if you trigger the compilation again, it will be successful!

Thank you for reading this article carefully. I hope the article "how to use Jenkins + Git Submodule to achieve automatic compilation" shared by the editor will be helpful to you. At the same time, I also hope you will support us and pay attention to the industry information channel. More related knowledge is waiting for you to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.