Shulou(Shulou.com)06/01 Report--

This article is about how to deploy and scale Jenkins on Kubernetes. I think it is very practical, so I share it with you. I hope you can get something after reading this article.

We will use Rancher to deploy and scale Jenkins on Kubernetes. Step by step, you will use the master-agent architecture we used to test the actual build job to create a fully functional Jenkins.

Jieshao

Jenkins is an open source continuous integration and continuous delivery tool that can be used to build, test, and deploy software automatically. More than 1 million users around the world use Jenkins, which is currently the most popular automated server. The benefits of Jenkins include:

It is open source software with huge community support.

Java-based code base, making it portable to all major platforms

A rich ecosystem with more than 1000 plug-ins

Jenkins can work well with mainstream source code management systems (Git, SVN, Mercurial and CVS), mainstream construction tools (Ant, Maven, Grunt), shell scripts and Windows batch commands, test framework, and report generator. Jenkins plug-ins also provide support for Docker and Kubernetes, which can create and deploy cloud-based microservice environments, and put them into test and production deployments.

Jenkins supports the master-agent architecture (many build agents/ build agents complete tasks based on master server scheduling), making it highly scalable. Master's job is to schedule build jobs, distribute them to agents for actual execution, monitor these agents, and get the results of the build. In addition, the master server can also execute build jobs directly.

The task of the agent is to build the job sent from the master server. The job can be configured to run on a specified type of agent, and if there is no special need, Jenkins simply selects the next available agent.

The scalability of Jenkins brings many conveniences:

Run multiple build scenarios in parallel

Automatically mount and remove agents to save overhead

Distribute the load

Of course, although Jenkins includes this out-of-the-box scalability feature, the process of configuring it is not easy. There are many options for extending Jenkins, and one of the powerful options is to use Kubernetes.

What is Kubernetes?

Kubernetes is an open source container orchestration tool. It is mainly used to help operators deploy, scale, update and maintain services, and provide a service discovery mechanism to manage containerized applications on node clusters. You can check the official documentation to learn more about the content and use of Kubernetes:

Https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/

Kubernetes is one of the best tools for managing scalable, container-based workloads. Most applications, including Jenkins, can be containerized, which makes Kubernetes a good choice.

Project goal

Before we begin, take a moment to describe the system we are going to build.

We will first deploy the Jenkins master instance to the Kubernetes cluster. We will use Jenkins's kubernetes plug-in. We will use Jenkins's kubernetes plug-in to extend Jenkins on the cluster by providing dynamic proxies to adapt to the current workload. The plug-in starts the agent based on a specific Docker image, creating a Kubernetes pod for each build. After the build is complete, Jenkins will delete the pod to save resources. The agent is started using JNLP (Java Network Launch Protocol,Java Network Startup Protocol), so the container can automatically connect to the Jenkins master after it is up and running.

Pre-preparation and installation

You need to prepare these things to complete this tutorial:

The Linux machine is used to run Rancher: we will also use it to build custom Jenkins images. You can install Docker and Rancher on the host according to the Rancher installation getting started guide:

Https://rancher.com/quick-start/

Docker Hub account: we need an account with a container image repository to push custom images for Jenkins master and agents.

GCP account: we will deploy the Kubernetes cluster on GCP. Free-tier on Google's cloud platform should be able to do this. You actually use other public clouds, and the operation is the same.

Build a custom image for the Jenkins component

So let's start by building custom images of Jenkins components and pushing them to Docker Hub.

Log in to the Linux server, where you can run Rancher and build the image. If you have not already installed Docker and Rancher, follow the Rancher Quick start Guide to install Docker and Rancher on the host. Once the mainframe is ready, we can prepare the dockerfile.

Write Jenkins Master Dockerfile

Let's first create a file called Dockerfile-jenkins-master under the current folder to define the Jenkins master image:

Inside the file, add the following Dockerfile build command. These commands use the primary Jenkins Docker image as the basis to configure the plug-ins we use to deploy to the Kubernetes cluster:

When you are finished, save and close the file

Write Dockerfiles for Jenkins agent

Next, we can create a Dockerfile file for the Jenkins agent. We will create two proxy images to demonstrate how Jenkins correctly identifies the correct agent prepared for each job.

Create an empty file in the current directory. We will copy it to the mirror as an identifier for each agent being built:

Now, create a Dockerfile for the first proxy image

The image will copy the empty file to a unique name, marking the agent used.

Save and close the file when you are finished

Finally, define the second agent. This is the same as the previous agent, but uses a different file identifier:

Save and close the file

Now your working directory should look like this:

Build the image and Push it to Docker Hub

With the Dockerfile ready, we are now ready to build and push mirroring to Docker Hub.

First, build an image of Jenkins master:

Note: in the following command, it will be replaced with your own Docker Hub account name

[root@rancher-instance jenkins-kubernetes] # docker build-f Dockerfile-jenkins-master-t / jenkins-master.

When the results of the command are returned, view the newly created image:

Log in to Docker Hub using your account credentials:

[root@rancher-instance jenkins-kubernetes] # docker loginLogin with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.Username:Password: Login Succeeded

Now, push the image to Docker Hub:

Note: in the following command, also pay attention to replace it with your own Docker Hub account

You may also need the same command to build a second image to the Jenkins JNLP agent:

Note: in the following command, note that it will be replaced with your own Docker Hub account name

If all goes well, you can see the following status in your Docker Hub account:

Deploy a cluster using Rancher

Now that our image has been released, we can use Rancher to help deploy the GKE cluster. If you previously installed Rancher, you can log in to the instance by accessing the server's ip address through a web browser.

Next, create a new GKE cluster. In order to create a service account with access, you need to log in to the Google Cloud account. The steps required to use other public cloud services are similar. For more information, please see the documentation to learn how to create a service account and how to deploy a cluster with Rancher:

Https://rancher.com/docs/rancher/v2.x/en/cluster-provisioning/hosted-kubernetes-clusters/

Deploy Jenkins on a cluster

Once the cluster is ready, we can deploy Jenkins master and create some services. If you are familiar with kubectl, you can do it directly from the command line, but with Rancher's UI, you can easily deploy all the required components.

No matter how you choose to add the workload to the cluster, you need to create the following file on the local computer to define the objects that need to be created.

First create a file to define the Jenkins deployment:

Paste the following in the file:

Note: the following content will be replaced with your own Docker Hub account name

`apiVersion: extensions/v1beta1

Kind: Deployment

Metadata:

Name: jenkins

Spec:

Replicas: 1

Template:

Metadata:

Labels:

App: jenkins

Spec:

Containers:

Name: jenkins

Image: / jenkins-master

Env:

Name: JAVA_OPTS

Value:-Djenkins.install.runSetupWizard=false

Ports:

Name: http-port

ContainerPort: 8080

Name: jnlp-port

ContainerPort: 50000

VolumeMounts:

Name: jenkins-home

MountPath: / var/jenkins_home

Volumes:

Name: jenkins-home

EmptyDir: {} `

Next, create a file to configure the two services we need.

One is the LoadBalancer service, which will provide a public IP address for us to access Jenkins on Internet. The other is the ClusterIP service, which is used for internal communication between the master and the agent, which is later used:

Within the file, copy the following YAML structure:

`apiVersion: v1

Kind: Service

Metadata:

Name: jenkins

Spec:

Type: LoadBalancer

Ports:

Port: 80

TargetPort: 8080

Selector:

App: jenkins

ApiVersion: v1

Kind: Service

Metadata:

Name: jenkins-jnlp

Spec:

Type: ClusterIP

Ports:

Port: 50000

TargetPort: 50000

Selector:

App: jenkins`

On Rancher, click on the cluster you manage (named jenkins in the example). In the menu in the upper left corner, select the Default project, and then select the Workloads tab.

Now, click Import YAML. On the next page, click the Read from a file button in the upper right corner. Select the deployment.yml file that was created locally and click Import.

Rancher will deploy a pod based on your Jenkins master image on the cluster

Next, we need to configure how to access UI on Jenkins master.

In the Load Balanced tab, follow the same action as the previous import file. Click the Import YAML button, then click the Read from a file button. Then select the service.yml file from your computer and click the Import button.

Rancher will start creating your service. It takes some time to deploy load balancers.

After the service status becomes Active, you can click on the three vertical points behind the load balancer line and select View/Edit YAML to find its public IP address. Here, scroll down to find the IP address under status- > loadBalancer- > ingress- > ip.

This allows us to access Jenkins UI in the web browser through this IP address.

Configure a dynamic build agent

With Jenkins master up and running, we can further configure the dynamic build agent to start Pods automatically if necessary.

Disable the default Master build agent

Under Build Executor Status on the left side of Jenkins UI, two Executor are configured by default, waiting for the build job to be executed, which are provided by Jenkins master.

The primary instance should only be responsible for scheduling build jobs, distributing jobs to agents for execution, monitoring agents, and obtaining build results. Because we do not want the primary instance to perform a build, we want to disable these.

Click Manage Jenkins after Manage Nodes.

Click the gear icon on the line with master.

On the next page, set # of executors to 0 and click Save.

These two idle executors will be removed from the Build Executor Status on the left side of the UI.

Collect configuration information

In order to automatically deploy the build agent on the Kubernetes cluster, we need some information to configure Jenkins. We need three pieces of information from the GCP account and one from the ClusterIP service.

In your GCP account, select Kubernetes Engine, then Clusters, and then click the name of the cluster. In the Detail column, copy the endpoint IP for later use. This is the URL that we need to connect Jenkins to the cluster:

Next, click Show credentials to the right of the endpoint. Copy Username and Password.

Now, switch to Rancher UI. In the upper-left menu, select the Default project on the Jenkins cluster. Select the Workloads tab in the navigation window above, and then click the Service Discovery tab on the page:

Click the three vertical dots on the jenkins-jnlp line, and then click View/Edit YAML. Copy the values in spec > clusterIP and spec > ports > port.

Configure the Jenkins Kubernetes plug-in

Return to the main Jenkins dashboard, click Manage Jenkins, and then select Manage Plugins:

Click the Installed tab and see if the Kubernetes plug-in is installed:

Now let's configure the plug-in. Go to Manage Jenkins and select Configure System:

Slide to the Cloud section at the bottom of the page. Click Add a new cloud and select Kubernetes.

In the form below, enter https://, on the Kubernetes URL field and then enter the cluster endpoint IP address copied from the GCP account.

Under Credentials, click the Add button and select Jenkins. On the form that appears, enter the user name and password copied from the GCP account, and click the Add button at the bottom.

Return to the Kubernetes form, select the credentials you just added from the Credentials drop-down menu and click the Test Connection button. If the configuration is correct, "Connection test successful" is displayed.

Now scroll down to the Images section at the bottom of the page, click the Add Pod Template button, and then select Kubernetes Pod Template. Fill in the "Name" and "Labels" fields with unique values to identify your first agent. We will use tags to specify which proxy image should be used to run each build.

Next, in the Jenkins tunnel field, enter the IP address and port that you retrieved from the jenkins-jnlp service in Rancher UI, separated by a colon:

Now, in the Container field, click the Add Container button and select Container Template, and fill in the following fields in the pop-up:

Name: jnlp (required by the Jenkins agent)

Docker image:/Jenkins-slave-jnlp1 (make sure to change the Docker Hub user name)

Command to run: delete the value here

Arguments to pass to the command: delete the value here

The remaining fields remain as they are.

Next, click the Add Pod Template button and select Kubernetes Pod Template again. Repeat the process for the second proxy image created, and note that you need to modify the values corresponding to the second image as needed:

Click the Save button to save the changes and continue.

Test dynamic build job

Now that we have completed the configuration, we can create some build jobs to ensure that Jenkins can scale on top of Kubernetes. Here we will create five build jobs for each Jenkins agent.

On the Jenkins main page, click New Item on the left, enter a name for the first build of the first agent, select Freestyle project, and click the OK button.

In the Label Expression field on the next page, enter the label you set for the first Jenkins proxy image, and if you click outside the field, a message appears indicating that the tag is provided by the cloud.

Scroll down to the Build Environment section and check the Color ANSI Console Output.

In the Build section, click Add build step and select Execute shell. Paste the following script in.

Click Save when you are finished.

To create four more jobs for the first agent, click New Item, fill in the new name, and use the Copy from field to copy from the first build. You can save each build without making changes to the first build.

Next, configure the first job for the second Jenkins agent. Click New Item, choose a name for the first job of the second agent, and copy the job from the first agent again. This time we will modify the fields on the configuration page before saving.

First, modify the Label Expression field to match the label of the second agent.

Next, replace the script in the Build part of the text box with the following script:

Click Save when you are finished.

Also following our process just now, create four more builds for the second agent.

Now, go back to the main page and click the icon on the far right of each row to start all the 10 jobs you just created. After startup, they are queued for execution as directed by the Build Queue section:

After a few seconds or so, the Pods will be created to perform the build (you can verify this in the Rancher's Workload tab). Jenkins creates a pod for each job. When each agent starts, it connects to the master and receives jobs to be executed from the queue.

When the agent finishes its work, it is automatically removed from the cluster:

To check the status of a job, you can click a job in each agent. Click build from Build History, and then click Console Output. The job executed by the first agent should specify the use of the jenkins-slave1 Docker image, while the build performed by the second agent should specify the use of the jenkins-slave2 image:

If you get the output above, then Jenkins is configured correctly and is running as expected. Now you can start customizing your own Kubernetes build system to help your team test and release the software.

We configured Jenkins to automatically deploy the build agent on demand, connecting it to the Kubernetes cluster managed by Rancher. To this end, we have completed the following steps:

Create a cluster using Rancher

Created custom Docker images for Jenkins master and agents

Deploy Jenkins master and L4 LoadBalancer services on a Kubernetes cluster

The Jenkins kubernetes plug-in is configured on the cluster to generate dynamic agents automatically.

Test scenarios using multiple build jobs with dedicated agent images

The editor highlights the basic configuration necessary to set up the Jenkins master and agent architecture. We learned how Jenkins uses JNLP to start the agent and how the container automatically connects to Jenkins master to accept instructions. To achieve this, we use Rancher to create clusters, deploy workloads, and monitor the resulting Pods. After that, we rely on the Jenkins Kubernetes plug-in to connect all the different components together.

The above is how to deploy and scale Jenkins on Kubernetes. The editor believes that there are some knowledge points that we may see or use in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.