Shulou(Shulou.com)05/31 Report--

How to create K8S Job and deal with batch processing, aiming at this problem, this article introduces the corresponding analysis and solution in detail, hoping to help more partners who want to solve this problem to find a more simple and easy way.

Kubernetes jobs is mainly aimed at short-term and batch workloads. It runs for closure, rather than running continuously like other objects like deployment, replicasets, replication controllers, and DaemonSets.

Here are some tips on how to create Kubernetes jobs and cronjobs.

Kubernetes Jobs runs until the task specified in Job is completed. That is, if pods gives the exit code 0, then Job exits. In a normal Kubernetes, regardless of the exit code, the deployment object creates a new pod when it terminates or when an error occurs to maintain the ideal state of the deployment.

During the job operation, if the node hosting the pod fails, the Job pod will be automatically rescheduled to another node.

Kubernetes Jobs use case

The best use case practice for Kubernetes Jobs is:

Batch tasks: for example, you want to run batch tasks once a day, or on a specified schedule. It may be like reading files from a repository or database, assigning them to a service to process files.

Operations / ad-hoc tasks: for example, if you want to run a script / code, the script / code will run a database cleanup activity, or even back up a Kubernetes cluster.

How to create a Kubernetes Job

In this case, we will use the Ubuntu container to run a shell script with a for loop and respond to the message according to the parameters you pass to the container. This parameter is a number that determines how many times the shell script loop should be run.

For example, if you pass parameter 100, the shell script will respond to the message 100 times and the container will exit.

You can visit the following link to view Dockerfile and shell scripts: https://github.com/devopscube/Kubernetes-jobs-example/tree/master/Docker

Let's start with a simple set of job.

Step1: use a custom Docker image to create a job.yaml file with a command parameter of 100. 100 will be passed as an argument to the docker ENTRYPOINT script.

ApiVersion: batch/v1 kind: Job metadata: name: kubernetes-job-example labels: jobgroup: jobexample spec: template: metadata: name: kubejob labels: jobgroup: jobexample spec: containers:-name: C image: devopscube/kubernetes-job-demo:latest Args: [100th] restartPolicy: OnFailure

Step2: job for creating a job.yaml file using kubectl

Kubectl apply-f job.yam

Step3: use kubectl to check the status of job

Kubectl get jobs

Step4: use kubectl to get the pod list

Kubectl get po

Step5: use kubectl to get the job pod log. Replace the original Pod name with the Pod name you see in the output.

Kubectl logs kubernetes-job-example-bc7s9-f

Running multiple Job pods in parallel

When a job is deployed, you can have it run in parallel on multiple Pod. For example, if you want to run six pods and two pods in parallel in one job, you need to add the following two parameters to your job manifets:

Completions: 6parallelism: 2

Here is the manifest with those parameters:

ApiVersion: batch/v1kind: Jobmetadata: name: kubernetes-parallel-job labels: jobgroup: jobexamplespec: completions: 5 parallelism: 2 template: metadata: name: kubernetes-parallel-job labels: jobgroup: jobexamplespec: containers:-name: C image: devopscube/kubernetes-job-demo:latest args: [100 "] restartPolicy: OnFailure generates a random name for Kubernetes Job

You cannot create multiple job from one job manifest file because Kubernetes will report an error saying that there is a job with the same name. To circumvent this problem, you can add the generateName name parameter to the metadata.

For example:

ApiVersion: batch/v1kind: Jobmetadata: generateName: kube-job- labels: jobgroup: jobexample

In the above example, each time you run the manifest,job, it will be prefixed with kube-job-, followed by a random string to create it.

How to create a Kubernetes CronJob

If you want to run batch job on a specific schedule, for example, every 2 hours. You can create a Kubernetes cronjob using cron expressions. Job will start automatically according to the schedule you mentioned in job.

Next we will show you how to specify a cron plan, which you can use the crontab generator (https://crontab-generator.org/) to generate your own time plan.

Schedule: "0pr 15pr 30pr 45 *"

The following figure shows the Kubernetes cronjob schedule syntax. If we ran our previous job,manifest every 15 minutes in the form of cronjob, it would look like this. Create a file named cron-job.yaml and copy the following manifest:

ApiVersion: batch/v1beta1kind: CronJobmetadata: name: kubernetes-cron-jobspec: schedule: "0pje 15pje 30pc 45 *" jobTemplate: spec: template: metadata: labels: app: cron-batch-job spec: restartPolicy: OnFailure containers:-name: kube-cron-job image: devopscube/kubernetes-job-demo:latest args:

Let's deploy cronjob using kubectl. Kubectl create-f cron-job.yaml lists cronjobs:

Kubectl get cronjobs

You can check the Cronjob log by listing the cronjob pod and getting the log from the running or completed pods.

Run Kubernetes CronJob manually

In some cases, you may want to execute cronjob in an ad hoc manner. You can do this by creating a job from an existing cronjob.

For example, if you want to trigger a cronjob manually, we should do this:

Kubectl create job-from=cronjob/kubernetes-cron-job manual-cron-job

-- from=cronjob/kubernetes-cron-job copies the cronjob template and creates a job called manual-cron-job.

Key parameters of Kubernetes Job

Depending on your needs, you can also use several key parameters of kubernetes jobs/cronjobs:

FailedJobHistoryLimit & successfulJobsHistoryLimit: delete the job history of failures and successes based on the number of reservations you provide. This is useful for reducing all failed entries when you try to list job. For example:

BackoffLimit: if your Pod fails, the total number of retries.

ActiveDeadlineSeconds: you can use this parameter if you want to impose a hard limit on the running time of cronjob. For example, if you want to run cronjob for only 1 minute, you can set it to 60.

On how to create K8S Job and batch processing questions to share the answers here, I hope the above content can be of some help to you, if you still have a lot of doubts have not been solved, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.