Shulou(Shulou.com)05/31 Report--

This article mainly shows you "Kubernetes 1.21 how to introduce Indexed Jobs features", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "Kubernetes 1.21 how to introduce Indexed Jobs features" this article.

Once you containerize a non-parallel job (Job), it's easy to start and run it on Kubernetes without modifying the binaries. In most cases, when running parallel and distributed jobs, you must set up a separate system to divide work between work resources. For example, you can set up a task queue, assign a work item to each Pod [2], or assign multiple items to each Pod until the queue is empty [3].

Kubernetes version 1.21 introduces a new field to control the job completion mode, which is a configuration option that allows you to control how Pod completion affects the overall progress of the job. There are two possible options (currently):

NonIndexed (default): a job is considered completed when the number of successfully completed Pods equals the number specified in. Spec.completions. In other words, each Pod completion is homologous to each other. Any job created before the completion mode is introduced is implicitly non-indexed. Indexed: when there is a successfully completed Pod associated with each index from 0 to .spec.completions-1, the job is considered complete. The index is exposed to batch.kubernetes.io/job-completion-index comments and JOB_COMPLETION_INDEX environment variables in each Pod.

You can start using jobs with index completion mode, called index jobs (Indexed Jobs) for short, to easily start parallel jobs. Each worker Pod can then have a statically assigned data partition based on the index. This eliminates the need to set up a queuing system or even modify binaries!

Create an index job

To create an index job, you simply add completionMode: Indexed to the job specification and use the JOB_COMPLETION_INDEX environment variable.

ApiVersion: batch/v1

Kind: Job

Metadata:

Name: 'sample-job'

Spec:

Completions: 3

Parallelism: 3

CompletionMode: Indexed

Template:

Spec:

RestartPolicy: Never

Containers:

-command:

-'bash'

-'- c'

-'echo "My partition: ${JOB_COMPLETION_INDEX}"'

Image: 'docker.io/library/bash'

Name: 'sample-load'

Note that completion mode is an alpha feature in version 1.21. In order to be able to use it in the cluster, make sure that the IndexedJob feature gate is enabled on the API server and controller manager.

When you run the example, you will see that each of the three created Pods gets a different completion index. For the convenience of users, the control plane sets the JOB_COMPLETION_INDEX environment variable, but you can choose to set your own or expose the index as a file.

plan

SIG Apps envisions that there may be more completion patterns, enabling more use cases for Job API. We welcome your suggestion in kubernetes/kubernetes [5].

In particular, we are considering an IndexedAndUnique pattern where indexes are not only available as comments, but also part of the Pod name, similar to StatefulSet. This should facilitate communication between tightly coupled Pod. You can take part in the discussion of this open question [6].

Summary

Index jobs (Indexed Jobs) allow you to statically divide jobs between jobs that run in parallel. SIG Apps hopes that this feature will facilitate the migration of more bulk workloads to Kubernetes.

These are all the contents of the article "how to introduce Indexed Jobs features in Kubernetes 1.21". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.