Shulou(Shulou.com)05/31 Report--

Today, I will talk to you about how to achieve TKE and Kubernetes access control. Many people may not know much about it. In order to make you understand better, the editor has summarized the following for you. I hope you can get something according to this article.

The following describes the access control and Kubernetes access control links on Tencent Cloud's TKE platform side, and demonstrates how to interface the platform-side accounts into the Kubernetes.

When you are using Tencent Cloud CCS TKE (Tencent Kubernetes Engine), do you encounter the following problems if multiple people share one account?

The secret key is shared by many people, so the risk of leakage is high.

It is impossible to restrict the access of others, and misoperation by others can easily cause security risks.

To solve the above problems, Tencent Cloud CAM (Cloud Access Management) provides an authentication system for main accounts and sub-accounts, as well as role-based permission control.

While different sub-accounts have a relatively coarse granularity of control over resources on the TKE platform (cluster instance level), they may encounter the following problems:

The same cluster can be accessed by multiple sub-accounts, so read and write control at the cluster resource level and namespace level cannot be guaranteed.

The high-authority sub-accounts of the cluster cannot authorize and manage the low-authority sub-accounts.

In order to solve the above two problems, TKE carries out corresponding access control management for platform-side resources and Kubernetes resources respectively.

Platform-side access control

First of all, what are platform-side resources? platform-side resources are Tencent Cloud resources such as Cluster resources, CVM resources, CLB resources, VPC resources and so on. Users visited are mainly divided into users and service role carriers.

The user is the main account, sub-account or collaborator account that we usually log on to the console.

A service role is a defined role with certain permissions, which can be assigned to a carrier, which can be another account or a service provider for the next product of Tencent Cloud. CAM defaults to a default carrier and default role provided by the product. For example, the default role of TKE is TKE_QCSRole, and the carrier is ccs.qcloud.com.

And what is the use of this character? Take TKE as an example. For example, the service-controller of TKE will Watch the Service resources in the cluster. If you need to create a Service of LoadBalance type, you will purchase and create CLB resources through the cloud API. While service-controller is deployed for users on the TKE platform, you need to have an identity to access the cloud API. This identity is the ccs.qcloud.com carrier, and the permission requires the user to grant a role to the carrier, that is, TKE_QCSRole. Only after the user authorizes the TKE carrier can the TKE purchase the CLB instead of the user by acting as a service. Below I will briefly show you how to authorize users and how to grant roles to the TKE platform.

Customized strategy

TKE subdivides the permissions of the API interface level of the cluster by connecting to CAM. You need to grant different permissions to sub-accounts in the CAM console. At the same time, TKE also provides default permissions on the CAM side, providing you with default choices, such as:

You can also customize policies. For specific policy customization, please refer to the CAM product introduction document.

For example, if a sub-account with read-only permission attempts to modify the cluster name, it will fail to verify the CAM permission during the API interface.

Divide user groups

User groups can be divided according to the responsibilities of the team, and the previously planned custom policies can be bound to a user group to facilitate rights management. For example, when new students join a designated user group (such as operation and maintenance group), they can obtain the permissions of the user group and avoid tedious permission configuration operations.

Grant TKE role permissions

To use TKE CCS, you need to grant the TKE platform permission to operate CVM\ CLB\ VPC\ CBS for you, so you need to agree to the authorization to access the TKE console for the first time, that is, to create a preset role TKE_QCSRole, which is granted to the TKE carrier by default. This role will be used to obtain the temporary key to operate your cluster through CAM to perform the corresponding cloud API operation.

More

For more rich platform-side access control usage, please visit the CAM product instructions.

Kubernetes access control

After introducing the access control of resources on the platform side, let's take a look at how to manage the permissions of resources in the TKE cluster. When different sub-accounts have access to the same TKE Kubernetes cluster, how to ensure that different sub-accounts have different roles and permissions for resources in the cluster? Let's first analyze the whole process from the Kubernetes access link of the community, so as to show you how TKE implements the interface between CCS sub-account and Kubernetes authentication and authorization system.

Overview

First of all, look at how the request link of Kubernetes is carried out from a macro point of view. The picture comes from the official website of K8s community.

You can roughly understand that a requested link is successively through Authentication (authentication, referred to as Authn), Authorization (authorization, referred to as Authz), AdmissionControl (admission control), so as to obtain the persistent data at the back end.

From the figure, you can see that Authn, Authz, and AdmissionControl are made up of multiple modules, and each step is constructed in a variety of ways.

Before entering the authentication module, the Request of HTTP is used to build context, and the context contains the user's RequestInfo,userInfo, Verb, APIGroup, Version, Namespace, Resource, Path and so on.

With this information, let's take a look at each step of the entry process at once.

Kubernetes certification

The process of certifying the identity of user.

There are two types of users in Kubernetes, one is ServiceAccount, and the other is the real users of the cluster.

The ServiceAccount account is created and managed by API (resources) provided by Kubernetes. ServiceAccount can be regarded as a special Secret resource, which can be used for user cluster resources to access APIServer authentication. It can be mounted into Pod through mount to use.

Real users usually initiate requests to access APIServer from outside, and administrators manage authentication credentials, while Kubernetes itself does not manage any user and credential information, that is, all users are logical users, and cannot create real users by calling Kubernetes API through API.

There are many ways of Kubernetes authentication, including TLS client certificate two-way authentication, BearerToken authentication, BasicAuthorization or authentication agent (WebHook).

All authentication methods are connected in series in the authentication link in the form of plug-ins, as long as there is a way of authentication, it can pass through the authentication module, and the subsequent authentication methods will not be implemented.

Referring to a little bit of the code of the Kubernetes APIServer Authentication module here, you can find that any authentication method is implemented by receiving a http Request request and then returning a user.Info structure, a bool, and an error.

/ / Request attempts to extract authentication information from a request and returns// information about the current user and true if successful, false if not successful,// or an error if the request could not be checked.type Request interface {AuthenticateRequest (req * http.Request) (user.Info, bool, error)}

User.Info contains the user's information, including UserName, UUID, Group, Extra.

Bool returns whether the user is authenticated or not. If false returns failed authentication, it returns a 401 error.

Error returns an error when the Request cannot be checked, and if it encounters an error, it will proceed to the next registration for authentication.

If the authentication is passed, the user.Info will be written to the requested context, and the subsequent request process can obtain user information at any time, such as authentication during authorization.

Next, I will select several authentication methods in the order of authentication methods in the Kubernetes code, and introduce you to the default authentication policy of the Kubernetes cluster created by TKE combined with the authentication method enabled by TKE.

Basic Authentication

APIServer startup parameter-basic-auth-file=SOMEFILE specifies the basic-certified csv file. Modifying this file after APIServer startup will not take effect. You need to restart APIServer to update the token of basic authentication. The csv file format is: token,user,uid, "group1,group2,group3".

When requesting, you need to specify the Authentication in HTTP Header as Basic, followed by the value of Base64Encode (user:passward).

X509 client certificate

The APIServer startup parameter-client-ca-file=SOMEFILE specifies the CA certificate, and during the creation of the K8s cluster of TKE, the cluster will be managed with a self-signed CA key and certificate. If the client certificate issued by the user is issued by the key of the CA certificate, it can be authenticated through the client certificate, and the CommonName and Group fields in the client certificate can be used as the Username and Group information in the UserInfo of the Kubernetes, respectively.

Currently, TKE peering sub-accounts use self-signed CA credentials to sign the client certificate of Uin corresponding to CN.

Bearer Token

There are many authentication methods for Bearer Token, such as those specified by startup parameters, ServiceAccount (also a special BeaerToken), BootstrapToken, OIDCIssure, WebhookToken

1. Specify Token csv files by default

APIServer startup parameter-token-auth-file=SOMEFILE specifies the Bearer Token certified csv file. It is similar to Basic Authentication, except that when requesting APIServer, the specified HTTP authentication method is Bearer. This Bearer can be followed directly with passward. The csv file format is: password,user,uid, "group1,group2,group3".

When requesting, you need to specify the Authentication in HTTP Header as Bearer, followed by the value of Base64Encode (user:passward).

2. ServiceAccount

ServiceAccount is also a special beaer token,ServiceAccount resource in Kubernetes. After creating a ServiceAccount resource, a Secret resource is created by default, and the Secret resource contains a Token field in JWT format, which can be verified by requesting Kube-APIServer,Kube-APIServer to parse part of the user information in the token in the way of Bearer Token, and whether the ServiceAccount below validate exists. This method is the common intra-cluster authentication method in the "two types of users" mentioned earlier, ServiceAccount, which is mainly used to access APIServer for resources in the cluster, but not limited to the cluster.

3. BootstrapToken

This switch is only the stable version in Kubernetes v1.18, and this type of Token is specifically used to boot the cluster installation and needs to work with controller-manager 's TokenCleaner.

This configuration is enabled by default for TKE.

4. OpenID Connect Tokens

The authentication method of OIDCToken is to access the APIServer by obtaining ID Token from the identity provider combined with OAuth3.

To enable this feature, you need to specify the configuration parameters of oidc in the startup parameters of APIServer. For example,-- oidc-issuer-url specifies the address of the oidc identity provider, and-- oidc-client-id specifies the user name of the account ID,--oidc-username-claim identity provider on the identity provider side.

For more information, please see the official documentation of Kubernetes. Currently, public cloud TKE does not use this parameter to interface with Tencent Cloud accounts, because users need active login authorization before returning to Id Token, which conflicts with the current official website and can be implemented in subsequent CLI tools.

5. Webhook Token Server

Webhook Token is a hook way to verify whether the authentication has been passed.

The APIServer startup parameters-- authentication-token-webhook-config-file and-- authentication-token-webhook-cache-ttl specify the Webhook address and the cache ttl of the token, respectively.

If APiServer enables this method for authentication and verification, after receiving the user's Request, the Bearer Token will be packaged as a TokenReview and sent to the WebHookServer. After receiving it, the server will verify it and return to the TokenReview API to feedback whether the verification and user.Info information is passed in the status field.

Summary

The above are several ways of Kubernetes APIServer authentication, and TKE supports each of them. For users to use flexibly.

At present, TKE is promoting the use of x509 client certificate for authentication management to facilitate the creation, authorization management and update of sub-accounts.

Kubernetes authorization

Kubernetes's authorization mode supports the following. Just like authentication, refer to the RequestInfo context at the beginning. We can see that the context of the user Reqeust, in addition to the userInfo required for authentication, has some other fields such as Verb, APIGroup, APIVersion, Resource, Namespaces, Path...

Authorization is to determine whether user has the appropriate permissions to manipulate resources.

Kubernetes supports AlwaysAllow, AlwaysDeny, Node, ABAC, RBAC and Webhook authorization Mode. Just like authentication, resources can be returned as long as one authentication module is passed.

Here we focus on the following two ways

RBAC

RBAC (Role-Based Access Control). Kubernetes provides ClusterRole and Role resources, which correspond to cluster dimension and Namespace dimension role permission control respectively. Users can customize the corresponding ClusterRole and Role resources and bind them to the authenticated User.

The following tke:pod-reader ClusterRole defines the get/watch/list operations that the role can access to pods resources under core apigroup

ApiVersion: rbac.authorization.k8s.io/v1kind: ClusterRolemetadata: name: tke:pod-readerrules:- apiGroups: ["] #" specify core API group resources: ["pods"] verbs: ["get", "watch" "list"]-apiVersion: rbac.authorization.k8s.io/v1# this role binding enables the user "alex" to read Podskind: ClusterRoleBindingmetadata: name: alex-ClusterRolesubjects:- kind: User name: alex apiGroup: rbac.authorization.k8s.ioroleRef: kind: ClusterRole name: tke:pod-reader # in the "default" namespace where the name must be the same as the Role or ClusterRole name you want to bind apiGroup: rbac.authorization.k8s.io

Through the above yaml configuration, the userInfo information in the requestInfo of the authorization module is the request of alex through the authentication module. When you go to the RBAC authorization module in the authorization module, you will query the ClusterRole/ClusterRoleBinding information of the cluster. Determine whether you have the right to operate on context.

TKE's permission authorization policy for connecting sub-accounts is to use the native RBAC of Kubernetes to control the access of sub-account resources, which accords with the native and K8s usage habits of users.

WebHook

The Webhook mode is based on HTTP callback by configuring the authorized webhook server address. When APIServer receives the request, it wrappers the SubjectAccessReview request Webhook Server,Webhook Server to determine whether it is accessible or not, and then returns the allow information.

The following is an example of the kubernetes community for reference.

"apiVersion": "authorization.k8s.io/v1beta1", "kind": "SubjectAccessReview", "spec": {"resourceAttributes": {"namespace": "kittensandponies", "verb": "get", "group": "unicorn.example.org", "resource": "pods"}, "user": "alex", "group": ["group1" "group2"]} {"apiVersion": "authorization.k8s.io/v1beta1", "kind": "SubjectAccessReview", "status": {"allowed": true}}

Currently, TKE does not consider using Webhook mode, but Webhook mode provides powerful flexibility, such as interfacing with CAM to achieve K8s permission docking to the platform side, but there are also certain risks and challenges, such as relying on the stability of CAM; request latency, cache / TTL configuration; CAM action configuration and K8s permission correspondence. This authorization model is still under consideration, and users who need it can give feedback.

Admission control

What is admission controller?

In a nutshell, Kubernetes admission controllers are plugins that govern and enforce how the cluster is used.

Admission controllers is a plug-in for K8s to manage and force users to operate the cluster.

Admission controllers is mainly divided into two phase, one is mutating, the other is validating. Both of these phases come after authn&authz, and the change admission made by mutating is to transform the resource of request, such as filling in the default requestLimit? Validating admission means authentication admittance, for example, the number of Pod copies must be greater than 2.

API Server request Link:

ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks, both of which are in beta status as of Kubernetes 1.13.

K8s supports more than 30 admission control plug-ins, and two of them have strong flexibility, namely ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks. These two kinds of control transformation and admission are provided to users in the way of Webhook, which greatly improves the flexibility. Users can create a custom AdmissionWebhookServer in the cluster to adjust the admission policy.

ValidatingAdmissionWebhooks and MutatingAdmissionWebhooks are also enabled by default in TKE version 1.10 and above.

For more information on Admission Controller, refer to here.

Kubernetes permission to connect sub-accounts

The main scheme of TKE authority to connect sub-accounts is x509 client authentication + Kubernetes RBAC authorization.

Authentication

Each sub-account has its own client certificate for accessing KubernetesAPIServer.

When users use TKE's new authorization mode, when different sub-accounts obtain cluster access credentials, that is, when the foreground accesses the cluster details page or calls DescribeClusterKubeconfig, the sub-account will display its own x509 client certificate, which is issued by the self-signed CA of each cluster.

When the user accesses Kubernetes resources in the console, the backend defaults to use the client certificate of this sub-account to access the user Kubernetes APIServer.

Support sub-account to update your certificate.

Accounts with primary account or cluster tke:admin permission are supported to view and update other sub-account certificates.

Authorization

The TKE console provides fine-grained Kubernetes resource-granularity permission control to sub-accounts through Kubernetes's native RBAC authorization policy.

An authorization management page is provided to allow the main account and cluster creator to have administrator permissions by default, and other sub-accounts with the DescribeCluster Action permission of this cluster can be managed.

And provide a preset ClusterRole.

Developer (tke:ns:dev): you need to select the specified namespace to read and write to the resources visible to the console under the selected namespace.

Read-only user (tke:ns:ro): you need to select the specified namespace for read-only access to resources visible to the console under the selected namespace.

Administrator (tke:admin): read and write permissions to resources under all namespaces, read and write permissions to cluster nodes, storage volumes, namespaces, quotas, but sub-accounts and permissions

Operation and maintenance staff (tke:ops): read and write permissions to console visible resources under all namespaces, read and write permissions to cluster nodes, storage volumes, namespaces and quotas

Developer (tke:dev): read and write access to console visible resources under all namespaces

Restricted person (tke:ro): read-only access to console-visible resources under all namespaces

User-defined ClusterRole

All namespace dimensions:

Specify the namespace dimension:

All preset ClusterRole will have a fixed label:cloud.tencent.com/tke-rbac-generated: "true"

All default ClusterRoleBinding comes with a fixed annotations:cloud.tencent.com/tke-account-nickname: yournickname, and label:cloud.tencent.com/tke-account: "yourUIN"

More

Of course, in addition to the default authorization policy provided by the TKE console, administrators can also use kubectl to operate ClusterRole/Role to flexibly configure fine-grained permissions for custom roles, and operate ClusterRoleBinding/RoleBinding to bind permissions to any role permissions.

For example, if you want to set the user group on the CAM side to be the pod-dev of productA products, you can only use the pods resources under the get/list/watch product-a namespace, then you can do this:

Create a custom ClusterRole/Role:dev-pod-reader,yaml instance as follows, with the file name developer.yaml

ApiVersion: rbac.authorization.k8s.io/v1kind: ClusterRole # here ClusterRole can be reused to other product namespaces metadata: name: pod-dev # pod-dev this role is the development rules:- apiGroups that can only read pod: ["] #" specify the core API group resources: ["pods"] verbs: ["get", "watch", "list"]

Use kubectl or create resources through the TKE console YAML to create the above Role

Bind the dev1, dev2 and dev3 users under the dev user group, and bind the custom permission pod-dev to the product-a namespace

From then on, dev1,dev2,dev3 users can only use get/list/watch to access pods resources under product-a.

$kubectl-kubeconfig=./dev.kubeconfig get podsError from server (Forbidden): pods is forbidden: User "10000001xxxx-1592395536" cannot list resource "pods" in API group "in the namespace" default "$kubectl-kubeconfig=./dev.kubeconfig get pods-n product-aNo resources found. After reading the above, do you have any further understanding of how to implement TKE and Kubernetes access control? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.