Shulou(Shulou.com)06/02 Report--

This article will explain in detail how to configure safety management for K8S production system. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

PX-Security: multi-tenant authorization, authentication, and RBAC security for Kubernetes persistent volumes are critical for enterprises, especially for micro-service-based applications running on Kubernetes. Kubernetes provides RBAC authorization to manage access to specific resources within Kubernetes according to different role settings. These mechanisms are useful for managing access to specific objects of Kubernetes, such as services, namespaces, quotas, and so on. But namespaces and pod's security policy alone are not sufficient to limit who has permission to request changes to the underlying data management system. Many enterprises use API, such as CNI or CSI, to invoke other platform capabilities to provide network and storage infrastructure. What Kubernetes lacks is the ability to extend RBAC to these systems, and external systems are needed to ensure the implementation of authorization and authentication functions. This is why Portworx and Kubernetes work together to provide RBAC, encryption, and control permissions through access role control over the persistent volumes that support PVCs in Kubernetes, which creates a seamless layer of protection that provides the following protection for your PVCs:

1. Users in the same namespace can be restricted by their roles, such as they can have read, write, administrator, or other defined access.

two。 Users can authenticate automatically through Token, so that the authorization of audit requests can be done for a specific namespace.

3. Users can be placed in a tenant-based namespace to provide secure multi-tenancy access to the PVCs.

4. Even if users see the storage class, it does not mean that they are authorized to create a PVC.

5. Using Portworx RBAC with encryption means that the data is secure on the host and that unauthorized users in the namespace cannot access the data.

6. If a request comes from outside the Kubernetes without a Token, it will be blocked.

To learn more about what Portworx can provide for your Kubernetes platform, take a look at the security reference architecture on the Portworx website. We will focus on topics such as how to set up PX-Security and how to use Token to authenticate role users with corresponding persistent volume access.

First, Portworx supports RBAC by using Token. In this article, PX-Security will use Token stored in Kubernetes Secrets, which provide the most flexible operation without sacrificing any security.

Let's go! Before we discuss what Token is and how to use it, it is important to note that each request to Portworx is authenticated and authorized using the information stored in Token. It contains information about all relevant authentication and signatures required by the user and his role. Therefore, after we have configured PX-Security, we will create a Token.

Let's configure PX-Security to achieve security. Please visit https://central.portworx.com

Click install and run. Fill in the information required by the installer, download your YAML file when you are finished, and save the file for later editing.

Next, we will create a Secrets for secure sharing of PX-Security. We must first create these shared Secrets because the storage administrator will use them to generate and verify the Token. For security reasons, this data is stored in the Kubernetes Secrets in the Kube-system namespace, which is accessible to only a small number of administrators and applications.

PORTWORX_AUTH_SYSTEM_KEY=$ (cat / dev/urandom | base64 | fold-w 65 | head-N1) PORTWORX_AUTH_STORK_KEY=$ (cat / dev/urandom | base64 | fold-w 64 | head-n 1) PORTWORX_AUTH_SHARED_SECRET=$ (cat / dev/urandom | base64 | fold-w 64 | head-n 1)

Run the following command to put these values into Kubernetes Secret:

Kubectl-n kube-system create secret generic pxkeys\-- from-literal=system-secret=$PORTWORX_AUTH_SYSTEM_KEY\-- from-literal=stork-secret=$PORTWORX_AUTH_STORK_KEY\-- from-literal=shared-secret=$PORTWORX_AUTH_SHARED_SECRET

You can test the share-secret in Kubernetes using the following command.

Kubectl-n kube-system get secret pxkeys-o json | jq-r '.data. "shared-secret"' | base64-d

Open your YAML file, find PortworxDaemonset, you can see args, under image:portworx/oci-monitor. Here we will add the security parameters (bold) and the Secrets required by PX-Security:

-name: portworx image: portworx/oci-monitor:2.1.5 imagePullPolicy: Always args: ["- c", "px-cluster", "- s", "/ dev/xvdf", "- secret_type", "k8s", "- b", "- x", "kubernetes" "- jwt_issuer" "example-domain.com"] env:-name: "AUTO_NODE_RECOVERY_TIMEOUT_IN_SECS" value: "1500"-name: "PX_TEMPLATE_VERSION" value: "v4"-name: "PORTWORX_AUTH_JWT_SHAREDSECRET" valueFrom: secretKeyRef: Name: pxkeys key: shared-secret-name: "PORTWORX_AUTH_SYSTEM_KEY" valueFrom: secretKeyRef: name: pxkeys key: system-secret-name: "PORTWORX_AUTH_STORK_KEY" valueFrom: SecretKeyRef: name: pxkeys key: stork-secret

We also need to find the Stork deployment and editing environment to include our shared secret. See below.

Containers:-command:-/ stork-- driver=pxd-verbose-- leader-elect=true-- health-monitor-interval=120 imagePullPolicy: Always image: openstorage/stork:2.2.5 env:-name: "PX_SERVICE_NAME" value: "portworx-api"-name: " PX_SHARED_SECRET "valueFrom: secretKeyRef: name: pxkeys key: stork-secret

After completing these two steps, save the YAML file. Now we need to create the shared Secret that we referenced in the Portworx installation YAML.

Next, create a Portworx cluster using the downloaded and edited YAML files.

$kubectl apply-f px-sec-cluster-spec.yaml$ kubectl get po-n kube-system-l name=portworxNAME READY STATUS RESTARTS AGEportworx-4vmcx 1 to 1 Running 0 3m54sportworx-hwrxh 1 to 1 Running 0 3m54sportworx-rbqzk 1 to 1 Running 0 3m54s

Users and Token

We need to define several users and generate Token for them. Typically, these users have attributes assigned to them, which define their user type.

First, we will create a storage administrator with full permissions. There should be only one or two such administrators.

Create a file named admin.yaml with the following:

Name: Storage Administratoremail: storageadmin@example.comsub: storageadmin@example.com/storageadminroles: ["system.admin"] groups: ["*"]

Next, we will create a Kubernetes user who, as an authenticated customer, Kubernetes allows the user to interact with Portworx, and these requests come from Kubernetes. Your storage administrator needs to set up this user.

Create a file named kubernetes.yaml with the following:

Name: Kubernetesemail: kubernetes@local.netsub: kubernetes@local.net/kubernetesroles: ["system.user"] groups: ["kubernetes"]

Finally, we will create a view-only-only user to demonstrate how Portworx restricts access to the underlying data management API.

Create a file named viewer.yaml with the following:

Name: Vieweremail: viewer@example.comsub: viewer@example.com/viewerroles: ["system.view"] groups: ["viewers"]

Note: the Sub tag is the unique identifier for this user and cannot be shared with other Token according to the JWT standard. This value is used by Portworx to track the ownership of resources. If the email is also used as a unique Sub identifier, make sure that it is not used by any other Token.

Note that one user's role is system.admin, another user's role is system.user, and the last user's role is system.view. These roles are the default in PX-Security, but you can also create other roles as needed. Let's demonstrate how users can use Portworx resources, such as creating or deleting volumes. This has nothing to do with the user's configuration on Kubernetes RBAC.

That is, users with system.view may be able to list and create PVC objects within Kubernetes, but if they try to create volumes directly using Portworx, they will fail. We will also demonstrate why a user who can create a PVC object cannot actually get PV in this security mode unless the user has a valid Token configured by the storage administrator to verify their roles and permissions.

Once these users with relevant permissions are configured, we can use our shared Secret and Portworx client tool pxctl to generate self-signed certificates for these users.

Note: you can create your own application to generate Token, or you can build on our open source golang sample program openstorage-sdk-auth

In this scenario, we will use shared Secret and pxctl auth token generate commands. Let's create a Token for the above two users.

First, get the shared Secret.

PORTWORX_AUTH_SHARED_SECRET=$ (kubectl-n kube-system get secret pxkeys-o json\ | jq-r '.data. "shared-secret"'\ | base64-d)

Next, configure Admin Token. First, SSH, to the Portworx node, so that you can use the pxctl command.

$PX_POD=$ (kubectl get pods-l name=portworx-n kube-system-o jsonpath=' {.items [0] .metadata.name}') $kubectl exec-it-n kube-system $PX_POD bash

Then, create the admin Token using admin.yaml and shared Secret.

Note: be sure to copy the auth_config file and PORTWORX_AUTH_SHARED_SECRET into the Portworx container that is using pxctl.

ADMIN_TOKEN=$ (/ opt/pwx/bin/pxctl auth token generate\-- auth-config=admin.yaml\-- issuer=example-domain.com\-- shared-secret=$PORTWORX_AUTH_SHARED_SECRET\-- token-duration=1y) $pxctl context create admin-- token $ADMIN_TOKEN

Next, configure Kubernetes Token.

KUBE_TOKEN=$ (/ opt/pwx/bin/pxctl auth token generate\-auth-config=kubernetes.yaml\-issuer=example-domain.com\-shared-secret=$PORTWORX_AUTH_SHARED_SECRET\-token-duration=1y)

Next, configure Viewer token.

VIEWER_TOKEN=$ (/ opt/pwx/bin/pxctl auth token generate\-- auth-config=viewer.yaml\-- issuer=example-domain.com\-- shared-secret=$PORTWORX_AUTH_SHARED_SECRET\-- token-duration=1y) $pxctl context create viewer-- token $VIEWER_TOKEN

Now that we have created a user environment (Context), such as Portworx's Kubectl environment, for two users, we can interact with the Portworx system as one of the users.

Note that you can use $pxctl contextlist to list all environments:

$pxctl context set viewer$ pxctl volume create-- size 5 myvolVolumeCreate: Access denied to roles: [system.view]

What happened? Remember, we set up the user environment for Viewer with the system.view role. This is the default role of Portworx and can only run read-only commands and does not have write permission, so access is denied.

How do I use it with Kubernetes?

In order for Kubernetes users to use PX-Security, users must use their own Token when making requests to the cluster. One way is to have the administrator configure Token in the Kubernetes storage class. The administrator can set the save Secret Token in the Secret named px-k8-user in the Portworx namespace.

ApiVersion: storage.k8s.io/v1kind: StorageClassmetadata: name: px-storage-repl-1provisioner: kubernetes.io/portworx-volumeparameters: repl: "1" openstorage.io/auth-secret-name: px-k8s-user openstorage.io/auth-secret-namespace: portworxallowVolumeExpansion: true

If you are using CSI, be sure to set other parameters.

Note: this currently works only when using CSI through Portworx.

Parameters: repl: "1" csi.storage.k8s.io/provisioner-secret-name: px-k8s-user csi.storage.k8s.io/provisioner-secret-namespace: portworx csi.storage.k8s.io/node-publish-secret-name: px-k8s-user csi.storage.k8s.io/node-publish-secret-namespace: portworx csi.storage.k8s.io/controller-expand-secret-name: px-k8s-user csi.storage.k8s.io/controller-expand-secret-namespace: portworx

"after this is done, users with access to the storage class can create volumes."

Kind: PersistentVolumeClaimapiVersion: v1metadata: name: mysql-dataspec: storageClass: px-storage-repl-1 accessModes:-ReadWriteOnce resources: requests: storage: 12Gi

Multi-tenant architecture

When you create the above PVC, it uses KubernetesToken as the user for authentication, ensuring that the Kubernetes user is the requesting user. This is good, but in a multi-tenant environment, they can all use storage classes, so we need a way to use multi-tenant Token in different namespaces. This is because Kubernetes provides a good way to isolate account resources using namespaces, but you need a more secure multi-tenant solution. Portworx can achieve multi-tenant security management by adding access controls to applied storage volumes. Using PX-Security for multi-tenant management, you can do the following.

First, create a namespace for the tenant.

$kubectl create namespace tenant-a-ns

Create a file called tenant-a.yaml using the following:

Name: tenant-aemail: tenant@tenant-a.comsub: tenant@tenant-a.com/tenantroles: ["system.user"] groups: ["developers"]

Use tenant-name.yaml to create a token for Kubernetes:

TENANT_A_TOKEN=$ (/ opt/pwx/bin/pxctl auth token generate\-auth-config=tenant-a.yaml\-issuer=example-domain.com\-shared-secret=$PORTWORX_AUTH_SHARED_SECRET\-token-duration=1y)

Save the tenant's Kubernetes Token in a Secret named / px-k8-user:

$kubectl-n tenant-a-ns create secret\ generic px-k8s-user\-- from-literal=auth-token=$TENANT_A_TOKEN

You can now set up the Kubernetes storage class, use this Secret to gain Token permissions, and start communicating with Portworx.

The following CSI storage class, once created, will enable your multi-tenancy to create volumes using the Token stored in their namespace Secret by looking for Secret in the namespace. When using CSI, the storage class references secret:provision, node-publish (mount/unmount), and and controller-expand of the three supported operations.

ApiVersion: storage.k8s.io/v1kind: StorageClassmetadata: name: px-storageprovisioner: pxd.portworx.comparameters: repl: "1" csi.storage.k8s.io/provisioner-secret-name: px-k8s-user csi.storage.k8s.io/provisioner-secret-namespace: ${pvc.namespace} csi.storage.k8s.io/node-publish-secret-name: px-k8s-user csi.storage.k8s.io/node-publish-secret-namespace: ${pvc.namespace} csi.storage .k8s.io / controller-expand-secret-name: px-k8s-user csi.storage.k8s.io/controller-expand-secret-namespace: ${pvc.namespace} allowVolumeExpansion: true

Please note ${pvc.namespace}. This ensures that the CSI controller gets the correct Token, which is associated with the namespace of PVC. You now have a multi-tenant solution based on Token authentication.

The PX-Security function that we have not covered in this Blog is the encryption of Portworx volumes. You can see more documents about PVC encryption here (how to work with Encrypted PVCs). You can use Portworx RBAC with volume encryption to make the data in Kubernetes more secure.

On how to configure safety management for K8S production system is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.