Shulou(Shulou.com)05/31 Report--

This article introduces the knowledge of "how to configure keys in config and cluster.spec". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Description of keys in config in kops and keys in cluster.spec

The description here is incomplete, but it is intended to document key that are not easy to explain. Our godoc reference provides a more detailed list of API values. There are two top-level API values that describe the cluster in the YAML file: ClusterSpec is defined as kind:YAML,InstanceGroup and kind:YAML is defined.

Specapi

This object configures how we expose API:

Dns will allow direct access to the master instance and configure the DNS to point directly to the master node.

LoadBalancer will configure a load balancer (ELB) in front of the primary node and configure DNS to point to ELB.

DNS example:

Spec: api: dns: {}

When configuring LoadBalancer, you can choose to use public ELB or intranet (VPC only) ELB. The type field should be Public or Internal.

In addition, you can add other pre-created security groups to the load balancer by setting up additionalSecurityGroups.

Spec: api: loadBalancer: type: Public additionalSecurityGroups:-sg-xxxxxxxx-sg-xxxxxxxx

In addition, you can increase the idle timeout (idle timeout) of the load balancer by setting idleTimeoutSeconds. The default idle timeout is 5 minutes, and AWS allows up to 3600 seconds (60 minutes). For more information, see configuring Idle timeouts.

Spec: api: loadBalancer: type: Public idleTimeoutSeconds: 300etcdClusters v3 & tls

Although kops is not currently etcd3 by default, v3 and TLS authentication can be turned on for communication between cluster members. These options can be enabled through the cluster configuration file (only in manifests, no command line options). Warning: there is currently no upgrade path to migrate from v2 to v3, so do not try to enable this feature on the v2 running cluster, as it must be done when the cluster is created. The following sample code snippet assumes a HA cluster of three primary servers.

EtcdClusters:- etcdMembers:-instanceGroup: master0-az0 name: a muri 1-instanceGroup: master1-az0 name: a Muay 2-instanceGroup: master0-az1 name: B Mel 1 enableEtcdTLS: true name: main version: 3.0.17-etcdMembers:-instanceGroup: master0-az0 name: a Muir 1-instanceGroup: master1-az0 name: a Muir 2-instanceGroup: master0-az1 name: BMI 1 enableEtcdTLS: true name: events version: 3.0.17kubernetesApiAccess

The array is configured with CIDR that can access the kubernetes API. On AWS, this is represented as an inbound security group rule on the ELB or primary security group.

For example, use this key to restrict cluster access to the office IP address range.

Spec: kubernetesApiAccess:-12.34.56.78/32cluster.spec Subnet Keysid

Subnet ID to be shared in the existing VPC.

Egress

The resource identifier (ID) in the existing VPC that you want to use as the "exit" of the network.

This feature was originally conceived to allow the reuse of NAT gateways. The usage is as follows. Although NAT gateways are targeted at "public" resources, in the cluster specification, they must be specified in the private private subnet section. One way to think about this is if you specify "egress", which is the default route from this private subnet.

Spec: subnets:-cidr: 10.20.64.0 name 21 name: us-east-1a egress: nat-987654321 type: Private zone: us-east-1a-cidr: 10.20.32.0 name: utility-us-east-1a id: subnet-12345 type: Utility zone: us-east-1apublicIP

The IP of the existing EIP that you want to add to the NAT gateway.

Spec: subnets:-cidr: 10.20.64.0 type 21 name: us-east-1a publicIP: 203.93.148.142 type: Private zone: us-east-1akubeAPIServer

This section contains the configuration of kube-apiserver.

Oidc flag for Open ID Connect tokens spec: kubeAPIServer: oidcIssuerURL: https://your-oidc-provider.svc.cluster.local oidcClientID: kubernetes oidcUsernameClaim: sub oidcUsernamePrefix: "oidc:" oidcGroupsClaim: user_roles oidcGroupsPrefix: "oidc:" oidcCAFile: / etc/kubernetes/ssl/kc-ca.pemaudit loggingspec: kubeAPIServer: auditLogPath: / var/log/kube-apiserver-audit.log auditLogMaxAge: 10 auditLogMaxBackups: 1 auditLogMaxSize: 100 auditPolicyFile: / srv/kubernetes/audit.yaml

Note: auditPolicyFile is required. If the flag is omitted, no event is logged.

You can use the fileAssets feature to push advanced audit policy files on the master node.

The sample policy file can be found here

Max Requests Inflight

The maximum number of non-mutating requests in flight at a given time. When the server exceeds this, it rejects the request. Zero is unlimited. (default 400)

Spec: kubeAPIServer: maxRequestsInflight: 1000runtimeConfig

The keys and values here are translated into the values of-- runtime-config in kube-apiserver, separated by commas.

Use it to enable alpha features, such as:

Spec: kubeAPIServer: runtimeConfig: batch/v2alpha1: "true" apps/v1alpha1: "true"

The parameter, runtime-config=batch/v2alpha1=true,apps/v1alpha1=true, will be generated. Note that kube-apiserver accepts the value of true as the switch flag.

ServiceNodePortRange

This value is used as the-- service-node-port-range parameter of kube-apiserver.

Spec: kubeAPIServer: serviceNodePortRange: 30000-33000externalDns

This part of the configuration options are provided for external DNS. The current external DNS provider is kops dns-controller, which sets DNS records for Kubernetes resources. Dns-controller plans to phase out and replace it with external-dns.

Spec: externalDns: watchIngress: true

The default kops behavior is false. WatchIngress:true uses the default dns-controller behavior to monitor ingress controller changes. In some cases, setting this option risks interrupting service updates.

Kubelet

This section contains the configuration of kubelet.

Note: if the corresponding configuration value can be null, you can set the field to empty in spec so that the empty string can be passed to kubelet as the configuration value.

Spec: kubelet: resolvConf: ""

A flag, resolv-conf=, will be created.

Enable custom metrics support

To follow [Custom metrics doc We use custom metrics in kubernetes, we must set the flag-- enable-custom-metrics to true on all kubelets. It can be specified in the kubelet specification in our cluster.yml.

Spec: kubelet: enableCustomMetrics: truekubeScheduler

This section contains the configuration of kube-scheduler.

Spec: kubeScheduler: usePolicyConfigMap: true

Will cause kube-scheduler to use the "scheduler-policy" scheduler policy of configmap in the namespace kube-system.

Note that kube-scheduler from Kubernetes 1.8.0 does not automatically reload its configuration from configmap. You need to enter the master instance and restart the Docker container manually.

KubeControllerManager

This section contains the configuration of controller-manager.

Spec: kubeControllerManager: horizontalPodAutoscalerSyncPeriod: 15s horizontalPodAutoscalerDownscaleDelay: 5m0s horizontalPodAutoscalerUpscaleDelay: 3m0sFeature Gatesspec: kubelet: featureGates: Accelerators: "true" AllowExtTrafficLocalEndpoints: "false"

Flag-- feature-gates=Accelerators=true,AllowExtTrafficLocalEndpoints=false will be generated

Note: Feature gate ExperimentalCriticalPodAnnotation is enabled by default because some key components such as kube-proxy depend on it.

Spec: kubelet: kubeReserved: cpu: "100m" memory: "100Mi" storage: "1Gi" kubeReservedCgroup: "/ kube-reserved" systemReserved: cpu: "100m" memory: "100Mi" storage: "1Gi" systemReservedCgroup: "/ system-reserved" enforceNodeAllocatable: "pods,system-reserved,kube-reserved"

A flag will be generated:-- kube-reserved=cpu=100m,memory=100Mi,storage=1Gi-- kube-reserved-cgroup=/kube-reserved-- system-reserved=cpu=100mi,memory=100Mi,storage=1Gi-- system-reserved-cgroup=/system-reserved-- enforce-node-allocatable=pods,system-reserved,kube-reserved.

NetworkID

On AWS, this is the ID of the VPC that created the cluster. If you need to create a cluster from scratch, you can not specify this field at creation time; kops will create a VPC for you.

Spec: networkID: vpc-abcdefg1hooks

Hooks (hooks) allow you to perform some actions before each node in the cluster installs Kubernetes. For example, you can install the Nvidia driver to use GPU. This hook can be in the form of a Docker image or a manifest (systemd unit). Hooks can be placed in the cluster spec, which means that they will be deployed globally, or they can be placed in instanceGroup spec. Note: when the instanceGroup is the same as the service name on the cluster spec, instanceGroup takes precedence and ignores the definition in the cluster spec; that is, if there is a cell file "myunit.service" in the cluster and there is a unit file "myunit.service" in the instanceGroup, the file in the instanceGroup is applied.

Spec: # many sections removed hooks:-before:-some_service.service requires:-docker.service execContainer: image: kopeio/nvidia-bootstrap:1.6 # these are added as-e to the docker environment environment: AWS_REGION: eu-west-1 SOME_VAR: SOME_VALUE # or a raw systemd unit hooks:-name: iptable-restore.service roles:-Node-Master before :-kubelet.service manifest: | [Service] EnvironmentFile=/etc/environment # do some stuff # or disable a systemd unit hooks:-name: update-engine.service disabled: true # or you could wrap this into a full unit hooks:-name: disable-update-engine.service before:-update-engine.service manifest: | Type=oneshot ExecStart=/usr/bin/systemctl stop update-engine.service

Install Ceph

Spec: # many sections removed hooks:-execContainer: command:-sh-- c-chroot / rootfs apt-get update & & chroot / rootfs apt-get install-y ceph-common image: busyboxfileAssets

FileAssets is an alpha feature that allows you to put the contents of inline files into your cluster and instanceGroup configuration. It is designed as alpha, and you can replace it with kubernetes daemonsets.

Spec: fileAssets:-name: iptable-restore # Note if not path is specificied the default path it / srv/kubernetes/assets/ path: / var/lib/iptables/rules-save roles: [Master,Node,Bastion] # a list of roles to apply the asset to, zero defaults to all content: | some file contentcloudConfigdisableSecurityGroupIngress

If you use aws as a cloudProvider, you can disable authorization of the ELB security group to the Kubernetes Nodes security group. In other words, it does not add security group rules. This can be useful to avoid AWS restrictions: 50 rules per security group.

Spec: cloudConfig: disableSecurityGroupIngress: trueelbSecurityGroup

Warning: this applies only to Kubernetes versions above 1.7.0.

To avoid creating a security group for each elb, you can specify the security group id, which will be assigned to the LoadBalancer. It must be a security group ID, not a name. Api.loadBalancer.additionalSecurityGroups must be empty because Kubernetes adds rules for each port specified in the service file. This avoids the AWS limit: 500 security groups per zone and 50 rules per security group.

Spec: cloudConfig: elbSecurityGroup: sg-123445678docker

The Docker daemon option for all masters and nodes in the cluster can be overridden. Check the API documentation for a complete list of options.

RegistryMirrors

If you have a bunch of Docker instances (physicsal or vm) running, whenever one of them has an image that does not exist on the pull host, it will pull it from the DockerHub. By caching these images, you can keep traffic on the local network and avoid egress bandwidth usage. This setting is not only conducive to cluster configuration, but also conducive to images pull.

@ see Cache-Mirror Dockerhub For Speed @ see Configure the Docker daemon.

Spec: docker: registryMirrors:-https://registry.example.comstorage

You can specify a Docker storage driver to override the default values. Make sure the driver you choose is supported by the operating system and the docker version.

Docker: storage: devicemapper storageOpts:-"dm.thinpooldev=/dev/mapper/thin-pool"-"dm.use_deferred_deletion=true"-"dm.use_deferred_removal=true" sshKeyName

In some cases, you may need to use an existing AWS SSH key instead of allowing kops to create a new key. The name of the key provided in AWS is an alternative to-- ssh-public-key.

Spec: sshKeyName: myexistingkeytarget

In some use cases, you may want to use additional options to increase the target output. Goals support the least number of options, and you can do that. Currently, only the goal of terraform supports this, but if other use cases arise, kops may eventually support more.

Spec: target: terraform: providerExtraConfig: alias: foo "how to configure keys in config and cluster.spec" ends here. Thank you for reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.