Shulou(Shulou.com)06/02 Report--

Editor to share with you the example analysis of ingress resources and ingress controller in docker, I believe that most people do not know much about it, so share this article for your reference, I hope you can learn a lot after reading this article, let's go to know it!

Ingress: a tool that can expose services in a cluster using load balancers such as Nginx (rarely used), Haproxy (infrequent), Traefik (commonly used), Envoy (commonly used), etc.

Ingress provides you with layer-7 load balancer capability. You can provide externally accessible URL, load balancer, SSL, name-based virtual hosts, etc., through Ingress configuration. As the access layer of cluster traffic, the high reliability of Ingress is particularly important.

Little knowledge: we publish the pod service in K8s outside the cluster, either using ingress or NodePort.

ExternalLB: external load balancer

Service site: it's just for grouping and categorizing pod.

[root@master manifests] # kubectl explain ingress

Create a namespace:

[root@master manifests] # kubectl create namespace ingress-nginxnamespace/dev created [root@master manifests] # kubectl get nsNAME STATUS AGEdefault Active 17dingress-nginx Active 8skube-public Active 17dkube-system Active 17d

Visit https://github.com/kubernetes/ingress-nginx, go to the deploy directory, and there is the yaml file we want to use.

Function of each file: configmap.yaml: provide configmap configuration for nginx online default-backend.yaml: provide a default background error page 404namespace.yaml: create an independent namespace ingress-nginxrbac.yaml: create a corresponding role rolebinding for rbactcp-services-configmap.yaml: modify L4 load balancer configuration configmapudp-services-configmap.yaml: modify L4 load balancer configuration configmapwith-rbac.yaml: nginx-ingress-controller component with application rbac

Visit https://kubernetes.github.io/ingress-nginx/deploy/#generice-deployment, which contains the deployment documents for ingress

[root@master ~] # mkdir ingress-nginx [root@master ~] # cd ingress-nginx

Deployment ingress method 1 (step by step deployment):

Download the following configuration file:

[root@master ingress-nginx] # for file in namespace.yaml configmap.yaml rbac.yaml tcp-services-configmap.yaml with-rbac.yaml udp-services-configmap.yaml; do wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/$file; done

[root@master ingress-nginx] # lsconfigmap.yaml namespace.yaml rbac.yaml tcp-services-configmap.yaml udp-services-configmap.yaml with-rbac.yaml

1. Create a namespace:

[root@master ingress-nginx] # kubectl apply-f namespace.yaml namespace/ingress-nginx configured

2. Apply all the remaining ymal files

[root@master ingress-nginx] # kubectl apply-f. / configmap/nginx-configuration creatednamespace/ingress-nginx configuredserviceaccount/nginx-ingress-serviceaccount createdclusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole createdrole.rbac.authorization.k8s.io/nginx-ingress-role createdrolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding createdclusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding createdconfigmap/tcp-services createdconfigmap/udp-services createddeployment.extensions/nginx-ingress-controller created [root@master ingress-nginx] # kubectl get pods-n ingress-nginx-wNAME READY STATUS RESTARTS AGEdefault-http-backend-6586bc58b6-qd9fk 0 running 0 4mnginx-ingress-controller-6bd7c597cb-zcbbz 0 running 0 1m

You can see that there are two pod in the ingress-nginx namespace that are in the running state

Deploy ingress method 2 (one-click deployment):

Download only the mandatory.yaml file, because it contains the contents of all the above yaml files. This is an one-click deployment.

[root@master ingress-nginx] # wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml[root@master ingress-nginx] # kubectl apply-f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml[root@master ~] # kubectl get pods-n ingress-nginx-w-o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODEdefault-http-backend-6586bc58b6-qd9fk 1/1 Running 0 11h 10.244.1.95 node1 nginx-ingress-controller-6bd7c597cb-jlqzp 1/1 Running 3 11h 10.244.1.96 node1

You can see that there are two pod in the ingress-nginx namespace that are in the running state

Install service-nodeport

Above, we deployed ingress-nginx to node 1. Next, we need to deploy a service-nodeport service to connect external traffic to the cluster.

[root@master ingress] # wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml

In order to prevent service nodeport from automatically assigning ports, we specify nodeport ourselves, and add two nodePort parameters to the file. The final result is as follows:

[root@master ingress] # cat service-nodeport.yaml apiVersion: v1kind: Servicemetadata: name: ingress-nginx namespace: ingress-nginx labels: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec: type: NodePort ports:-name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30080-name: https port: 443targetPort: 443protocol: TCP nodePort: 30443 selector: app.kubernetes.io/name: Ingress-nginx app.kubernetes.io/part-of: ingress-nginx [root@master ingress] # kubectl apply-f service-nodeport.yaml service/ingress-nginx created [root@master ingress] # kubectl get svc-n ingress-nginxNAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGEdefault-http-backend ClusterIP 10.110.74.183 80/TCP 12hingress-nginx NodePort 10.102.78.188 80:30080/TCP 443:30443/TCP 2m

Above, I see 80 corresponding to 30080 and 43 corresponding to 30443.

We can access the application directly through the ip of the node1 node:

[root@master ingress] # curl http://172.16.1.101:30080default backend-404

Define myapp service

[root@master manifests] # mkdir / root/manifests/ingress [root@master ~] # kubectl explain service.spec.ports [root@master ingress] # cat deploy-demo.yaml apiVersion: v1kind: Service# must be set to headless servicemetadata: name: myapp namespace: defaultspec: selector: app: myapp release: canary ports:-name: http targetPort: 80 # this is the container port port: 80 # this is service port---apiVersion: apps/v1kind: Deploymentmetadata: name: myapp-deploy Namespace: defaultspec: replicas: 2 selector: # tag selector matchLabels: # matches the tag app: myapp release: canary template: metadata: labels: app: myapp # and the above myapp to match release: canary spec: containers:-name: myapp image: ikubernetes/myapp:v1 ports:-name: http containerPort: 80 [ Root@master ingress] # kubectl apply-f deploy-demo.yaml service/myapp createddeployment.apps/myapp-deploy unchanged [root@master ingress] # kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGEkubernetes ClusterIP 10.96.0.1 443/TCP 17dmyapp ClusterIP 10.108.177.62 80/TCP 1m [root@master ingress] # kubectl get podsNAME READY STATUS RESTARTS AGEmyapp-deploy-69b47bc96d-79fqh 1/1 Running 0 1dmyapp-deploy-69b47bc96d-tc54k 1/1 Running 0 1d

Publish myapp service through ingress

Let's define a manifest file to publish the myapp application through Ingress (equivalent to the reverse proxy function of nginx):

[root@master ingress] # cat ingress-myapp.yaml apiVersion: extensions/v1beta1 kind: Ingressmetadata: name: ingress-myapp namespace: default # to be in the same namespace as deployment and the service to be released annotations: # this note indicates that the ingress-controller we are going to use is nginx, not traefic,enjoy kubernetes.io/ingress.class: "nginx" spec: rules:-host: myapp.zhixin.com # means to access the domain name It will be forwarded to the service on the pod managed by the backend myapp: http: paths:-path: backend: serviceName: myapp servicePort: 80 [root@master ingress] # kubectl apply-f ingress-myapp.yaml ingress.extensions/ingress-myapp created [root@master ingress] # kubectl get ingressNAME HOSTS ADDRESS PORTS AGEingress-myapp myapp.zhixin.com 808m [root@master ingress] # kubectl describe ingress [root@master ingress] # kubectl get pods-n ingress-nginx NAME READY STATUS RESTARTS AGEdefault-http-backend-6586bc58b6-qd9fk 1 Running 0 12hnginx-ingress-controller-6bd7c597cb-jlqzp 1 Running 3 12h

When you enter the ingress-controller interactive command line, you can clearly see how nginx proxies our myapp.zhixin.com in reverse:

[root@master ingress] # kubectl exec-n ingress-nginx-it nginx-ingress-controller-6bd7c597cb-jlqzp-/ bin/sh$ cat nginx.conf## start server myapp.zhixin.comserver {server_name myapp.zhixin.com; listen 80; set $proxy_upstream_name "-"; location / {set $namespace "default"; set $ingress_name "ingress-myapp"; set $service_name "myapp"; set $service_port "80";.

Test, let's resolve the myapp.zhixin.com domain name to node1 ip 172.16.1.101.

[root@master ingress] # curl myapp.zhixin.com:30080Hello MyApp | Version: V1 | Pod Name

Publish tomcat service via ingress (new example)

[root@master ingress] # cat tomcat-demo.yaml apiVersion: v1kind: Service# must be set to headless servicemetadata: name: tomcat namespace: defaultspec: selector: app: tomcat release: canary ports:-name: http targetPort: 8080 # this is the container port port: 8080 # this is service port-name: ajp targetPort: 8009 port: 8009---apiVersion: apps/v1kind: Deploymentmetadata: name: tomcat-deploy namespace: defaultspec: replicas: 2 selector: # tag selection The tag matching matchLabels: # is app: tomcat release: canary template: metadata: labels: app: tomcat # and the above myapp should match release: canary spec: containers:-name: tomcat image: tomcat:8.5.34-jre8-alpine # look for ports on https://hub.docker.com/r/library/tomcat/tags/: -name: http containerPort: 8080-name: ajp containerPort: 8009 [root@master ingress] # kubectl apply-f tomcat-demo.yaml service/tomcat createddeployment.apps/tomcat-deploy created [root@master ingress] # kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGEkubernetes ClusterIP 10.96.0.1 443/TCP 17dtomcat ClusterIP 10.109.76.87 8080/TCP 8009/TCP 1m [root@master ingress] # kubectl get podsNAME READY STATUS RESTARTS AGEtomcat-deploy-64c4d54df4-68sk8 1 7b58g 1 Running 0 51stomcat-deploy-64c4d54df4-7b58g 1 Running 0 51s [root@master ingress] # cat ingress-tomcat.yaml apiVersion: extensions/v1beta1 kind: Ingressmetadata: name: ingress-tomcat Namespace: default # should be in the same namespace as deployment and the service to be released annotations: # this note indicates that the ingress-controller we are going to use is nginx Instead of traefic,enjoy kubernetes.io/ingress.class: "nginx" spec: rules:-host: tomcat.zhixin.com # means to access the domain name It will be forwarded to the service on the pod managed by the backend myapp: http: paths:-path: backend: serviceName: tomcat servicePort: 8080 [root@master ingress] # kubectl apply-f ingress-tomcat.yaml ingress.extensions/ingress-myapp configured [root@master ingress] # kubectl get ingressNAME HOSTS ADDRESS PORTS AGEingress-tomcat tomcat.zhixin.com 8011s [root@master ingress] # kubectl describe ingress ingress-tomcatName: ingress-tomcatNamespace: defaultAddress: Default backend: default-http-backend:80 () Rules: Host Path Backends-tomcat.zhixin.com tomcat:8080 () Annotations: Kubernetes.io/ingress.class: nginxEvents: Type Reason Age From Message-Normal CREATE 1m nginx-ingress-controller Ingress default/ingress-tomcat

Parse tomcat.zhixin.com to node physical ip on node1 (mine is 172.16.1.101)

Test, you can see the tomcat welcome interface:

[root@master ingress] # curl tomcat.zhixin.com:30080

Access using https (new example)

1. First make a self-signed certificate (we will not demonstrate the example of CA here)

[root@master ingress] # openssl genrsa-out tls.key 2048 [root@master ingress] # openssl req-new-x509-key tls.key-out tls.crt-subj / C=CN/ST=Beijing/O=DevOps/CN=tomcat.zhixin.com

2. Inject the certificate into pod through secret.

[root@master ingress] # kubectl create secret tls tomcat-infress-secret-- cert=tls.crt-- key=tls.key secret/tomcat-infress-secret created [root@master ingress] # kubectl get secretNAME TYPE DATA AGEdefault-token-5r85r kubernetes.io/service-account-token 3 17dtomcat-ingress-secret kubernetes.io/tls 2 41s [root@master ingress] # kubectl describe secret tomcat-ingress-secretName: tomcat-ingress-secretNamespace: defaultLabels: Annotations: Type: kubernetes.io/tlsData====tls.crt: 1245 bytestls.key: 1679 bytes

3. Configure ingress in tls mode

[root@master ingress] # cat ingress-tomcat-tls.yaml apiVersion: extensions/v1beta1 kind: Ingressmetadata: name: ingress-tomcat-tls namespace: default # to be in the same namespace as deployment and the service to be released annotations: # this note indicates that the ingress-controller we are going to use is nginx, not traefic The name found by the enjoy kubernetes.io/ingress.class: "nginx" spec: tls:-hosts:-tomcat.zhixin.com secretName: tomcat-ingress-secret # kubectl get secret command rules:-host: tomcat.zhixin.com # indicates access to this domain name It will be forwarded to the service on the pod managed by the backend myapp: http: paths:-path: backend: serviceName: tomcat servicePort: 8080 [root@master ingress] # kubectl get ingressNAME HOSTS ADDRESS PORTS AGEingress-tomcat tomcat.zhixin.com 80 2hingress-tomcat-tls tomcat.zhixin.com 80 443 3m [root@master ingress] # kubectl describe ingress ingress-tomcat-tlsName: ingress-tomcat-tlsNamespace: defaultAddress: Default backend: default-http-backend:80 () TLS: tomcat-ingress-secret terminates tomcat.zhixin.comRules: Host Path Backends-tomcat.zhixin.com Tomcat:8080 () Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion": "extensions/v1beta1" "kind": "Ingress", "metadata": {"annotations": {"kubernetes.io/ingress.class": "nginx"}, "name": "ingress-tomcat-tls", "namespace": "default"}, "spec": {"rules": [{"host": "tomcat.zhixin.com", "http": {"paths": [{"backend": {"serviceName": "tomcat", "servicePort": 8080}, "path": null}]}] "tls": [{"hosts": ["tomcat.zhixin.com"], "secretName": "tomcat-ingress-secret"} kubernetes.io/ingress.class: nginxEvents: Type Reason Age From Message-Normal CREATE 4m nginx-ingress-controller Ingress default/ingress-tomcat-tls

4. Connect ingress-controller to view the configuration of nginx.conf.

[root@master ingress] # kubectl get pods-n ingress-nginxNAME READY STATUS RESTARTS AGEdefault-http-backend-6586bc58b6-qd9fk 1 + 1 Running 0 16hnginx-ingress-controller-6bd7c597cb-jlqzp 1 + 1 Running 3 16h [root@master ingress] # kubectl exec-n ingress-nginx-it nginx-ingress-controller-6bd7c597cb-jlqzp-/ bin / sh$ $cat nginx.conf # # start server tomcat.zhixin.com server {server_name tomcat.zhixin.com Listen 80; set $proxy_upstream_name "-"; listen 443 ssl http2

I see listen 443s.

5. Test https

[root@master ingress] # curl https://tomcat.zhixin.com:30443 deployment method 3 (using port 80 of ingress)

The first two deployment methods use node ip + non-80 ports to access services within the K8s cluster. However, in actual production, we prefer the way of node ip + 80 port to access the services in the K8s cluster. I think this method is the best, so let's introduce this method.

This part refers to the blog post http://blog.51cto.com/devingeng/2149377.

Download address

Https://github.com/kubernetes/ingress-nginx/archive/nginx-0.11.0.tar.gz

The ingress-nginx file is located in the deploy directory Function of each file: configmap.yaml: provide configmap configuration for nginx online default-backend.yaml: provide a default background error page 404namespace.yaml: create an independent namespace ingress-nginxrbac.yaml: create a corresponding role rolebinding for rbactcp-services-configmap.yaml: modify L4 load balancer configuration configmapudp-services-configmap.yaml: modify L4 load balancer configuration configmapwith-rbac.yaml: nginx-ingress-controller component with application rbac

Modify with-rbac.yaml

ApiVersion: extensions/v1beta1kind: Daemonsetmetadata: name: nginx-ingress-controller namespace: ingress-nginx spec: selector: matchLabels: app: ingress-nginx template: metadata: labels: app: ingress-nginx annotations: prometheus.io/port: '10254' prometheus.io/scrape: 'true' spec: serviceAccountName: nginx-ingress-serviceaccount hostNetwork: true containers: -name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.11.0 args:-/ nginx-ingress-controller-default-backend-service=$ (POD_NAMESPACE) / default-http-backend-configmap=$ (POD_NAMESPACE) / nginx-configuration-tcp-services-configmap=$ (POD_NAMESPACE ) / tcp-services-udp-services-configmap=$ (POD_NAMESPACE) / udp-services- annotations-prefix=nginx.ingress.kubernetes.io env:-name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name-name: POD_NAMESPACE valueFrom: FieldRef: fieldPath: metadata.namespace ports:-name: http containerPort: 80-name: https containerPort: 443livenessProbe: failureThreshold: 3 httpGet: path: / healthz port: 10254 scheme: HTTP initialDelaySeconds : 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 readinessProbe: failureThreshold: 3 httpGet: path: / healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 1 nodeSelector: custom/ingress-controller-ready: "true"

Areas that need to be modified:

Kind: DaemonSet: the official original file uses a deployment,replicate of 1, which will launch the corresponding nginx-ingress-controller pod on a node. External traffic is accessed to the node and the load is shared by the node to the internal service. In order to prevent a single point of failure, the test environment changes to DaemonSet and then deletes replicate, starts nginx-ingress-controller pod on established nodes in conjunction with affinity deployment, ensures that multiple nodes start nginx-ingress-controller pod, and then adds these nodes to external hardware load balance groups to achieve high availability.

HostNetwork: true: add this field to expose the service port of nginx-ingress-controller pod (80)

NodeSelector: add affinity deployment, and only nodes with custom/ingress-controller-ready tags will deploy the DaemonSet

Set up lable for nodes that need to deploy nginx-ingress-controller

Kubectl label nodes vmnode2 custom/ingress-controller-ready=truekubectl label nodes vmnode3 custom/ingress-controller-ready=truekubectl label nodes vmnode4 custom/ingress-controller-ready=true

Load yaml Fil

Kubectl apply-f namespace.yamlkubectl apply-f default-backend.yamlkubectl apply-f configmap.yamlkubectl apply-f tcp-services-configmap.yamlkubectl apply-f udp-services-configmap.yamlkubectl apply-f rbac.yamlkubectl apply-f with-rbac.yaml

Check whether the pod is created properly

# # it may be slow to download the image. Wait a while for all pod to be Running, then press Ctrl + c to exit.

[root@vmnode1 deploy] # kubectl get pods-- namespace=ingress-nginx-- watchNAME READY STATUS RESTARTS AGEdefault-http-backend-6c59748b9b-hc8q9 1 Running 0 6mnginx-ingress-controller-7fmlp 1 6mnginx-ingress-controller-7fmlp 1 Running 1 13dnginx-ingress-controller-j95fb 1 13dnginx-ingress-controller-j95fb 1 Running 1 13dnginx Ld2jw 1/1 Running 1 13d

Test ingress

Create a Service for tomcat

[root@k8s-master1 test] # cat mytomcat.yaml apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: mytomcatspec: replicas: 2 template: metadata: labels: run: mytomcatspec: containers:-name: mytomcat image: tomcat ports:-containerPort: 8080---apiVersion: v1kind: Servicemetadata: name: mytomcat labels: run: mytomcatspec: type: NodePort ports:-port: 8080 targetPort: 8080 selector: Run: mytomcat [root@k8s-master1 test] # kubectl apply-f mytomcat.yaml

Configure ingress forwarding files:

[root@k8s-master1 test] # cat test-ingress.yaml apiVersion: extensions/v1beta1kind: Ingressmetadata: name: test-ingress namespace: defaultspec: rules:-host: test.zhixin.com http: paths:-path: / backend: serviceName: mytomcat servicePort: 8080

Host: the corresponding domain name

Path: url context

Backend: forward to the corresponding serviceName: servicePort:

[root@k8s-master1 test] # kubectl apply-f test-ingress.yaml ingress.extensions/test-ingress created

Nginx-ingress-controller runs on both node1 and nod2 nodes. If there is a dns server in the network, map these two domain names to any node on which the nginx-ingress-controller is running in dns, and if there is no dns server, you can only modify the host file.

The normal practice is to install keepalive on the node1 and node2 nodes to generate a vip. Map the domain name to vip on dns.

I operate directly on the node1 node here:

Here the ip of the node1 node is 172.16.22.201 the ip of the Node2 node is 172.16.22.202

[root@k8s-master1 test] # echo "172.16.22.201 test.zhixin.com" > > / etc/hosts [root@k8s-master1 test] # echo "172.16.22.202 test.zhixin.com" > > / etc/hosts

Then access the test:

You can see that we bind the domain name test.zhixin.com to the ip of the Node node, and then we directly access the http://test.zhixin.com to access the pod service in the K8s cluster.

The above is all the content of the article "sample Analysis of ingress Resources and ingress controller in docker". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.