Shulou(Shulou.com)06/03 Report--

Introduction to kubernetes Service exposure

So far, kubernetes has a total of three ways to expose services:

LoadBlancer ServiceNodePort ServiceIngressLoadBlancer Service

LoadBlancer Service is a component of kubernetes combined with cloud platform, such as foreign GCE,AWS, domestic Aliyun and so on. Using the underlying cloud platform used by it to create a load balancer is convenient for clusters using cloud platform, but it has limitations and high cost.

NodePort Service

When we exposed the service in our previous blog post, we all used nodeport, which is essentially achieved by exposing a port on each node of the cluster, and then mapping this port to a specific service, which is more intuitive and convenient. Although there are many ports for each node (0,065535), it is messy because of security and ease of use. There is also the problem of port conflicts) and there are certain risks to the security of the host (intranet environment, the problem is small), so the actual use is not much, of course, for small-scale cluster services, it is quite good.

The above two services do not seem to be ideal in all aspects, so in the case, we will implement them through Ingress objects (secure and convenient for unified management).

Ingress Overview 1, what is Ingress?

In the kubernetes cluster, we know that the ip of service and pod is accessed only within the cluster. If the external application wants to access the services within the cluster, the requests outside the cluster need to be forwarded to the NodePort exposed by service on Node through load balancing, and then forwarded to the relevant pod by the kube-proxy component.

Ingress is a collection of routing rules for requests to enter the cluster. The popular point is to provide external access to the cluster and forward external HTTP or HTTPS requests to the internal service of the cluster.

As shown in the flow chart above, the Ingress proxy is not pod's service, but pod. The reason why service is configured when configuring is to obtain all pod information through service.

.

Second, Ingress-nginx composition

Ingress-nginx generally consists of three components:

Reverse proxy load balancer Ingress ControllerIngress1) reverse proxy load balancer: usually runs in the port mode of service, receives and forwards according to the rules defined by ingress, such as nginx,Haproxy,Traefik, etc. Nginx is used in this article. 2) ingress-nginx-Controller: listen to the APIServer, dynamically change the configuration file of the ingress service according to the ingress rules written by the user (write the yaml file of the nginx), and the reload overload makes it effective, this process is automated (through the lua script). 3) Ingress: abstract the configuration of nginx into an Ingress object. When users add a new service, they only need to write a new yaml file of ingress. Third, the working principle of Ingress-nginx: 1) ingress controller dynamically perceives the changes of ingress rules in the cluster by interacting with kubernetes api. 2) then read it. According to the custom rules, the rule is to specify which service the domain name corresponds to, and generate a nginx configuration. 3) in the pod written to nginx-ingress-controller, there is a Nginx service running in the pod of this Ingress controller, and the controller will write the generated nginx configuration to the / etc/nginx.conf file. 4) then reload to make the configuration effective, thus achieving the problem of allocation and dynamic update. Fourth, what problems does ingress-nginx solve in the production environment?

1) dynamically configure the service:

If we follow the traditional way, when we add a new service, we may need to add a reverse proxy at the traffic entry to point to our new service, but with ingress, we only need to configure the ingress, and when the service starts, it will automatically register into the ingress without additional operation.

2) reduce unnecessary Port exposure (secure, ports easy to manage)

We know that when deploying K8s, the firewall needs to be disabled, mainly because many services of K8s will be mapped out in nodeport mode, which is very insecure to the host. Ingress can avoid this problem. You only need to map out the ingress services to proxy all the back-end services, while the back-end services do not need to be mapped out.

Deploy Ingress-nginx

This article uses the following configuration examples to practice ingress-nginx:

In this article, you can download all the docker mirror packages used in ingress-nginx from my GitHub. Download link: https://github.com/sqm-sys/Ingress-nginx/releases/tag/v1

First, prepare before deploying ingress-nginx:

1. Build a private repository (registry), and push the test image (apache,tomcat) to the repository.

# run registry private repository: [root@master ~] # docker run-d-- name registry-- restart=always-p 5000name registry 5000-v / data/registry:/var/lib/registry registry# modify docker configuration file: [root@master ~] # vim / usr/lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd-H unix://-- insecure-registry 172.16.1.30 name registry 500 copies the modified docker configuration file to the cluster In other nodes: [root@master ~] # scp / usr/lib/systemd/system/docker.service node01:/usr/lib/systemd/system/ [root@master ~] # scp / usr/lib/systemd/system/docker.service node02:/usr/lib/systemd/system/# cluster all hosts reload process And restart the docker service: [root@master ~] # systemctl daemon-reload [root@master ~] # systemctl restart docker#push test image to private repository: [root@master ~] # docker push 172.16.1.30:5000/apache:v1 [root@master ~] # docker push 172.16.1.30:5000/tomcat:v1

2, create namespace

# create a yaml file:

ApiVersion: v1kind: Namespacemetadata: name: ingress-test# create and view namespace: [root @ master ~] # kubectl create-f ns.yaml namespace/ingress-nginx created [root@master ~] # kubectl get nsNAME STATUS AGEdefault Active 92dingress-test Active 4s

3. Create deployment and service resources for testing

1) create httpd and service resources:

[root@master ~] # mkdir ingress-nginx [root@master ~] # cd ingress-nginx/ [root@master ingress-nginx] # vim httpd.yamlapiVersion: extensions/v1beta1kind: Deploymentmetadata: name: ingress-testspec: replicas: 3 template: metadata: labels: name: httpd spec: containers:-name: httpd image: 172.16.1.30:5000/apache:v1 # Images are performed from the private repository Pull-apiVersion: v1kind: Servicemetadata: name: httpd-svc namespace: ingress-testspec: type: NodePort selector: name: httpd ports:-protocol: TCP port: 80 targetPort: 80 nodePort: 32134

2) create tomcat and service resources:

[root@master ingress-nginx] # vim tomcat.yaml apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: tomcat namespace: ingress-testspec: replicas: 3 template: metadata: labels: name: tomcat spec: containers:-name: tomcat image: 172.16.1.30:5000/tomcat:v1---apiVersion: v1kind: Servicemetadata: name: tomcat-svc namespace: ingress-testspec: type: NodePort selector: name: tomcat ports: -protocol: TCP port: 8080 targetPort: 8080 nodePort: 32135

# create the above two applications and check whether they have been created successfully:

[root@master ingress-nginx] # kubectl create-f httpd.yaml deployment.extensions/httpd createdservice/httpd-svc create [root@master ingress-nginx] # kubectl create-f tomcat.yaml deployment.extensions/tomcat createdservice/tomcat-svc created

3) Test whether the external can access the internal application of the cluster through nodeport:

You can see that there is no problem with exposing through nodeport, but this approach has great shortcomings in large-scale cluster services, so it is next implemented through ingress-nginx.

Second, create an ingress-nginx

1) download the yaml file on GitHub: https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md

After entering, the following page is displayed:

# Note: the complete command on the web page is to execute the yaml file directly. Let's not execute it yet, but download the yaml file to the local host. (just copy the command)

[root@master ingress-nginx] # wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.28.0/deploy/static/mandatory.yaml[root@master ingress-nginx] # lshttpd.yaml mandatory.yaml tomcat.yaml

2) modify the yaml file:

[root@master ingress-nginx] # vim mandatory.yaml

# add the following fields:

HostNetwork: true

If you use this network parameter, applications running in pod can directly use the node node port so that other hosts on the network where the node node host resides can access the application.

NodeSelector:

Set the node tag selector to specify which node to run on. (just keep the default and let it choose. Of course, you can also customize the tag and choose to run the corresponding node)

.

The version of the above mirror package may be updated frequently. I use the latest version 0.28.0. You can download it according to the version you want to use.

Note: because the download address specified in the default yaml file is a foreign image, and due to the limitations of the domestic network environment, we cannot download the image directly from the mirror station of google, so you can download it in the following way: [root@master ingress-nginx] # docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.28.0

3) Import image package-Import nginx-ingress-controller.0.28.0 image package to each node in the cluster:

[root@node01 ~] # docker load < nginx-ingress-controller.0.28.0.tar [root@node01 ~] # docker images | grep nginx-ingressregistry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller 0.28.0 61de39f77f45 7 days ago 305MB

/ / execute the yaml file on master:

/ / check whether ingress-nginx pod is running: [root@master ingress-nginx] # kubectl get pod-n ingress-nginx-o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-ingress-controller-685985b5b8-8mzjc 1 Running 0 43s 172.16.1.32 node02

4) create ingress rules (YAML file)

The yaml file is as follows:

[root@master ingress-nginx] # vim ingress-rule.yamlapiVersion: extensions/v1beta1kind: Ingressmetadata: name: test-ingress namespace: ingress-test annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules:-host: ingress.test.com # backend applications access the web page root directory of http: paths:-path: / # apache through this domain name: / backend: ServiceName: httpd-svc servicePort: 80-path: / tomcat # tomcat webpage change directory: / tomcat backend: serviceName: tomcat-svc servicePort: 8080

This rule is created to proxy the apache and tomcat services at the back end, respectively.

/ / create an ingress and view the binding information: [root@master ingress-nginx] # kubectl create-f ingress-rule.yaml ingress.extensions/test-ingress created [root@master ingress-nginx] # kubectl get ingresses. -n ingress-test NAME HOSTS ADDRESS PORTS AGEtest-ingress ingress.test.com 80 17s

# View describe to view details (make sure the backend is successfully bound to get apache and tomcat services)

.

# # enter the ingress container to view the configuration file of nginx:

[root@master ingress-nginx] # kubectl exec-it-n ingress-nginx nginx-ingress-controller-685985b5b8-8mzjc / bin/sh/etc/nginx $cat nginx.conf

By looking at the nginx configuration file, we can see that ingress-controller will dynamically write the nginx configuration to the nginx configuration file and make its reload effective according to the ingress rules we wrote (that is, the proxy backend application and will generate a section of nginx configuration).

5) access services in the cluster through domain name

# # currently we need to know which node ingress runs on: [root@master ingress-nginx] # kubectl get pod-o wide-n ingress-nginx NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-ingress-controller-685985b5b8-8mzjc 1 Running 031m 172.16.1.32 node02

We have not built a DNS server, so we need to add the ip address of node02 to the hosts file on the windows host to do domain name resolution:

Modify the path: C:\ Windows\ System32\ drivers\ etc\ hosts (permission to modify is required) 172.16.1.32 ingress.test.com # add this item

# # accessing httpd:

# # accessing tomcat:

6) create a service resource object for ingress:

Although ingress has been able to proxy backend applications above, we can find that domain name resolution can only be done by specifying the address of the running node (accessing web pages), but not through other nodes in the cluster. If the node dies, ingress will not be able to proxy backend applications, so we need to create service resources for ingress.

.

The YAML file is available for us on GitHub:

# # We download it locally:

[root@master ingress-nginx] # wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.28.0/deploy/static/provider/baremetal/service-nodeport.yaml[root@master ingress-nginx] # cat service-nodeport.yaml # We don't need to modify it Just take a look at apiVersion: v1kind: Servicemetadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginxspec: type: NodePort ports:-name: http port: 80 targetPort: 80 protocol: TCP-name: https port: 443 targetPort: 443 protocol: TCP selector: app.kubernetes.io/name: ingress-nginx App.kubernetes.io/part-of: ingress-nginx

/ / We create the service resource object:

[root@master ingress-nginx] # kubectl create-f service-nodeport.yaml service/ingress-nginx created [root@master ingress-nginx] # kubectl get svc-n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGEingress-nginx NodePort 10.101.206.247 80purl 30842 then TCPMagi 44332550lapTCP 20s

# you can see that nodeport maps the random port of http and https to 30842 Universe 32550 for us. Through this port, we can specify that the address of any node in the cluster can proxy the backend application. Even if a node in the cluster goes down, it will not affect external access through the domain name.

# # accessing apache:

# # accessing tomcat

.

At this time, in the host file in the windows host, you can bind the ip address of any node in the cluster to parse the web page (you can test it yourself), thus solving the problem we were worried about before. compared with the previous nodeport method, it is proved that ingress can solve unnecessary port exposure. only need to map the ingress port, you can proxy all the applications at the back end, thus meeting our needs.

.

Third, ingress based on virtual host

The purpose of building a virtual host: to access the same web interface with different domain names

This article is only based on the above httpd service, for which we bind two different domain names to access:

# # modify ingress rule file (the complete yaml configuration is as follows)

ApiVersion: extensions/v1beta1kind: Ingressmetadata: name: test-ingress namespace: ingress-test annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: rules:-host: ingress.test.com http: paths:-path: / backend: serviceName: httpd-svc servicePort: 80-path: / tomcat backend: serviceName: tomcat-svc servicePort: 8080-host: ingress.test2.com # just need to add another host field under the rules field Only httpd is implemented here, and tomcat can be added if there is a need. Http: paths:-path: / backend: serviceName: httpd-svc # Note: all virtual hosts in serviceName must be consistent servicePort: 80

/ / create a new virtual host and view:

[root@master ingress-nginx] # kubectl apply-f ingress-rule.yaml ingress.extensions/test-ingress configured [root@master ingress-nginx] # kubectl get ingresses. -n ingress-test NAME HOSTS ADDRESS PORTS AGEtest-ingress ingress.test.com,ingress.test2.com 10.101.206.247 80 83m

/ / access httpd web pages through different domain names:

Since we do not have a dns server, we need to bind the new domain name in the windows hosts file (the address can be bound to any node in the cluster): 172.16.1.31 ingress.test.com172.16.1.32 ingress.test2.com

So far, a number of different domain names can access back-end applications at the same time.

.

Fourth, Ingress based on HTTPS (port 443)

After deploying ingress, we do not have to issue a certificate for all the backend pod as usual, we only need to issue a certificate for the domain name of the ingress proxy.

1) create a CA certificate:

[root@master HTTPS] # openssl req-x509-sha256-nodes-days 365newkey rsa:2048-keyout tls.key-out tls.crt-subj "/ CN=nginxsvc/O=nginxsvc" # parameters can be customized to Generating a 2048 bit RSA private key..+++..+++writing new private key to 'tls.key'- as needed

# after the creation is completed, the key and certificate files will be generated in the current directory:

[root@master HTTPS] # lstls.crt tls.key

2) create deployment,service,ingress resources: (practice with nginx services)

[root@master HTTPS] # vim nginx-ingress2.yaml

# the complete yaml file is as follows:

ApiVersion: extensions/v1beta1 # create deploymentkind: Deploymentmetadata: name: web namespace: ingress-testspec: template: metadata: labels: name: test-web spec: containers:-name: web image: 172.16.1.30:5000/nginx:v1 # Image in the private repository (provided)-- apiVersion: v1 # create service Associate the above deploymentkind: Servicemetadata: name: web-svc namespace: ingress-testspec: selector: name: test-web ports:-protocol: TCP port: 80 targetPort: 80---apiVersion: extensions/v1beta1 # create ingress rules kind: Ingressmetadata: name: test-ingress2 namespace: ingress-testspec: tls: # issue certificates for domain names-hosts:-ingress.nginx.com secretName: tls-secret rules:-host: Ingress.nginx.com http: # Note The field here is http, and https paths:-path: / backend: serviceName: web-svc servicePort: 80 is not supported # create a https-based ingress service (nginx): [root@master HTTPS] # kubectl create-f nginx-ingress2.yaml deployment.extensions/web createdservice/web-svc createdingress.extensions/test-ingress2 created

# # View ingress resources and mapped service ports:

[root@master HTTPS] # kubectl get pod-n ingress-test | grep webweb-74cf864c58-rnvpx 1 Running 0 24s [root@master HTTPS] # kubectl get ingresses. -n ingress-test NAME HOSTS ADDRESS PORTS AGEtest-ingress ingress.test.com,ingress.test2.com 10.101.206.247 80 127mtest-ingress2 ingress.nginx.com 10.101.206.247 80 443 5m32s# can see that the newly created ingress provides ports 80 and 443 / the port to view the service mapping of ingress [root@master HTTPS] # kubectl get svc-n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT (S) AGEingress-nginx NodePort 10.101.206.247 80RU 30842Think TCP 443 32550andTCP 86m

3) access the nginx service through port 443 of the ingress proxy:

# bind domain name in hosts file: 172.16.1.32 ingress.nginx.com

# visit the website: https://ingress.nginx.com:32550/

The HTTPS-based ingress service is successfully implemented, and the ingress-nginx deployment is complete.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.