Shulou(Shulou.com)05/31 Report--

This article introduces the relevant knowledge of "what is Istio Pilot code?". In the operation of actual cases, many people will encounter such a dilemma. Then let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

Introduction of Istio Pilot components

In the Istio architecture, the Pilot component is the core component, which is responsible for the traffic management in the service grid and the configuration and distribution between the control plane and the data plane. The internal code structure of Pilot is quite complex. In this paper, we will understand the principle of Pilot implementation through an in-depth analysis of the code of Pilot.

First of all, let's take a look at the functional positioning of Pilot in Istio. Pilot converts the service information and configuration data into the standard data structure of the xDS interface and distributes it to the Envoy on the data side through gRPC. If you think of Pilot as a black box for processing data, it has two inputs and one output:

At present, the input of Pilot includes two parts of data sources:

Service data: derived from various service registries (Service Registry), such as services in Service,Consul Catalog registered in Kubernetes, etc.

Configuration rules: various configuration rules, including routing rules and traffic management rules, are defined by Kubernetes CRD (Custom Resources Definition) and stored in Kubernetes.

The output of the Pilot is the data plane configuration data that conforms to the xDS interface, and the configuration data is pushed to the Envoy of the data plane through the gRPC Streaming interface.

Note: the Istio code base is constantly changing and updating. The code commit based on this analysis is: d539abe00c2599d80c6d64296f78d3bb8ab4b033

Pilot-Discovery code structure

The code of Istio Pilot is divided into Pilot-Discovery and Pilot-Agent, in which Pilot-Agent is used to manage the life cycle of Envoy on the data side, and Pilot-Discovery is the component of traffic management on the control plane. This paper will focus on the code of the control plane, that is, Pilot-Discovery.

The entry function of Pilot-Discovery is the main method in pilot/cmd/pilot-discovery/main.go. The Discovery Server,Discovery Server is created in the main method, which mainly contains three parts of logic:

Config Controller

Config Controller is used to manage a variety of configuration data, including user-created traffic management rules and policies. Istio currently supports three types of Config Controller:

Kubernetes: use Kubernetes as the storage of configuration data, which is directly attached to Kubernetes's powerful CRD mechanism to store configuration data. It is simple and convenient, and is the first configuration storage scheme used by Istio.

MCP (Mesh Configuration Protocol): the use of Kubernetes to store configuration data leads to the coupling of Istio and Kubernetes, limiting the use of Istio in non-Kubernetes environments. In order to solve this coupling, the Istio community proposed that MCP,MCP defines a standard protocol for sending configuration data to the Istio control plane, Istio Pilot as MCP Client, any Server that implements the MCP protocol can send the configuration to Pilot through the MCP protocol, thus uncoupling Istio and Kubernetes. If you want to learn more about MCP, please refer to the link at the end of the article.

Memory: an in-memory Config Controller implementation, mainly for testing.

The current configuration of Istio includes:

Virtual Service: defines traffic routing rules.

Destination Rule: defines traffic handling rules related to a service or subset, including load balancing policy, connection pool size, circuit breaker settings, subset definition, and so on.

Gateway: defines the service for which the portal gateway is exposed.

Service Entry: an external service can be manually added to the service grid by defining a Service Entry.

Envoy Filter: add a custom Filter to the configuration of Envoy through Pilot.

Service Controller

Service Controller is used to manage various Service Registry and propose service discovery data. Currently, Service Registry supported by Istio includes:

Kubernetes: interfacing with Kubernetes Registry, the Service and Instance defined in Kubernetes can be collected into Istio.

Consul: interface with Consul Catalog and collect the Service defined in Consul into Istio.

MCP: similar to MCP config controller, get Service and Service Instance from MCP Server.

Memory: an in-memory Service Controller implementation, mainly for testing.

Discovery Service

The Discovery Service mainly contains the following logic:

Start gRPC Server and receive a connection request from the invoke side.

Receive the xDS request on the invoke side, get the configuration and service information from Config Controller and Service Controller, generate a response message and send it to Envoy.

Listen for configuration change messages from Config Controller and service change messages from Service Controller, and push configuration and service changes to Envoy through the xDS interface. (note: at present, Pilot does not implement incremental change push, and each change pushes full configuration. There may be performance problems when there are many services in the grid.

Pilot-Discovery business process

Pilot-Disocvery includes the following major business processes:

Initialize the major components of Pilot-Discovery

The entry to the Pilot-Discovery command is the main method in pilot/cmd/pilot-discovery/main.go, in which the Pilot Server,Server code is created in the file pilot/pkg/bootstrap/server.go. Server mainly does the following initialization work:

Create and initialize the Config Controller.

Create and initialize the Service Controller.

Create and initialize HTTP Discovery Server based on Envoy V1 API and GPRC Discovery Server based on Envoy V2 API in Discovery Server,Pilot. Because V1 has been abandoned, this paper will mainly analyze the gRPC Discovery Server of V2 API.

Register Discovery Server as Event Handler for Config Controller and Service Controller to listen for configuration and service change messages.

Create gRPC Server and receive connection requests from Envoy

Pilot Server creates a gRPC Server to listen for and receive xDS requests from Envoy. The DiscoveryServer.StreamAggregatedResources method in pilot/pkg/proxy/envoy/v2/ads.go is registered as the service processing method of gRPC Server.

When gRPC Server receives a connection from Envoy, it will call the DiscoveryServer.StreamAggregatedResources method, create a XdsConnection object in this method, and open a goroutine to receive the client's xDS request from the connection and process it. If the configuration of the control plane changes, the Pilot will actively push the configuration change to the connection.

Push updates to Envoy after configuration changes

This is the most complex business process in Pilot, mainly because multiple channel and queue are used in the code to merge and forward change messages. The business process is as follows:

Config Controller or Service Controller notifies Discovery Server,Discovery Server through the callback method to put the change message into the Push Channel when the configuration or service changes.

Discovery Server receives change messages from the Push Channel through a goroutine and merges the change messages that occur continuously over a period of time. If there is no new change message beyond the specified time, the merged message is added to a queue Push Queue.

Another goroutine takes the change message from the Push Queue, generates a XdsEvent, and sends it to the Push Channel of each client connection.

In the DiscoveryServer.StreamAggregatedResources method, take the XdsEvent from the Push Channel, then generate a DiscoveryResponse that conforms to the xDS interface specification according to the context, and push it to the invoke side through gRPC. (gRPC allocates a separate goroutine for each client connection for processing, so the StreamAggregatedResources processing methods for different client connections are handled in different goroutine.)

Respond to the xDS request initiated by Envoy

A two-way Streaming gRPC service call is established between Pilot and Envoy, so Pilot can push to Envoy when the configuration changes, and Envoy can also initiate a xDS call request to obtain the configuration. The process for Envoy to initiate a xDS request is as follows:

Envoy sends a DiscoveryRequest over a created gRPC connection

Discovery Server receives the DiscoveryRequest from Envoy from XdsConnection through a goroutine and sends the request to ReqChannel

The other goroutine of the Discovery Server receives the DiscoveryRequest from the ReqChannel, generates a DiscoveryResponse that conforms to the xDS interface specification based on the context, and returns it to the Envoy.

Key code snippets of Discovery Server business processing

The following is the key code snippet of Discovery Server and the corresponding business logic comments. For ease of reading, only the logic trunk is left in the code, and some unimportant details are removed.

Key code for handling xDS requests and pushes

This part of the key code is located in the StreamAggregatedResources method of the istio.io/istio/pilot/pkg/proxy/envoy/v2/ads.go file. The StreamAggregatedResources method is registered as the handler of gRPC Server, and for each client connection, gRPC Server launches a goroutine for processing.

The code mainly contains the following business logic:

Receive the xDS request from the Envoy from the gRPC connection and put it into a channel reqChannel.

Receive a xDS request from reqChannel, construct a response based on the type of xDS request, and send it to Envoy.

Receive the notification after the change of Service or Config from the pushChannel of connection, construct the xDS response message, and push the content of the change to the invoke side.

/ StreamAggregatedResources implements the ADS interface.func (s * DiscoveryServer) StreamAggregatedResources (stream ads.AggregatedDiscoveryService_StreamAggregatedResourcesServer) error {. / / create a goroutine to receive the xDS request from Envoy, and put the request in reqChannel con: = newXdsConnection (peerAddr, stream) reqChannel: = make (chan * xdsapi.DiscoveryRequest, 1) go receiveThread (con, reqChannel, & receiveError). For {select {/ / receives the xDS request case discReq initiated by the Envoy side from reqChannel, ok: =

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.