Shulou(Shulou.com)06/03 Report--

Author: DJ

Revision: Kevin Wang

New Innovation Direction of Container Biosphere

The year-end event of the container technology circle in 2018, North American KubeCon, has finally come to an end in Seattle. In total, the North American KubeCon attracted more than 8000 viewers, a record high. First put a picture to feel the popularity of the scene.

Friends who follow Kubernetes should already feel it, and in sharp contrast to the enthusiasm of the audience, this KubeCon sends a signal: there are fewer and fewer changes to the Kubernetes itself, and it is getting harder and harder for us to see those exciting features. It has become an established fact that Kubernetes is becoming "boring".

For the core of Kubernetes, "boredom" is not a bad thing, it is a sign of project maturity. Nowadays, Kubernetes has increasingly become the infrastructure of public cloud / enterprise IT systems, and extends to large-scale, complex scenarios, integrating more and more closely with AI, big data, IoT, vertical industries and other fields. Recently, more and more innovations around the Kubernetes biosphere are taking place in these areas. For example, Huawei's open source KubeEdge project, which was unveiled at KubeCon in North America, gave people a pleasant surprise and exclaimed, "Oh, it turns out Kubernetes can still play on edge computing!" While lamenting the powerful charm of Kubernetes, I also have to admire the creativity of Huawei engineers-you know, edge computing and cloud computing scenarios are very different, using K8S edge computing, before more just stay in the discussion stage, the practice process of the pit is too numerous, Huawei unexpectedly did it, but also open source!

K8S IoT Edge Working Group

2018 is not only a year of universal praise for containers and Kubernetes, but also a year for edge computing to accelerate the landing and application of the industry. The "edge" of edge computing refers to all kinds of end devices that are ubiquitous in our daily life and affect our food, clothing, housing and transportation, such as mobile phones, cameras, routers, self-driving cars, industrial robots and so on. Compared with the centralized computing model, edge computing can provide enterprises / users with low latency and high performance services, ensure the localization and security of data, and solve the problems of local data artificial intelligence. However, the heterogeneity of the Internet of things and the long life cycle of industrial system assets bring severe challenges to the management of edge devices. Gartner directly lists edge computing as the key technology direction for the next decade.

The container-centric IT infrastructure revolution led by Kubernetes is in the ascendant, and someone is already planning the next decade. Now that Kubernetes has become the de facto standard for containerized orchestration and continues to work magic in the cloud, an intuitive idea is whether Kubernetes can bring this magic to marginal computing scenarios.

However, when Kubernetes really involves edge-side deployment, there are still many challenges, such as:

1. There may not be enough resources on the edge side to run a complete Kubelet

2. Can local autonomy be realized when the network between the edge node and the cloud is unstable or even completely impassable?

3. Communication between edge side nodes

4. How to manage multi-tenant edge resources, including devices, in the cloud

5. Serverless is not supported on the edge side, such as functions.

To address these challenges, Huawei, Google, Red Hat and VMWare jointly launched a new working group in the Kubernetes community: IoT Edge Working Group, which aims to improve the ability to deploy Kubernetes in marginal scenarios.

On the third day of KubeCon, four engineers from Huawei, Red Hat, Cisco and Oort held a round table on the technical challenges and future trends of edge computing.

Cindy Xing, an architect from Huawei (co-chair of the Kubernetes Community IoT Edge working Group), talked about the relationship between cloud computing, containers and edge computing-unlike cloud computing, edge computing means hundreds of thousands of edge nodes and millions or even hundreds of millions of end devices in different geographic locations. The scale and geographical span of edge computing far exceed that of the data center. The bandwidth that the cloud can provide will not be able to meet the needs of the edge. At the same time, the connection between the edge and the cloud is unstable. The edge side needs to achieve autonomy in the case of disconnection from the cloud, and at the same time needs to connect with the cloud to achieve centralized management and batch application deployment management. If you deploy update applications to hundreds of thousands of edge nodes, this consumption is unimaginable. In addition, in edge computing, the devices are extremely diverse, and the communication protocols of the devices are also very different. The device can be a Raspberry-Pi of 128MB or 1CPU, or a multicore server with GB memory. The device communication protocol can be either Http/Amqp/Mqtt, Bluetooth,Zigbee, etc. Edge computing will be much more complex than cloud computing. For developers, after an application is developed and verified in the office, it will want the application to be deployed directly on the edge without worrying about functional problems. The portability of the container meets this need.

It is precisely because of the natural convergence between K8S and edge computing that Huawei has developed and opened up KubeEdge-- 's first Kubernetes-based edge computing platform for multi-rent, scalable and K8S API-compatible container applications. KubeEdge is also a reference example of K8S IoT Edge Working Group output using Kubernetes to support edge computing.

The combination of KubeEdge:K8S and Edge Computing

On the fourth day of KubeCon, Huawei organized a small party for community members of CNCF IOT/Edge.

At this casual gathering, members of the K8S/CNCF community sat down, chatted, and shared their needs and technical understanding of the scenarios of edge computing. Huawei engineers demonstrated IEF, Huawei's cloud edge computing product. In the following KubeCon CNCF IOT/Edge Deep Dive,co-chair Cindy Xing, the architecture, function and future development roadmap of IEF's open source implementation KubeEdge are introduced in depth.

For the pain points in the edge computing scenario, KubeEdge provides users with the basic network infrastructure, and KubeBus can be used to realize the network routing communication between the edge and the cloud, the edge and the edge. With the help of the MetaStore of cloud and edge and the synchronization between them, KubeEdge solves the problem of network disconnection and bandwidth of cloud edge and realizes the autonomy of edge side. In KubeEdge, AppEngine is a lightweight agent, and 10MB's runtime memory consumption means it can run on almost any device. AppEngine maintains the data model and communication protocol of K8S, manages the life cycle of container applications and feedback the running status of container applications by interacting with container engine.

Functionally, KubeEdge opens up the entire software and hardware stack from the underlying device to the device driver / SDK, to the edge side Runtime, cloud controller and cloud applications. It brings a solution and platform for IOT manufacturers, developers and IOT application enterprises.

From the perspective of architecture, KubeEdge includes two sets of components: cloud side and edge side. The edge side of the device supports the MQTT protocol and adopts a modular design, which is composed of five modules of Edged,MetaManager,DeviceTwin,EventBus,EdgeHub. The modules communicate with each other through the channel of golang.

 Edged is a compact AppEngine that is deeply customized for edge computing. Although concise, this AppEngine supports K8S API primitives such as Pod,Volume,Configmap, as well as Pod probe and Event reporting.

 EdgeHub is a client of web socket, which is responsible for message communication with the cloud, including synchronizing cloud resource updates to the edge side and reporting edge node and device status updates to the cloud. This data channel is the KubeBus mentioned above.

 MetaManager is a message processor that acts as a bridge between edged and edgehub. It also interacts with the back-end data store to read and write some metadata.

 EventBus is the link between edge nodes and devices. It can not only subscribe to device status update events from MQTT Broker and publish them to other interested components, but also send operation instructions to MQTT Broker. At the same time, app on cloud communicates with users' own applications deployed on edge, and also uses EventBus.

 DeviceTwin is responsible for storing device metadata to data store and synchronizing device status with the cloud. Users can issue device operation instructions to DeviceTwin from the cloud. KubeEdge supports the MQTT protocol for devices on the edge side and provides device shadow services. Devices can currently be registered through MQTT Broker (mosquitto).

In Cindy's view, the architecture of KubeEdge achieves the loose coupling of cloud and edge, and the edge can be autonomous under the condition of network disconnection, and synchronize with the cloud side after the network is restored. KubeEdge introduces the powerful container editing capability of K8S, and customers can manage edge nodes and deploy applications as easily as K8S clusters on the cloud, which is undoubtedly a boon for users who are familiar with K8S.

Future work

KubeEdge currently includes a complete implementation of the edge side, and the open source work on the cloud side is under intensive planning. I believe it will also be open source soon. For more information, please see the project address: https://github.com/kubeedge/kubeedge.

Since KubeCon China announced open source in November, KubeEdge has attracted wide attention and once appeared on the Github trending list. At present, star 220 has attracted 50 users and developers, and has absorbed several external code contributors.

In the future, Huawei will continue to improve KubeEdge, including, but not limited to:

Support more device access protocols: AMQP, Bluetooth, ZigBee, etc.

Support for very large-scale node and device management

Combine with Istio to build service grid capability

Provide functional services on the edge side

Support devices to connect directly to the cloud through the MQTT protocol

Edge nodes use UDP hop to communicate

Intelligent scheduling function is introduced.

Huawei also welcomes any interested vendors and developers to participate in the KubeEdge community to jointly promote innovation in edge computing. We expect marginal computing to develop by leaps and bounds in the next year or two, and we have every reason to believe that KubeEdge, which has a first-mover advantage, will create a new ecology.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.