Shulou(Shulou.com)06/02 Report--

On June 20, 2019, the third Enterprise Container Innovation Conference (Enterprise Container Innovation Conference, hereinafter referred to as ECIC) hosted by Rancher Labs (hereinafter referred to as Rancher) was held at the Sheraton Hotel in Beijing. This year's ECIC has a large scale, with a total of 17 keynote speeches throughout the day, attracting nearly 1,000 container technology enthusiasts, and more than 10000 viewers watched the event on the live broadcast platform online.

Technical leaders from more than a dozen enterprises, including Rancher, Aliyun, Baidu Yun, Ping an Technology, China Unicom, Flying loan Financial Technology, China Life Insurance, SmartX, Huatai Insurance, Xiamen Airlines, JFrog, New Oriental, Cisco and so on, attended this ECIC, bringing a wonderful sharing of practical experience on corporate container projects and a sharing of corporate containerization experiences for container technology enthusiasts participating in the conference.

Cdn.xitu.io/2019/7/4/16bbafa6869063e9?w=1920&h=1280&f=jpeg&s=268490 ">

At the scene of the conference, Wang Zhijun, general manager of China Unicom data center, brought the content sharing with the theme of "China Unicom containerized big data cloud platform exploration and practice" to container enthusiasts on the spot.

China Unicom is one of the three major domestic operators, and it is also one of the first domestic enterprises to deploy big data platform on container cloud. On China Unicom's development and exploration on the containerized big data cloud platform, Wang Zhijun shared: "through research, exploration and practice, we found that the technical route of Kubernetes+Docker is more in line with the actual needs of Unicom. It supports almost all types of container business, and it is based on Unicom's technology selection. We have introduced Rancher product deployment and Kubernetes cluster management functions. To provide stronger container technology and container service support for Unicom's containerized big data cloud platform. "

The following is a transcript of the speech delivered by Wang Zhijun, General Manager of China Unicom data Center:

Hello, everyone. Thank you very much for Rancher inviting us to give a speech at the Enterprise Container Innovation Conference. The topic of my speech today is "China Unicom containerized big data Cloud platform Exploration and practice". It is about how China Unicom connects big data with the containerized cloud platform.

Construction background

Let's briefly review the development of China Unicom big data and cloud computing. Big data and cloud computing belong to two different fields. Big data is mainly concerned with how to centralize the data and mine the value of the data. Cloud computing is mainly concerned with how to use resources more efficiently and improve the efficiency of resource utilization. When big data develops to a certain stage, it will encounter cloud computing unexpectedly.

In big data's side, there are several landmark events and processes, one is the emergence of Hadoop in 2006, the other is the emergence of CDH release in 2009, to 2012, big data appeared a new way of resource scheduling management, streaming computing technologies such as Spark and Flink.

The landmark event of cloud computing began when Amazon put forward EC2 in 2006. The emergence of EC2 marks the beginning of the era of cloud computing. In 2010, OpenStack appeared, which is a technology widely used in the deployment of private cloud. 2013 is the first year of Docker, which makes container technology popular in the field of cloud computing. The emergence of Kubernetes in 2014 has turned Container as a Service into a new concept widely accepted by the industry. A large number of architectures that we used to deploy single applications or distributed applications on virtual machines have gradually become container-based micro-service processing.

The ABC fusion we proposed refers to the integration of AI+Bigdate+Cloud.

In the era of Bigdate 2.0, the emergence of Hadoop commercial version provides a better way for everyone to use Hadoop to deal with big data. On the other hand, SQL on Hadoop has gradually matured. In our view, SQL on Hadoop is a language closer to human nature for data processing, it and our relational database is not a very close coupling relationship, a large number of our real-time processing is based on SQL on Hadoop. The third point is that at the beginning, when we did big data, a large number of batch processing methods were used, and now we are more likely to use the combination of loss processing and batch processing and interaction is a combination of analysis.

In the Bigdate 3.0 era, the big data cloud and AI have become one, and customers want to provide AI, Bigdate and Cloud on a unified platform.

China Unicom is the enterprise that achieves data centralization in the whole operator industry. We have enterprise-level global data storage center, computing center, capability center and incubation center. In the operator industry, their system architecture models are basically built by provinces, but on the first day of building the big data platform, China Unicom collected the data to a node in its headquarters. we firmly believe that only when data are converged can chemical reactions occur and maximize value.

China Unicom has 100 petabytes of data throughput capacity and unified data service capacity. Our data center has more than 100 PB. Of course, the larger the amount of data, the better. The data itself has a cost. We hope that the cost of the data and the value of the data can reach a balance. In addition, we have more than 6500 server data nodes for internal services and more than 2000 server data nodes for external services, totaling about 9000 nodes. In addition, from the storage capacity of China Unicom, our current storage capacity may be close to 200 PB.

The global data aggregation and management center precipitates massive computing capacity, storage capacity and data capacity, which causes China Unicom to face the problems of intelligent resource scheduling, maximizing utilization and capacity sharing.

China Unicom has the overall data center node, while under the headquarters are the provincial branches and subsidiaries of China Unicom. They hope to use the big data platform of the headquarters for their respective data processing and data analysis, thus creating the demand for cloud computing. They hope that the nodes of the headquarters can provide it with a platform for data processing, and provincial branches and subsidiaries will carry out their own data processing and processing on the platform.

This is the source of our own optimization, that is, how China Unicom's own nodes avoid unbalanced scheduling of computing and storage resources and innovatively provide tenants with the same capabilities. At this time, big data of China Unicom and cloud computing naturally came together.

Exploration course

We have invested a lot of efforts in the construction of China Unicom big data cloud platform since 2016, and have gone through several different stages of development.

In the initial construction stage, our resources are physical deployment, manual allocation, system operation and maintenance. We have experienced the overall development of big data. At the beginning, if you want to be big data, you must achieve it through a physical machine. You have to deploy a Hadoop machine. If you need kafka, you also need to deploy a kafka with a physical machine. This is the inevitable development stage of big data platform construction.

The next stage belongs to the optimization and promotion stage. We hope to manage resources through a centralized working group. When others have resource needs, we will do semi-automatic deployment, semi-manual allocation, and simple monitoring of system operation and maintenance.

The third stage is to provide one-click deployment through big data cloud platform. You need a big data platform. Through one-click deployment, I will provide you with a big data platform on which you can process and process your own data. Your data can come from the data platform of the headquarters, or from your own data. In this way, automatic distribution, retrenchment, unified monitoring and unified operation and maintenance are realized. We are now in the third stage.

When China Unicom carries on the technical route selection, we are faced with the choice between Kubernetes and Mesos.

Why did we choose Kubernetes? Because Kubernetes supports almost all types of container business, including long-term servo, batch processing, node-deamon and stateful applications. At the beginning, when we were doing container applications or microservice applications, there were more stateless applications. But when we provide big data services, many applications are stateful.

We have made a very in-depth analysis of Kubernetes and Mesos, especially the ecological activity of Kubernetes and the attention of the community are increasing sharply. When we carry on the technology selection, including its use scenario, external application, intermediate node and database, service status and so on. In addition, whether the maturity of technology is widely blessed by industry manufacturers, such as Google, Amazon, Rancher, IBM, Ali, Baidu and so on. Kubernetes has a very good ecology, so we chose Kubernetes to solve the actual needs of China Unicom.

When talking about Kubernetes, it is inevitable to mention the cooperation between China Unicom and Rancher.

In the process of building Kubernetes and Docker containerization platform for China Unicom, we introduced Rancher products to deploy and manage multiple Kubernetes clusters. We use Rancher Server to deploy and manage multi-tenant Kubernetes clusters through graphical and RKE methods.

On the other hand, from our point of view, Rancher has rich experience in containerization implementation cases, which just makes up for some shortcomings of China Unicom and becomes a strong backing for us in dealing with and solving problems. We are more focused on how to turn the service into a cloud and then open it to provincial branches and subsidiaries, and how we can process the data better. For the underlying services, we hope to use the industry partners to work with us to solve the problem.

In addition, open source products often have major security loopholes. In this respect, Rancher can provide a good technical support for China Unicom and protect the cloud platform of China Unicom.

Platform practice

China Unicom provides several services, one is big data as a service, for example, our provincial branch or subsidiary needs a big data platform, we will provide it with a big data platform, including Hadoop, Spark, Storm, impala and so on. Once I can provide big data platform for provincial branches and subsidiaries, they will not need to rebuild big data platform themselves.

Second, middleware and database as a service. For provincial branches and subsidiaries, it is not enough to have big data platform for data processing, in which a lot of middleware must be used, so we have to provide them with middleware and database-as-a-service, including kafka database as a service, Redis distributed cache service and MySQL relational database service.

With the above two, we can also provide data integration tools as a service, such as cloud ETL, and I can do data extraction and transformation to provide scheduling for provincial branches and subsidiaries.

We mentioned ABC earlier, and further extended, we can provide deep learning as a service, such as TensorFlow, Caffe and so on.

The last one is the container cloud service, where we can provide an application hosting environment.

With the above services, for tenants, covering the data processing, processing, transfer and application provision, basically have the framework of the PaaS platform. Our provincial branches or subsidiaries no longer have to build their own big data platform. If they want to do the big data application, they can do it on the headquarters platform, which is equivalent to deploying a task on the headquarters platform. For them, they have their own data and space to do a processing. We have met their needs through the big data platform.

The above also lists some PaaS capabilities such as Hadoop, Spark, Hive, HBase, ZooKeeper, Storm, Implpa, etc., as well as distributed data warehouse, data Mart, real-time computing, data mining engine, information retrieval engine and so on.

In addition, middleware and database service components such as kafka, MySQL, data integration tools include metadata management, data governance services, cloud ETL services, data collection management services and so on.

We currently have many other services, such as data security services, desensitization services, traceability services, which we provide to provincial branches and subsidiaries through the cloud.

Whether it is China Unicom's big data basic services, deep learning framework, middleware and database services, or the development and deployment of micro services, data integration tools, etc., these contents cannot be done by a single team. We have a lot of teams, some teams focus on big data basic services, some teams focus on deploying deep learning framework on Kubernetes. Some teams focus on deploying middleware as a service on the Kubernetes container cloud platform, which is done by different teams, so we need to have a unified management platform.

We use Kubernetes Service Catalog and Open Service Broker on this. A unified integration framework such as Kubernetes Service Catalog realizes the unified management of heterogeneous components, and the access and expansion of third-party components through the industry standard Open Service Broker. When there are new services under us, we access the new services through an overall open architecture and open them to our customers.

We have four main categories of supporting scenarios. The first scenario is that you need a big data platform. I will provide you with a big data platform, which is the most basic and the most difficult to achieve. The second scenario is to provide you with big data's service components, you carry out data processing and processing, such as providing a distributed data warehouse, you take the data in and take it away or use it for other applications. The third scenario is the containerized deployment of application microservices. The last scenario is that I provide you with an incubation platform, provide sample data, provide data processing components, and do some model training yourself.

From the perspective of the overall application scenario of tenants, the largest one is model training, with a proportion of 34%. Provincial big data platform construction accounted for 17%, containerized application and service development and deployment accounted for 27%, and big data processing accounted for 22%. This is our current application.

We also adopted a logical multi-rent approach before, because we have a big data platform, resource scheduling and management, we can use a logical multi-rent way to provide services for you, but this has some limitations. Big data platform based on container cloud adopts more physical multi-rent method, which can effectively isolate resources, data, services and business. The thing I applied for is mine, which is different from logical groups. Logical overrenting always feels like I share a space with others, while physical overrenting this space is yours. We don't care what you do in it. We just want you to talk about it and do it more easily.

We will certainly face some technical challenges in this process. Includes the integration of a variety of PaaS capabilities. The second is the interconnection of multiple Kubernetes clusters, because we have just shared that there are N teams under China Unicom. Big data platform services may be distributed on one Kubernetes cluster, or they may be distributed on more than one Kubernetes cluster, which is composed of multiple Kubernetes clusters. The third is the containerization of big data services, such as how to put Hadoop on the container cloud platform, the deployment of Hadoop needs to be highly planned, and the affinity of computing and data needs to be achieved. How should we solve this problem? Finally, there is the localization of computing resources.

We have some solutions to the above technical challenges. The integration of diversified PaaS capabilities we will use Open Service Broker to solve this problem.

The network interconnection between Kubernetes clusters will be solved through the interconnection of multiple models. We have multiple clusters. On big data Cloud platform, our application can access your Datanote, your application can directly access Datanote to write and read data, and the problem of API call is solved by customizing Flannel network plug-ins. The two clusters share a Flannel network to achieve direct cross-cluster pod IP connection.

As for the question of how to complete the deployment of Hadoop containerization, including how to complete role planning if the services of Hadoop are split? The Hadoop is split according to the smallest unit of the component, and the reasonable deployment of a cluster is realized through the affinity scheduling algorithm. How to solve the problem of dependency and discovery between services after the split? Within the cluster, we use Headless Service to solve the problem, and we directly call the services provided by the underlying pod. Discovery between services is done through DNS and configuration injection.

Third, multiple pad we share Domain Socket, and each tenant's Domain Socket is independent of each other.

China Unicom big data cloud platform provides more than 30 kinds of PaaS capabilities in 6 categories, and we have 437 clusters. On big data Cloud platform, we are open to tenants to use this platform, performing more than 160000 MR/Spark tasks and implementing more than 15000 data scheduling tasks per month.

At present, we are still gradually expanding the scope of use. previously, many of our small provincial companies have put big data platform on China Unicom big data cloud platform for deployment. In the next step, China Unicom will further expand its scale. In the future, most of the provincial branches and subsidiaries will deploy the big data platform on this platform.

In the future, we hope that the big data cloud platform of China Unicom will continue to evolve and provide Kubernetes itself as a service to our tenants. On the other hand, we hope to provide FaaS to provide customers with more convenient services by using Serviceless. It only needs to provide a function, and the backend service can be zoomed and scaled according to the function transfer.

Another direction of exploration is the cloud platform that supports the Internet of things. Now the speculation of 5G is very hot, no matter it is the operator industry or the whole society, everyone is paying attention to 5G. 5G mainly solves three major problems, one is large bandwidth, the second is high-density connection, and the third is low latency. Both high-density connectivity and low latency belong to the applications of the Internet of things. We build big data cloud platform to provide new services for the Internet of things, the Internet of everything will generate more data, how to process the data in real time? How to carry on the follow-up analysis and treatment? We hope to solve this problem through a public cloud platform.

Summary and prospect

The above is the construction of China Unicom's big data platform based on container cloud. At present, it seems that we have achieved very good results. In the future, we will integrate more capabilities, enable the foreground, achieve intelligent management, and improve the overall utilization efficiency.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.