Shulou(Shulou.com)11/24 Report--

Cloud acceleration, run-up closed loop, Huawei Cloud and Euro think tank released a white paper on the development of data closed loop.

High-level autopilot will gradually move into reality, massive data processing and efficient mining have become the first problem that enterprises must solve, so data closed-loop has been paid more and more attention. Because of its intelligent genes and strong data processing ability, cloud service providers have become an important force in the development of the self-driving industry at the present stage, and "go both ways" with the self-driving industry as a "third-party service provider". Through the core competence of cloud base and the deep data processing process of cloud large model, we can provide more intelligent and efficient data and AI services for mainframe factory, Tier1 and other enterprises, and accelerate the realization of data closed loop.

Based on this, Yiou think tank and Huawei Cloud jointly write a white paper to gain an in-depth insight into the development of self-driving data closed-loop, disassemble each link of data closed-loop in detail, and analyze the current situation of data closed-loop development from multiple angles. at the same time, explore the value that cloud service providers with professional accumulation and strong perception, computing, processing, storage and other capabilities can give to each link. On September 21, the 2023 Pangu model of Huawei fully connected Conference released the "White Paper on the Development of a New engine for Cloud Services, efficiently driving self-driving data closed Loop" (hereinafter referred to as "White Paper").

The following is a summary of the White Paper:

First, half-court fierce battle, the development of autopilot is facing data challenges

The development of self-driving industry has entered the second half, and intelligence has become the focus of industry competition. With the advent of large model technology in 2023, high-level autopilot will gradually move into reality. The iteration of the head enterprise system has accelerated, and in the past two years, L4 autopilot mass production and landing plans have been launched one after another, extending to city-level applications. Local governments have also issued policies to support the development of the self-driving industry and promote the application of front-end technology.

Under the background of the improvement of the level of intelligence and the expansion of the scope of application of the scene, the amount of data collected by self-driving vehicles has increased sharply, the corresponding preprocessing and model training have become more difficult, and the costs of transmission and storage have also increased significantly. All kinds of factors put forward higher requirements for the efficiency and security of data processing.

Therefore, the closed-loop path which is interlinked, data flows coherently and can circulate effectively is becoming more and more important. The mature data closed loop can improve the efficiency of data transmission and processing, enhance the controllability, and improve the efficiency of autopilot research and development. Nowadays, the construction and efficient operation of data closed loop has become a necessary condition for the landing of high-level autopilot.

Second, the closed loop is in the loop, and the efficient circulation of data can accelerate the landing of mass production.

With the increasing level of autopilot and the increase of the average number of sensors configured, the type and scale of heterogeneous data collected will naturally become numerous and large, and the mass production of L4 will produce millions of millions of data. The R & D process for a large amount of data requires targeted design and development tools, and the related adaptation work also requires a high cost, so it is difficult to count the manpower cost and time cost to support R & D. The rapid growth of data brings a series of processing problems, the data platform that should be closely connected with the AI platform is still in a state of separation, and the efficiency of data mining and utilization is still low. In addition, data in multi-level processing, flow also implies a large number of security risks, once the data is attacked, stolen, tampered with, it will cause serious damage to enterprises, citizens, society and even national security.

Yiou think tank believes that the road to self-driving data closed-loop development is full of thorns, obstacles and long, to achieve qualitative change requires the joint efforts of the industry, and cloud service providers are one of the most important links.

Based on cloud service, build a solid base of data closed loop

In terms of data processing, cloud services can give full play to efficient computing, super-large storage, fast transmission and other capabilities, targeted data processing. At the same time, cloud services also open up the underlying metadata management model, build a unified data governance base, realize the integration of data governance and AI development based on the connected and shared base, and create a positive cycle in which data and AI empower each other.

For example, Huawei Cloud ModelArts platform provides DataTurbo, TrainTurbo and InferTurbo layer acceleration, which provides data loading, model training and model reasoning services respectively, which can reduce data reading time by 50% and improve training efficiency by 40% +. The unified data lake reduces data storage costs by 20%, and the highest sinking rate of cold data is 96%. In addition, Huawei Cloud also integrates the data production line and the AI production line, realizes the seamless flow of data through a unified digital intelligence fusion development platform, and uses the Pangu model to deeply empower the data closed-loop core scene.

In terms of R & D cost, the traditional autopilot development tools are very fragmented, the development processes are separated from each other, tool link debugging is time-consuming and laborious, which seriously slows down the R & D process. Cloud services can completely cover all aspects of the data closed loop, providing data management platform, AI model training platform, simulation and evaluation platform, etc., forming a full-stack R & D tool chain. Series interworking tools enhance business continuity, shorten model training time, greatly reduce debugging costs and labor costs, and improve R & D efficiency.

Huawei Cloud has built a full-stack self-driving development platform on the tool chain. Not only provide customers with an one-stop platform, from development to testing to commercial use, help enterprises to achieve on-demand use, plug and play, but also provide modular solutions, mainly data services, training services, simulation services, to help customers achieve data-driven self-driving development.

In terms of data security, in the face of high-level self-driving with more data, the greater the risk, and the boom of cars going to sea, the security compliance programs provided by cloud services are very important. Through the full-cycle security compliance scheme of data, cloud manufacturers can achieve collection compliance, transmission confidentiality, storage security, use supervision and so on.

Wulanchabu Automobile Zone, a cloud infrastructure built by Huawei Cloud for the automotive industry, adopts a three-zone compliance structure and seven layers of security protection, and relies on the capabilities of qualified graphic partners to provide full-process security and compliance protection for self-driving data. At the same time, Huawei Cloud has also become an overseas operating partner of car companies, providing full-process services for car companies by connecting with global safety and compliance data centers, helping Chinese car companies feel at ease to go abroad.

Large model takes the stage to accelerate the closed-loop operation of data efficiently.

It is generally believed that the end of autopilot will be a very large-scale end-to-end autopilot data closed loop, and the large model is an important way to achieve this form. It enables autopilot data closed loop to reduce cost and increase efficiency, which is mainly reflected in two aspects: perception and regulation. It is embodied in multimodal perception, data processing and generation, decision-making and path planning in the driving process, so as to accelerate the realization of end-to-end autopilot.

Yiou think-tank survey found that this year, Tesla, Xiaopeng, ideal and other mainframe manufacturers, as well as Huawei Cloud, Baidu Cloud, Maomo Zhixing and other Tier1 manufacturers have released their own large models and tool platforms. Compared with the inherent properties of the car side, the cloud can deploy larger models, which makes cloud service providers play a vital role in the development of self-driving industry. Through the data acquisition, mining, tagging, storage, simulation and other links, with the help of the deep learning ability of the large model for many iterations, the performance is improved rapidly. Huawei Cloud has created a number of large autopilot vertical models, such as scene generation model, scene understanding model, pre-tagging model, multi-modal retrieval model and so on, to promote efficient data circulation and achieve closed loop.

Third, above the cloud, all aspects of the data closed loop will benefit from all aspects.

From the perspective of process disassembly, the data closed loop can be roughly divided into five links: data acquisition, data annotation, model training, simulation testing and application deployment. The services provided by cloud manufacturers can infiltrate into all aspects, so that the collection of cloud, data into the cloud, storage on the cloud, testing into the cloud, deployment depends on the full link support of the cloud.

In terms of collection, in the face of more and more complex data, cloud services can quickly clean, filter and mine high-value data in massive data. At the same time, by building the nearest dedicated access point for automobile data, the transmission delay caused by network handover can be reduced as much as possible.

In tagging, the large AI model carried on the cloud can be tagged automatically, intelligently extract difficult scenes through tags, graph search and other ways, build a high-quality database and scene set, and timely feedback to the algorithm to accelerate iteration to improve the labeling efficiency.

Model training is inseparable and closely related to data tagging. After basic data extraction, the algorithm can be trained repeatedly. At the same time, the continuous training algorithm can further enable automatic tagging and achieve higher accuracy.

Compared with the traditional stand-alone simulation with weak computing power, long cycle and low efficiency, cloud platform simulation has more flexible computing space and can be expanded rapidly at any time to achieve more efficient simulation testing. At the same time, some cutting-edge technologies based on cloud service capabilities are also accelerating research and development, such as NeRF is being used by many enterprises for 3D city-level simulation scene reconstruction and so on.

During deployment, efficient cloud services can manage a large number of software packages on the cloud and select the most secure and appropriate time and scenario to upgrade vehicles to avoid the risk of taking too long, affecting users' interactive experience and even large-scale vehicle recall.

Fourth, move at the right time, cloud manufacturers help to build a new data closed-loop paradigm

The challenges and pain points on the road of data closed-loop development are essentially the test of the hard power of products-whether autopilot can be enabled through data closed-loop. Based on this, the White Paper puts forward a new data closed-loop paradigm to break through the bottleneck of industrial development from the three aspects of technology, service and ecology, and provide high-quality and efficient services for automobile enterprises and Tier1 manufacturers.

New technology: developing both hardware and software, expanding the upper and lower limits to break through the technical barriers

The software determines the upper limit of autopilot. Traditional autopilot different scenes correspond to different small models, such as CNN, RNN, etc., which have small parameters and poor generalization, and can not support the massive data processing requirements of high-level autopilot. In the future, the data closed loop will be fully integrated into the AI model, relying on the efficient computing and processing capabilities of cloud services, with the underlying model + vertical domain layout of multi-tier structure, improve the upper limit of software technology, applied to a variety of scenarios.

Huawei Cloud recently released Pangu Model 3.0, including a "5+N+X" three-tier architecture. Layer L0 includes five basic models: natural language, vision, multimodal, prediction and scientific computing, which can meet the needs of a variety of skills in industry scenarios. Huawei Cloud can not only provide industry general models for training using industry open data, but also train proprietary models based on industry customers' own data. L2 layer provides customers with more models to refine the scene and provides "out-of-the-box" model services. Pangu large model uses a complete hierarchical decoupling design, which can quickly adapt and quickly meet the changing needs of the industry.

For the automobile industry, Huawei Cloud released a large model of Pangu car. Covering the automobile design, production, marketing, research and development and other business scenarios, the whole scene to improve quality and efficiency and accelerate the mass production of intelligent cars. Let every employee of the car company have his own expert assistant to make the work more efficient and easier. For self-driving, the large Pangu car model generates complex scene samples by building a digital twin space, which shortens the learning and training cycle of autopilot Corner Case from more than 2 weeks to 2 days. In the commercial special car scene, Pangu can simulate and mark automatically the dusty environment, up and down slopes and large curvature turns in the mining area, and can adapt to the new heavy truck model in 4 months. At present, Xingna in Xinjiang and Yimin opencast Coal Mine in Inner Mongolia have used Huawei commercial chauffeured vehicle self-driving cloud service, which can achieve a lateral error of 60 tons of heavy trucks of less than 0.2 meters and precision parking errors of less than 0.1 meters.

Hardware determines the lower limit of autopilot. A large amount of data needs to be supported by strong computing power, and the previous research and development of self-driving AI models around the world are very dependent on Nvidia's A100 / H100 chip. However, due to international political and economic factors, domestic autopilot research and development is facing a lack of core crisis, so there is an urgent need for domestic available computing power and services.

At this point, Huawei Cloud has enough "confidence". Based on the basic chip family of Pagoda, Huawei Cloud has built the AI computing platform for AI self-driving training, which realizes the autonomy of the whole stack of AI computing power and hardware business continuity, realizes the software-hardware coordination of the connection between upper and lower parts, and provides customers with a more comprehensive, coherent, safe and convenient R & D environment.

New service: mass production-oriented, commercialization promotes the application of products on the ground

In order to avoid the risk of technical jam, many car companies and self-driving solution providers prefer modular cloud service products. Therefore, the platform with openness and free choice for customers will become the mainstream business model in the future. In addition, as Chinese car companies are singing all the way into overseas markets, cloud service capabilities will also follow car companies to export, empowering Chinese brands to expand international markets from smart cars and smart road networks.

In the business model, Huawei Cloud not only provides in-depth ecological co-construction services, but also provides flexible modular self-driving solutions to achieve cloud-based, on-demand matching, helping enterprises to reduce a lot of cost waste of repeated development. At the same time, in the sea service, it also provides a full range of digital intelligence programs. Today, Huawei Cloud has 30 geographical regions and 84 availability zones around the world, has 120 + authoritative compliance certification, serves more than 170 countries and regions around the world, and meets the global business deployment and localized operations of automobile companies through a global car deposit network.

New ecology: end-cloud collaboration, car-road-cloud co-construction industry ultimate form

On June 2, 2023, the National standing Committee first mentioned the integrated development of "Channeng Road Cloud". For data closed-loop, from the initial realization of closed-loop for limited vehicles in the test area to the gradual generalization to the scale of production vehicles, autopilot will continue to cooperate with road-end data and overlay through multi-perceptions such as car, road, cloud, human, network, map, etc. At the same time, give full play to the supporting capacity of cloud services, open more imagination, and finally create a better travel ecological experience.

In terms of ecological contribution, deep corporate precipitation makes Huawei Cloud a leader in the industry. Huawei Cloud forms the "1+3+M+N" cloud infrastructure layout of the global automotive industry, using "M" distributed vehicle networking nodes and "N" vehicle data access points to achieve efficient cloud and high-speed transmission of massive data, building a strong foundation for end-to-end cloud collaboration. At the same time, with the advantage of ICT, Huawei Cloud has successively launched relevant products in cars, roads and clouds. From test parks to urban roads, Huawei Cloud has joined hands with ecological partners to connect smart cars with smart roads to jointly promote self-driving landing maturity.

Fifth, the line is coming, the data closed-loop run-up autopilot transition upgrade

The development of data closed loop is data first, then closed loop. Taking data as the carrier to form the continuous iteration of the algorithm, the core lies in the running of the "ring". In the process of enterprises working together in the industry, cloud service providers play a vital role, integrating multi-link data processing functions such as cleaning, mining, storage, labeling, computing and so on, so that the data can be based and closed loop.

Autopilot is a good wish, it must be done, and everything is in the ascendant. The outbreak of large model also gives autopilot more imagination space, which also becomes a necessary and sufficient condition for the development of data closed loop, while cloud service will be used as the foundation to support large model and empower data closed loop. In the future, with the enhancement of cloud service capabilities, the realization of large model + data closed loop will help autopilot develop faster and grow more steadily.

The above point of view comes from the "White Paper on the Development of the New engine for Cloud Services, efficiently driving the closed Loop of self-driving data". For more information about the closed loop of data, please see the "White Paper on the Development of the New engine for Cloud Services and efficiently driving the closed Loop of self-driving data".

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.