Shulou(Shulou.com)11/24 Report--

On November 4th, the 2nd Open Atom Open Source Conference OpenHarmony Technology Conference was successfully held in Beijing. In the OS native intelligence sub-forum held in the afternoon, Jin Xuefeng, chief architect of Huawei Shengsi MindSpore, served as the producer, Wang Lei, architect of Huawei AI large model application development platform, served as the host, and joined technical leaders from Huawei, ChinaSoft International Co., Ltd., Shenzhen Ruoxin Technology Co., Ltd. and other enterprises as well as from Tsinghua University. Experts and scholars from academic circles such as Shanghai Jiaotong University shared how to carry out technological innovation and break through technical problems in combination with AI large model related technologies in their respective fields in OpenHarmony ecosystem, and jointly discussed the technological development trend of combining AI large model with terminal operating system in the future, contributing to the co-construction of OpenHarmony prosperous ecosystem.

Producer: Jin Xuefeng, Chief Architect of Huawei Shengsi MindSpore

Host: Wang Lei, Architect of Huawei AI Large Model Application Development Platform

First of all, Li Yuanchun, an assistant researcher from Tsinghua University, gave a report on the topic of "Large Model-Driven Terminal Intelligent Agent," introduced the design and implementation of a large model-driven intelligent personal agent system, and shared how to organically combine the domain knowledge within the application with the common knowledge of the large model through automatic analysis of mobile applications and large model knowledge embedding, so as to realize more accurate and efficient task automation. Li Yuanchun pointed out that intelligent personal agent has always been one of the key technologies concerned by terminal system researchers and developers. However, due to the lack of user intention understanding, task planning, tool use and other capabilities, the intelligence and scalability of intelligent personal agent still have great defects. The emergence of large language models solves these difficulties. Li Yuanchun said:"The emergence of large models represented by large language models has brought new opportunities for the development of this field, and it is expected to greatly enhance the breadth and depth of intelligent personal agent support functions through powerful semantic understanding and common sense reasoning capabilities. "

Li Yuanchun, Assistant Researcher of Tsinghua University

Zhang Zhaosheng, General Manager of Product R & D Management Department of Intelligent Internet of Things Corps of Chinasoft International Co., Ltd. and member of Technical Steering Committee of OpenHarmony Project Group, brought a report entitled "Application of Visual Large Model in OpenHarmony's End-edge Cloud Side" based on his deep insight in the field of large model and end-edge collaboration. He pointed out that in the era of intelligence, with the continuous improvement of large models and computing technology, the deployment of large models on the edge side and terminal side has become an inevitable trend. He highly affirmed the value and significance of OpenHarmony as a digital base."As a technical base in the era of Internet of Everything, OpenHarmony can be widely used in a variety of computing scenarios, meeting the business needs of multi-connection, high real-time and massive heterogeneous data, and playing a role in exerting hardware computing power downward and enabling mass applications upward. "In Zhang Zhaosheng's view, based on OpenHarmony, the integration of large-scale edge-end model capabilities and the construction of end-edge collaborative business scenarios can provide more competitive innovative solutions for the industry and accelerate the expansion of the ecosystem.

Zhang Zhaosheng, General Manager of Product R & D Management Department of Intelligent Internet of Things Corps of Chinasoft International Co., Ltd. and Member of Technical Steering Committee of OpenHarmony Project Group

Zhou Jianhui, architect of Huawei Xiaoyi, based on Huawei Xiaoyi's intelligent service technology in terminal products, brought the theme report "Exploration of Ecological Construction of End-side Native Intelligent Large Model Based on Intent Framework." It is mentioned in this report that the intention framework is a system-level intention standard system, which constructs a global intention paradigm through multi-dimensional system perception, large model and other capabilities, realizes the understanding of users 'explicit and potential intentions, timely and accurately transmits user needs to ecological partners, matches timely services, and provides users with multi-modal and personalized advanced scene experience.

Huawei Small Arts Architect Zhou Jianhui Theme Sharing

Zheng Wenli, Associate Professor of Computer Science and Engineering Department of Shanghai Jiaotong University, focused on machine learning methods and presented a report entitled "Dynamic Segmentation of Deep Neural Networks in End-edge Cloud Collaborative Reasoning." In the end-edge cloud environment, in order to match the dynamic changes of network load and server load, Professor Zheng Wenli proposed a DNN segmentation optimization algorithm based on machine learning, and upgraded it to MindSpore, so that it can automatically adjust their distribution across the end and edge/cloud sides when executing various common CNNs and RNNs, thus maintaining the lowest inference delay. At the same time, he also pointed out that "resource constraints are the main challenges facing edge computing, and trying to break through resource constraints can maximize the edge advantage and make intelligence truly enter everyone's life." "

Zheng Wenli, Associate Professor, Department of Computer Science and Engineering, Shanghai Jiaotong University

Hou Lu, a researcher from Huawei Noah's Ark Laboratory, shared "Compression Acceleration and Efficient Deployment of Large Language Models" on site. This report introduces the challenges of memory, memory access and computation faced by large language models in inference stage from aspects of architecture design, cost, throughput, delay, long sequence, etc., as well as the benefits of model and KVcache quantitative compression, fusion of large operators and speculative reasoning in efficient inference of large language models.

Hou Lu, Researcher of Huawei Noah's Ark Laboratory

Zhou Peng, CTO of Shenzhen Ruoxin Technology Co., Ltd., gave a speech entitled "Ubiquitous Intelligence: Applications and Solutions of End-side Brain-like Large Models", introducing the computational overhead problem facing large language models and how to improve Transformer architecture by using brain-like information processing methods. By using the third generation neural network SNN to build neural network, the data in the network is stored and transmitted by pulse sequence, which greatly reduces the inference overhead without affecting the intelligence of the network. Zhou Peng said,"The large model trained by this technology not only achieves the optimal energy consumption, delay and computing power requirements at the same parameter level, but also successfully has the ability to deploy and operate locally in consumer-level equipment.

Shenzhen Ruoxin Technology Co., Ltd. CTO Zhou Peng Theme Sharing

Li Zheng, MindSpore architect of Huawei Shengsi, combined with MindSpore framework technology exploration and practical experience, shared the report "MindSpore end-side large model deployment helps OS intelligence" to explain how to solve the deployment and application challenges of AI large model. Li Zheng mentioned that "the generative AI model is quietly changing people's work and life, and the combination of AI model and intelligent terminal is bound to lead to new experience." However, due to the computing power and storage constraints of terminal equipment, the end-side deployment of AI large models faces many challenges." In his report, he introduced MindSpore's exploration and related technical ideas for end-side deployment and inference acceleration of large models. As an open source AI framework integrating cloud-side full-scene training and push, MindSpore deeply participates in the performance optimization of HarmonyOSAI intelligent business scenarios, and a large number of Huawei mobile phones, tablets, watches, notebooks, smart screens, smart cars and other terminal products are implemented.

Huawei Shengsi MindSpore Architect Li Zheng Theme Sharing

In the final part of the OS native intelligence sub-forum, Wang Lei, architect of Huawei AI large model application development platform, Jin Xuefeng, chief architect of Huawei Shengsi MindSpore and guest speakers held a round-table discussion on the theme of "technical frontier challenges and solution ideas for end-side large models and applications", further discussing the AI application architecture and framework, model training framework under OS native intelligence. Technical ideas and ideas on model architecture and algorithms.

Round table discussion on "technical frontier challenges and solutions for end-side large models and applications"

At this point, the 2nd OpenHarmony Technology Conference OS Native Intelligence Sub-Forum was successfully concluded. The sharing between technical experts and industry celebrities shows the technical potential and application practice of OpenHarmony in the field of native intelligence, and also looks forward to the "win-win" prospect of expanding the capability boundary of large model and making terminal equipment more intelligent after the deep combination of AI large model and operation terminal. As more and more technologists and industry elites in the field of native intelligence join the OpenHarmony community, it will also promote the OpenHarmony ecosystem. We look forward to more partners joining OpenHarmony technology and ecological co-construction, working together to "build ecology with technology, win the future with intelligent connection."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.