Shulou(Shulou.com)11/24 Report--

On July 6, the 2023 World artificial Intelligence Congress (WAIC) opened at the Shanghai World Expo Center. Many experts, scholars, science and technology leaders and enterprise representatives discussed the new changes in the development of artificial intelligence enabling industry and looked forward to the new trend of technology. Wang Haifeng, chief technology officer of Baidu and director of the National Engineering Research Center for Deep Learning Technology and Application, interprets the core technology of Wen Xin's model version 3.5, releases the latest progress in flying paddle ecology, and expounds the model of artificial intelligence industry. speak for the latest artificial intelligence technology and industry.

The flying oar has gathered 7.5 million developers, the effect of Wen Xin 3.5 has been increased by 50%, and the reasoning speed has been increased by 30 times.

At present, artificial intelligence technology, represented by the big language model, has set off a wave of science and technology and industrial innovation around the world, accelerating industrial upgrading and economic growth, and great changes will take place in various industries. The IT technology stack has fundamentally changed from three-tier architecture of chip, operating system and application to four-tier architecture of chip, framework, model and application. The deep learning framework and large models constitute the industrial intelligent base, which will support the intelligent reconstruction of applications in various industries and promote high-quality economic development.

It is understood that Baidu has layout and leading self-research technology in the four-tier technology stack of artificial intelligence, especially in the framework layer and model layer, which are located at the core of the four-tier architecture. The latest achievement of Wen Xin big model also benefits from the joint optimization of flying paddle deep learning platform and Wen Xin. Flying Propeller is the first industry-level deep learning open source open platform independently developed in China, which ranks first in the comprehensive market share of China's deep learning platform for two consecutive years. Wang Haifeng revealed on the spot that the flying oar has gathered 7.5 million developers so far, which is the first time that Baidu has disclosed the latest data of flying oar ecology since 2023.

After four years of deep technical ploughing and R & D iteration, Baidu has upgraded to Wenxin Model 3.5 after the release of version 1.0 in March 2019. Wang Haifeng said that the effect, function and performance of Wen Xin Da Model 3.5 have been comprehensively improved, realizing the upgrade of basic model, fine tuning technology innovation, knowledge point enhancement, logical reasoning enhancement, etc., the model effect has been increased by 50%, and the training speed has been increased by 2 times. the speed of reasoning is increased by 30 times.

The core technology continues to make breakthroughs, and the effect and efficiency leap together.

In March this year, Baidu released Wen Xin, the big language model, for the first time among the world's largest technology companies. Wenxin word is a large language model of knowledge enhancement. Firstly, the pre-training model is obtained by combining learning from trillions of data and hundreds of billions of knowledge, and on this basis, techniques such as intensive learning and prompts with supervised fine tuning and human feedback are adopted. and has the technical advantages of knowledge enhancement, retrieval enhancement and dialogue enhancement.

Wang Haifeng interpreted the core technological innovation of Wen Xin Big Model 3.5. In the basic model training, he adopted the most advanced adaptive hybrid parallel training technology and mixed precision calculation strategy, and adopted a variety of strategies to optimize the data source and data distribution. the iterative speed of the model is accelerated, and the effect and security of the model are significantly improved. At the same time, technologies such as multi-type and multi-stage supervised fine tuning, multi-level and multi-granularity reward model, multi-loss function hybrid optimization strategy and model optimization with the combination of two flywheels are innovated to further improve the model effect and scene adaptation ability.

On the basis of knowledge enhancement and retrieval enhancement, Wen Xin Great Model 3.5 puts forward the "knowledge point enhancement technology", which analyzes and understands the queries and questions entered by users, and parses out the relevant knowledge points needed to generate answers. then use the knowledge graph and search engine to find the corresponding answers for these knowledge points, and finally use these knowledge points to construct tips for input to the large model. Inject more specific, more detailed and more professional knowledge points into the large model, and significantly improve the mastery and application of the world knowledge of the big model.

In the aspect of reasoning, through large-scale logical data construction, logical knowledge modeling, multi-granularity semantic knowledge combination and symbolic neural network technology, the performance of Wen Xin big model 3.5 in logical reasoning, mathematical calculation and code generation is improved.

Add plug-in mechanism to expand the capability boundary of large model

Wen Xin big model 3.5 added a plug-in mechanism, Wen Xin Yiyan has released the official plug-ins Baidu search and ChatFile on June 17. Baidu search is the default built-in plug-in, which makes Wenxinyan have the ability to generate real-time and accurate information. ChatFile is a long text summary and question and answer plug-in that supports ultra-long text input.

Wang Haifeng said that Wenxinyan will release more high-quality Baidu official and third-party plug-ins to enable users to better apply the Wenxinda model, and will also gradually open up the plug-in ecology. to help developers build their own applications based on the Wenxin model.

Widely used in all kinds of scenarios to accelerate the intelligent upgrading of the industry

Wang Haifeng showed the application of Wen Xin word in office, conference, coding and other scenes. Wen Xin word became a "super assistant" at work, helping to summarize the main points of work communication and record the contents of the meeting in real time. to form key information such as meeting topics, summaries and summaries, you can complete instruction tasks through various plug-ins, including querying schedules, creating meetings, setting to-do, applying for leave, and so on. You can also automatically recommend and generate code during the engineer coding process. It is reported that at present, these functions have been applied to Baidu's workflow through the intelligent work platform "such as flow" to help improve work efficiency and improve the quality of decision-making.

Wang Haifeng said that all application scenarios that have to deal with language or program code may have the opportunity to show their talents in a single word. There have been many scenarios in which Wen Xin has been actively used, such as energy, finance, education, office, media, and so on. In the process of the landing of a large model industry such as Wen Xin, the mode of "intensive production and platform application" can be adopted, that is, enterprises with comprehensive advantages of algorithm, computing power and data encapsulate the complex process of model production, and provide large model services for thousands of industries through a low threshold and efficient production platform.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.