Shulou(Shulou.com)11/24 Report--

The war on the big model is getting hotter and hotter, and everyone is watching the latest changes in the industry.

If there is any news in the tech world dominating the headlines, there is a good chance that there will be new developments in the big model.

The big model idea brought by OPPO this time has aroused everyone's interest. On November 16, the 2023 OPPO developer Conference was held at the Shanghai World Expo Center, at which OPPO officially released the self-training personality model and agent-the Andean Great Model (AndesGPT).

At the press conference, OPPO pointed out that this is the "era of technology-driven experience", believing that the large model will comprehensively revolutionize the interactive experience of smart terminals and continue to bring high-quality experience to users in this direction. At the same time, in the aspect of strengthening the technological capability, OPPO will lay out the large model forward-looking technology, jointly with the Intelligent Computing Joint Laboratory established by the University of Science and Technology of China, and has taken the large model as the core technology research direction. At the same time, OPPO will also open source Agent framework to support efficient incubation, hosting and application of agents.

With "end-cloud collaboration" as the infrastructure design idea, AndesGPT offers model specifications with different parameter sizes ranging from 1 billion to more than 100 billion, which can flexibly support diversified application scenarios based on "end-cloud division of labor, end-cloud complementarity, end-cloud collaboration" and other ways. And AndesGPT emphasizes the technical characteristics of "dialogue enhancement, personality exclusivity, and end-cloud collaboration". It can be seen that OPPO has some of his own ideas on the big model track.

Hello, the new player in the big model field.

The announcement of this series of big moves once again confirms the era of all-round competition in the current big model.

The press conference starts with "smart experience", emphasizing "more useful, more professional, more exclusive". For example, the new Xiao Bu, released with ColorOS 14, has a wealth of agents built in, and different agents provide different scene experiences for users.

In this way, OPPO sets out the goal of AndesGPT: to create useful large models and agents for everyone, and will also open source the agent framework to create an one-stop "agent development platform".

According to reports, AndesGPT has three categories of specifications: AndesGPT-Tiny, AndesGPT-Turbo and AndesGPT-Titan, which can cover more than one billion to hundreds of billions of model specifications with different parameter sizes, and intelligently schedule according to different scenarios. In AndesGPT's network architecture, OPPO also proposed two combined innovations: "RoPE location coding to explore the optimal value of base"-- to improve the upper and lower "expansion energy", and "GQA combined with complex mobile window to accelerate training and reasoning"-- to achieve the linear complexity of O (Nd).

AndesGPT has a chronological memory mechanism, which can hold the upper and lower levels of the limit. Long-term memory mainly includes interaction history, personal data and structured information extracted from the process of user interaction. In order to solve the problem of the length of first word reasoning introduced by long-term memory, OPPO has developed a mechanism called SwappedAttention.

This mechanism has a certain degree of leadership and originality. Compared with PagedAttention, it has made the evolution and expansion of technology on the basis of PagedAttention.

For example, in the scene of first word generation, it is necessary to encode the history sequence. When the history sequence is long, it is easy to appear computing bottleneck, which is characterized by long time consuming and low concurrency. SwappedAttention caches historical KV values, greatly reducing the amount of headword computation. And use GPU video memory, host memory, GDS connection external storage and other multi-level cache methods, according to the cache time, conversation frequency and other strategies for hierarchical storage and exchange.

In practical applications, such as the common "multi-round conversation scenario", with the accumulation of chat rounds, the Prompt becomes longer and longer, and the first word time of each Query becomes longer and longer. SwappedAttention can cache historical KV values to multi-level cache, which can effectively reduce the time of first word inference during conversation and make users get faster system response.

In the "FileChat document dialogue scene", SwappedAttention can read the historical calculated KV value information, avoid the KV value re-calculation of long text, greatly reduce the computing overhead, shorten the first word reasoning time, and can improve the concurrency of the first word calculation; in the non-first word reasoning process, SwappedAttention can dynamically compress the KV value, further reduce the video memory footprint and improve the overall throughput.

From this conference full of practical information, we can see that OPPO has its own thinking on the big model, hoping to put the big model into practice and build a large model application ecology that everyone can participate in. The difficulty of the big model lies not only in "computing power, data, algorithms", etc., but also in how to land on everyone's "personalized needs" and "easily available and customizable", and the open source of agents is a key link found by OPPO.

Perhaps, OPPO has seen the irreplaceable important role of large model application ecology early, and will make global efforts to clearly put forward such a long-term idea at the moment of the official release of AndesGPT. This will also continue to be the underlying thinking of OPPO on the road to the development of large models in the future.

Empower the overall ecology of ColorOS, now it's time for AndesGPT

In connection with this release of ColorOS14, we will find that AndesGPT enables the overall ColorOS status, which helps the overall science and technology and intelligence of OPPO, and is also an inevitable path choice. As the core engine that drives OPPO's AI strategy, it is only a matter of time before AndesGPT enables OPPO smart terminals.

AndesGPT has become a "keyword" to understand the present and future of OPPO, and I believe we will see more of this keyword.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.