Shulou(Shulou.com)11/24 Report--

Even if 2023 has entered the countdown, we are still witnessing the rapid change of the AI large model industry. An ecological soil that is constantly giving birth to the brand-new vitality of large models is coming at an accelerated pace.

Don't let the spark of technology wait, a resounding official announcement.

On November 16, the 2023OPPO developer Conference was held at the Shanghai World Expo Center, at which OPPO officially released the self-training personality model and agent-the Andean Great Model (AndesGPT).

Taking "end-cloud collaboration" as the infrastructure design idea, AndesGPT offers model specifications with different parameter sizes ranging from 1 billion to more than 100 billion, which can flexibly support diversified application scenarios based on "end-cloud division of labor, end-cloud complementarity, end-cloud collaboration" and other ways. AndesGPT emphasizes the technical features of "dialogue enhancement, individuality, and end-cloud collaboration". It can be seen that OPPO has some of his own ideas on the big model track.

At the press conference, OPPO pointed out that this is the "era of technology-driven experience", believing that the large model will comprehensively revolutionize the interactive experience of smart terminals and continue to bring high-quality experience to users in this direction. At the same time, in the aspect of strengthening the technological capability, OPPO will lay out the large model forward-looking technology, jointly with the Intelligent Computing Joint Laboratory established by the University of Science and Technology of China, and has taken the large model as the core technology research direction. At the same time, OPPO will also open source Agent framework to support efficient incubation, hosting and application of agents.

AI track has entered the full competition, and AndesGPT will come firmly for everyone.

The announcement of this series of big moves once again confirms the era of all-round competition in the current big model.

The press conference starts with "smart experience", emphasizing "more useful, more professional, more exclusive". For example, the new Xiao Bu, released with ColorOS 14, has a wealth of agents built in, and different agents provide different scene experiences for users.

In this way, OPPO sets out the goal of AndesGPT: to create useful large models and agents for everyone, and will also open source the agent framework to create an one-stop "agent development platform".

According to reports, AndesGPT has three categories of specifications: AndesGPT-Tiny, AndesGPT-Turbo and AndesGPT-Titan, which can cover more than one billion to hundreds of billions of model specifications with different parameter sizes, and intelligently schedule according to different scenarios. In AndesGPT's network architecture, OPPO also proposed two combined innovations: "RoPE location coding to explore the optimal value of base"-- to improve the upper and lower "expansion energy", and "GQA combined with complex mobile window to accelerate training and reasoning"-- to achieve the linear complexity of O (Nd).

The four core competencies are highlighted, and AndesGPT enables multiple large model application scenarios.

AndesGPT focuses its core competencies on knowledge, memory, tools and creation.

In terms of knowledge ability, AndesGPT provides the ability to "integrate knowledge base, knowledge graph and general search to provide more professional question and answer". At the same time, it also integrates external knowledge and model into results by identifying knowledge dependence module, accurately identifying external knowledge reality, and introducing accurate recall mechanism, so as to reduce the occurrence of "hallucination".

In terms of memory ability, AndesGPT has a chronological memory mechanism, which can hold the upper and lower levels of the limit. Long-term memory mainly includes interaction history, personal data and structured information extracted from the process of user interaction. In order to solve the problem of the length of first word reasoning introduced by long-term memory, OPPO has developed a mechanism called SwappedAttention.

This mechanism has a certain degree of leadership and originality. Compared with PagedAttention, it has made the evolution and expansion of technology on the basis of PagedAttention.

For example, in the scene of first word generation, it is necessary to encode the history sequence. When the history sequence is long, it is easy to appear computing bottleneck, which is characterized by long time consuming and low concurrency. SwappedAttention caches historical KV values, greatly reducing the amount of headword computation. And use GPU video memory, host memory, GDS connection external storage and other multi-level cache methods, according to the cache time, conversation frequency and other strategies for hierarchical storage and exchange.

In practical applications, such as the common "multi-round conversation scenario", with the accumulation of chat rounds, the Prompt becomes longer and longer, and the first word time of each Query becomes longer and longer. SwappedAttention can cache historical KV values to multi-level cache, which can effectively reduce the time of first word inference during conversation and make users get faster system response.

In the "FileChat document dialogue scene", SwappedAttention can read the historical calculated KV value information, avoid the KV value re-calculation of long text, greatly reduce the computing overhead, shorten the first word reasoning time, and can improve the concurrency of the first word calculation; in the non-first word reasoning process, SwappedAttention can dynamically compress the KV value, further reduce the video memory footprint and improve the overall throughput.

In the use of tools, AndesGPT introduces the code and "book advance" in the training phase to strengthen training, and through instruction construction and fine-tuning to improve the effect of the model following instructions. At the same time, it also holds all kinds of tools, such as "system settings," three services, code interpreter, and so on.

In terms of creative ability, in addition to the continuous improvement of "dialogue" ability, AndesGPT also has the ability of "life picture, picture picture, Chinese style, high-definition wallpaper" and so on, and has supported music generation throughout the process. Previously, Xiao Bu just released the single "belongs to you: Brooks will always accompany", its lyrics, melodies, timbre and other elements are generated by AndesGPT.

Determined by the collaboration between the end and the cloud, AndesGPT methodically unfolds the future of the large model.

AndesGPT provides intelligent services to users through collaborative end-to-end cloud design: end-to-side solutions have obvious advantages, which can improve the scalability in extreme scenarios such as task execution and degraded response; at the same time, OPPO also provides trusted large model services in the cloud, which will be combined with secret computing technology to build encrypted and trusted channels to fully protect user privacy.

Not long ago, OPPO and MediaTek announced a "joint lightweight model end-to-side deployment plan". We can see the full consideration and thoughtful consideration of OPPO to build "end-cloud collaboration and full stack optimization" from the bottom, which is constantly laying a solid foundation for future technology development.

Think of the new OPPO Find N3 just released, several "ceiling" level of "security and privacy" design is really fascinating, it can be seen that OPPO has been working hard in the field of security, presumably the security and credibility of the large model field will also have its own unique technology accumulation to protect the data of customers.

Not afraid of the difficulty of Shu Road, the official announcement of "flashing the Fire of Ecological Hope of AI in China"

From this conference full of practical information, we can see that OPPO has its own thinking on the big model, hoping to put the big model into practice and build a large model application ecology that everyone can participate in. The difficulty of the big model lies not only in "computing power, data, algorithms", etc., but also in how to land on everyone's "personalized needs" and "easily available and customizable", and the open source of agents is a key link found by OPPO.

Perhaps, OPPO has seen the irreplaceable important role of large model application ecology early, and will make global efforts to clearly put forward such a long-term idea at the moment of the official release of AndesGPT. This will also continue to be the underlying thinking of OPPO on the road to the development of large models in the future.

In connection with this release of ColorOS14, we will find that AndesGPT enables the overall ColorOS status, which helps the overall science and technology and intelligence of OPPO, and is also an inevitable path choice. As the core engine that drives OPPO's AI strategy, it is only a matter of time before AndesGPT enables OPPO smart terminals.

The fire of technology will explode with the participation of everyone. Perhaps in the near future, we will be able to see a thriving, big model built by OPPO and developers.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.