Shulou(Shulou.com)11/24 Report--

Since the beginning of this year, the generative AI model track represented by ChatGPT has continued to be hot, and technology companies have rushed to the beach one after another to come up with their own AI model products one after another. Not long ago, smartphone brand Xiaomi also released its own AI model, and then launched Xiao Ai's version in September.

In addition, at the 2023 China International Trade in Services Fair held at the beginning of this month, generative AI also became a key hot word. Many enterprises, such as Zhisu AI and APUS, showed AI large model products, and everyone experienced the "meta-universe experience center" created by AIGC and digital human technology.

In the results release platform of this service trade fair, Qualcomm issued a printed version of the artificial intelligence AI white paper-"Hybrid AI is the Future of AI". In the white paper, Qualcomm combed its insights into technology and development in the field of AI, with the aim of exploring the road of AI inclusive with more partners.

At the same time, Hou Mingjuan, global vice president of Qualcomm, said at the meeting that terminal-side AI is the key to achieving hybrid AI architecture and expanding generative AI to a wider global scope.

Coincidentally, the Xiaomi AI model we just mentioned did not focus on expanding the super-large parameters, but tried to put the 1.3 billion-parameter model to run locally on the user's phone.

It can be seen that the terminal side of AI is causing widespread attention in the industry, and will play an extremely important role in the next development of AI technology and industry.

With the deepening development of generative AI, the advantages and value of terminal side highlight that terminal side AI is so important because it can alleviate the problems of cloud AI in terms of cost, energy consumption, performance, privacy, security and personalization, especially with the rapid development of generative AI. These problems will be particularly prominent.

For example, first of all, in terms of cost. It is estimated that the cost of each web search query based on generative AI is 10 times that of traditional search, and this is only one of many applications of generative AI. If these operations, processing are put in the cloud, no matter for the size of search engine enterprises, it is a very uneconomical cost burden.

By moving some processing from the cloud to the edge side terminals, you can reduce the pressure on the cloud infrastructure and reduce expenses. Like Qualcomm, they can take advantage of billions of AI-capable edge-side terminals that have been deployed for computing, greatly reducing the burden on cloud infrastructure.

For example, in terms of energy consumption, edge terminals can run the generative AI model with very low energy consumption, especially when combining processing and data transmission, which can help cloud service providers significantly reduce the energy consumption of the data center.

Terminal-side AI can also enhance the reliability of the entire AI application system. For example, when people use some large model products, they often encounter slow response or even generation failure during peak hours. However, with the terminal-side AI, a considerable part of the computing load can be transferred to the terminal side, so that when there is a demand for the cloud for the spanning AI query, it is more likely to avoid peak congestion, thus effectively reducing queuing and high latency, and even denial of service.

In addition, terminal-side AI essentially helps to protect user privacy, because sensitive data and information can be retained on the terminal, which is very important for both enterprise and individual users.

When privacy security is guaranteed, the immediate benefit is that personalized experience becomes possible.

Because service providers can provide corresponding services based on users' personalized data to the maximum extent without sacrificing users' privacy. For example, AI assistants on our mobile phones can have a more in-depth understanding of our habits, interests, and needs, so as to draw more accurate user portraits to provide personalized services. And these user portraits are retained in the terminal.

In short, on the road of future development of AI, terminal-side AI is as important as cloud-based AI, which, together with cloud-based AI, will really release the inclusive value that the development of generative AI technology can bring to the public, and the wider and deeper the application and popularization of generative AI, the more prominent the industry potential and value of terminal-side AI, and profoundly change all aspects of our production and life.

Qualcomm proactive layout generation AI, we always heard "AI changes the world" in the past, but in practical application, the perception is not very strong, but the emergence of generative AI seems to let people have a preliminary and real experience of "AI changing the world".

Imagine that in the future, our mobile phones will become more concise and intelligent because of the generative AI, and become real personal assistants; PC will become a more powerful productivity tool, greatly speeding up our daily workflow, and tasks that used to take hours or days can be done in a matter of minutes. And cars can also become highly personalized spaces, helping you plan your travel routes and provide customized music and blog experiences.

Of course, these only exist in our imagined future. If we want to turn this ideal into a reality and let AI really change the world, we need to further promote the large-scale expansion and application of generative AI. Behind this, based on the hybrid AI architecture, especially the terminal-side AI empowerment, is the key.

In the industry, there happens to be one company that has been committed to this, that is, Qualcomm.

Qualcomm has been involved in AI for more than 15 years and has provided industry-leading AI hardware and software solutions for edge terminals such as billions of mobile phones, cars, XR headsets and glasses, PC and the Internet of things, with considerable leadership in terminal-side AI.

In terms of software, Qualcomm first has strong leadership in terminal-side generative AI. Their research on generative AI can be traced back to spanning counternetwork (GAN) and variational self-encoder (VAE). Using VAE technology, Qualcomm has created better video and voice codecs to control the model size to less than 100 million parameters.

Recently, Qualcomm has implemented a generative AI model that supports more than 1 billion parameters on the terminal side, such as Stable Diffusion, and plans to support tens of billions of parameters on the terminal side in the future.

They are not only studying how to use generative AI models as general agents to build computing architectures and using languages to describe related tasks and behaviors, but also how to further develop this capability and environmental interaction capabilities by increasing perceptual inputs (such as visual and audio), such as generating instructions to robots or running software.

In terms of the key terminal-side AI, Qualcomm leads the reasoning of terminal-side AI through AI hardware acceleration and simplified development of software solutions. Such an AI acceleration architecture is flexible and robust and can cope with potential changes in the architecture of generative AI models. As the big language model and other generative AI models continue to evolve, Qualcomm's AI technology architecture will continue to evolve. The ability to easily develop hybrid AI applications is key, and their common AI architecture and AI tools across product portfolios are designed for this future.

1 billion parameter model is applied to the mobile terminal, but at the beginning, what kind of solution does Qualcomm adopt to realize the evolution of terminal-side generated AI?

The answer is that Qualcomm has conducted full-stack AI research and optimization for applications, neural network models, algorithms, software and hardware. In June last year, Qualcomm launched a leading software stack product specifically for edge-side AI, Qualcomm AI software stack, with excellent terminal-side AI technology, using heterogeneous computing methods to accelerate terminal-side AI using hardware and software to jointly develop the most optimized solution.

This is also due to Qualcomm's unique ability in AI basic research to support them in the development of full-stack terminal-side AI, rapid launch of enabling products and optimized deployment around key applications such as terminal-side generative AI.

For example, in the first half of this year, Qualcomm's Stable Diffusion terminal-side demonstration on the world's first Android phone highlighted the advantages of Qualcomm's full-stack strategy.

Stable Diffusion is a super-large neural network basic model with more than 1 billion parameters, which can generate pictures based on input text prompts. This terminal-side demonstration is conducted in flight mode. Through Qualcomm's full-stack AI optimization, this model can run completely on the terminal side, completing 20-step reasoning in 15 seconds and generating images full of details.

Qualcomm has made full-stack AI optimization for Stable Diffusion. For example, in terms of algorithm and model development efficiency, the previous models trained on cloud servers generally use 32-bit floating-point operations (FP32), which means that a lot of processing work is needed to complete model reasoning. For Stable Diffusion, Qualcomm uses 8-bit integer operation (INT8). At the end of last year, the second generation Snapdragon 8 mobile platform further supported 4-bit integer computing (INT4) capability, which will greatly improve the efficiency of AI computing.

In addition, through Qualcomm's AI engine Direct, Qualcomm is able to leverage hardware capabilities in the most efficient way, combined with the second generation of Snapdragon 8 industry-leading Hexagon processors, which will bring generative AI capabilities far ahead of this use case on the terminal.

Qualcomm AI software stack provides Qualcomm partners, users and developers with an integrated platform that integrates all AI frameworks, developers' libraries and operating systems, allowing them to create an excellent experience on terminals equipped with Snapdragon platform. At the same time, they can use all the tools provided by Qualcomm AI Studio.

On the hardware side, Qualcomm's hardware provides industry-leading energy efficiency, nearly twice as much as competitors in the mobile sector. Among them, Qualcomm AI engine is composed of a number of software and hardware components, which can achieve terminal-side AI acceleration on Snapdragon and Qualcomm platforms.

Qualcomm AI engine uses heterogeneous computing architecture, including Hexagon processor, Qualcomm Adreno GPU and Qualcomm Kryo CPU, all designed to run AI applications quickly and efficiently on the terminal side. Through heterogeneous computing, developers and OEM vendors can optimize the AI user experience on smartphones and other edge-side terminals.

On CVPR 2023 in June, Qualcomm also demonstrated an image generation model ControlNet with an order of magnitude of 1.5 billion, which is a generative AI painting solution that controls pre-training models such as Stable Diffusion through additional input, which can finely set the details of the generated image. First input a reference picture, and then pre-process according to the input prompt, the generated image can be accurately controlled.

In Qualcomm's demo, an ordinary Android phone can run ControlNet in 11.26 seconds to generate a picture, and it is completely localized, efficient and fast.

The outstanding performance of the ControlNet model architecture is also due to Qualcomm's hardware and software full-stack AI optimization of the model, which is supported by Qualcomm AI Model Enhancement Kit, Qualcomm AI engine and Qualcomm AI software stack.

Next, Qualcomm is planning to support models with tens of billions of parameters on the terminal side in the future, which means that a very large-scale AI model will be running on the terminal side soon, which will also become a major differentiation advantage of products based on Qualcomm technology.

In addition to the above, there is another point that can not be ignored, that is, the scale of edge-side terminals deployed by Qualcomm is very large, and the number of listed user terminals equipped with Snapdragon and Qualcomm platforms has reached billions. And hundreds of millions of new terminals are still entering the market every year. These terminals cover a wide range of products, including mobile phones, cars, XR, PC and the Internet of things and so on.

In short, Qualcomm relies on industry-leading hardware to support higher performance at given power consumption; the industry-leading Qualcomm AI software stack; and industry-leading tools such as Qualcomm AI Model Enhancement Kit (AIMET), constitute its unique advantage in enabling hybrid AI scale expansion on a global scale.

Conclusion the popularity of generative AI will not be a flash in the pan, but a technological trend that can really change the way we live and produce. According to forecasts, the AI application rate in market segments such as smartphones, PC / tablets, XR, cars and the Internet of things will increase from less than 10 per cent in 2018 to 100 per cent in 2025.

For Qualcomm, this is also one of their directions. With its strong leadership in terminal-side AI, Qualcomm is transforming to a hybrid AI architecture based on its unique advantages. They have more than 15 years of forward-looking research and product investment in the AI field, excellent terminal-side AI technology and full-stack optimization system, as well as an extensive and large number of global edge-side terminal layout and scale.

Next, what they need to do is to more closely support developers, OEM vendors and other ecosystem innovators to build new generative AI applications and solutions quickly and efficiently, so that the largest number of users can enjoy the convenience of generative AI more quickly.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.