Shulou(Shulou.com)11/24 Report--

The image above is an image generated by Stable Diffusion using text tips: "Super cute fluffy cat warrior in armor, lifelike, 4K, super detail, V-Ray rendering, illusory engine"

Basic models are sweeping the AI industry. The basic model refers to a large-scale neural network that carries out large-scale training based on massive data, which can adapt to a wide range of follow-up tasks with high performance. The popular basic model Stable Diffusion is an excellent generative AI model from text to image, capable of creating realistic images in tens of seconds based on any text input. Stable Diffusion has more than 1 billion parameters and has so far been mainly limited to running in the cloud. Next I'll show how Qualcomm AI Research uses Qualcomm AI software stack (Qualcomm AI Stack) to perform full-stack AI optimization and deploy Stable Diffusion on Android smartphones for the first time.

With full-stack AI optimization, Stable Diffusion runs completely efficiently on the terminal side.

Full-stack AI optimization supported by Qualcomm AI software stack

In the blog article "AI Initiative", we mentioned that Qualcomm AI Research is not only carrying out new AI research work, but also taking the lead in showing proof of concept on commercial terminals, paving the way for large-scale application of technology in the real world. Our full-stack AI research refers to optimization across applications, neural network models, algorithms, software and hardware, and cross-departmental collaboration within the company. For Stable Diffusion, we start with Hugging Face's FP32 1-5 open source model and optimize it through quantification, compilation and hardware acceleration, so that it can run on mobile phones equipped with the second generation Snapdragon 8 mobile platform.

In order to compress the model from FP32 to INT8, we used the post-training quantization of Qualcomm AI Model Enhancement Kit (AIMET). This is a tool based on the technology created by Qualcomm AI Research and has been integrated into the newly released Qualcomm AI Studio. By making the model run efficiently on our dedicated AI hardware and reducing memory bandwidth consumption, quantization can not only improve performance, but also reduce power consumption. Advanced Qualcomm AIMET quantization techniques such as adaptive rounding (AdaRound) can maintain model accuracy at a lower precision level without retraining. These technologies can be applied to all the component models that make up Stable Diffusion, that is, Transformer-based text encoders, VAE decoders and UNet. This is critical to making the model suitable for running on the terminal.

Qualcomm AI software stack brings together the best AI software products into one package to help OEM vendors and developers create, optimize and deploy their AI applications on our products, taking full advantage of the performance of Qualcomm AI engines.

For compilation, we use the Qualcomm AI engine Direct framework to map the neural network to programs that can run efficiently on the target hardware. The Qualcomm AI engine Direct framework performs sequential operations based on the hardware architecture and memory levels of Qualcomm Hexagon processors to improve performance and minimize memory overflows. Some of the above enhancements are the result of cooperation between AI optimization researchers and the compiler engineering team to improve memory management in AI reasoning. The overall optimization of Qualcomm AI engine can significantly reduce the delay and power consumption of runtime, and this much-needed trend also exists in Stable Diffusion.

With tight hardware and software co-design, Qualcomm AI engines with integrated Hexagon processors can unleash industry-leading edge-side AI performance. The latest second-generation Snapdragon 8 mobile platform that supports microslicing reasoning has the ability to run large models like Stable Diffusion efficiently, and the next generation of Snapdragon is expected to bring more enhancements. In addition, because all the component models that make up Stable Diffusion adopt the multi-head attention mechanism, the technical enhancements made to transformer models (such as MobileBERT) to speed up reasoning play a key role.

This full-stack optimization eventually allows Stable Diffusion to run on a smartphone and perform 20-step reasoning in 15 seconds to generate an image of 512x512 pixels. This is the fastest reasoning speed on a smartphone, comparable to cloud latency, and user text input is completely unrestricted.

Qualcomm AI Studio integrates all our current tools into a new GUI, as well as visualization tools to simplify the developer experience.

The image above is an image generated by Stable Diffusion using text tips: "Japanese gardens in wild valleys and mountains, high detail, digital illustrations, ArtStation, conceptual art, frosting, sharp focus, illustrations, dramatic, sunset, hearthstone, works of art by artgerm, greg rutkowski and lphonse mucha".

The era of marginal AI has come.

As the AI cloud model begins to run on edge terminals, Qualcomm's vision of building a smart Internet connection edge is accelerating before our eyes, and what was considered impossible a few years ago is becoming possible. This is attractive because terminal-side processing through edge AI has many advantages, including reliability, latency, privacy, network bandwidth efficiency, and overall cost.

Although the Stable Diffusion model looks too large, it encodes a lot of language and visual knowledge and can generate almost any picture imaginable. In addition, as a basic model, Stable Diffusion can do much more than generate images based on text prompts. Stable Diffusion-based applications are increasing, such as image editing, image restoration, style conversion and super-resolution, which will have a real impact. Being able to run the model completely on the terminal without having to connect to the Internet brings unlimited possibilities.

Extended edge side AI

Running Stable Diffusion on a smartphone is just the beginning. All the full-stack research and optimization that make this goal possible will be integrated into the Qualcomm AI software stack. With Qualcomm's unified technology roadmap, we can leverage a single AI software stack and extend it to different endpoints and different models.

This means that the optimizations made to make Stable Diffusion run efficiently on mobile phones can also be used on other Qualcomm enabled platforms, such as laptops, XR headers and almost any other terminal. Running all AI processing in the cloud is expensive, so efficient edge-side AI processing is important. Since there is no need to leave the terminal to enter text and generate images, edge-side AI processing ensures user privacy when running Stable Diffusion (and other generative AI models), which is of great benefit to both consumer and enterprise applications. The new AI software stack optimization will also help reduce time to market for the next generation of basic model products running on the edge side in the future. This is how we can scale across endpoints and underlying models so that the edge-side AI is truly ubiquitous.

At Qualcomm, we have made breakthroughs in basic research and expanded across terminals and industries to enable the edge of intelligent networking. Qualcomm AI Research works with all the company's teams to integrate the latest AI developments and technologies into our products, so that the AI advances achieved by laboratory research can be delivered faster and enrich people's lives.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.