Shulou(Shulou.com)06/01 Report--

PyTorch publishes prototype functions to perform sample analysis of hardware engines on machine learning model devices. I believe many inexperienced people are at a loss about this. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

PyTorch recently released four new PyTorch prototype features. The first three functions enable mobile machine learning developers to execute models on the full set of hardware (HW) engines that make up system-on-a-chip (SOC) systems. This allows developers to optimize their model execution to achieve unique performance, functionality, and system-level concurrency.

The new features include the following functions that enable the hardware engine on the device to perform:

Use DSP and NPU of Android neural network API (NNAPI) developed in cooperation with Google Android.

Execute GPU on Android through Vulkan

Execute GPU on iOS through Metal

The use of ARM is increasing in the PyTorch community with Raspberry Pis and Graviton (2) platforms. As a result, the new version also includes developer productivity benefits, as well as recently launched ARM64 build support for Linux.

NNAPI support for Google Android

PyTorch cooperated with the Google Android team to implement Android's neural network API (NNAPI) through PyTorch Mobile. On-device machine learning allows the ML model to run locally on the device without having to transfer data to the server. This reduces latency and improves privacy and connectivity. The Android neural network API (NNAPI) is designed to run compute-intensive processes for computer learning on Android gadgets. As a result, the machine learning model now has access to other hardware modules on the phone's system-on-chip, allowing developers to unlock high-performance execution on Android phones. NNAPI enables Android applications to run computationally accelerated neural networks, including DSP (digital signal processor) and NPU (dedicated neural processing unit), on the most powerful and active chips that power android.

The API was originally introduced in Android 8 and significantly extended in Android 10 and 11 to support richer AI models. This integration allows developers to access NNAPI directly from PyTorch Mobile. This initial release includes full support for a set of core features and operators. Google and Facebook will soon be working to expand their capabilities.

PyTorch Mobile GPU support

GPU inferences can provide excellent performance on many model types, especially those that use high-precision floating-point mathematical operations. As found in Qualcomm, MediaTek and Apple's SOC, using GPU to perform machine learning models all support CPU offload. This releases mobile CPU for non-machine learning use cases. The main level of prototype assistance for device GPU is through iOS's Metal API specification and Android's Vulkan API specification. The performance of this feature has not been optimized, and the coverage of the model is limited because it is still in its immature stage. The team expects this situation to improve significantly in 2021.

After reading the above, have you mastered the PyTorch release prototype function to perform sample analysis of hardware engines on machine learning model devices? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.