Shulou(Shulou.com)06/01 Report--

This article mainly shows you "what are the prototype functions of PyTorch", the content is easy to understand, clear, hope to help you solve your doubts, the following let the editor lead you to study and learn "what are the prototype functions of PyTorch" this article.

A prototype function of PyTorch: the neural network API (NNAPI) that supports Android. PyTorch Mobile is designed to provide ML developers with a best-in-class experience and high-performance execution on all mobile hardware. NNAPI support is critical to achieving this goal because it extends the set of hardware that we can use to quickly execute the model. This initial release includes full support for a small but powerful set of features and operations, which we will expand in the coming months.

NNAPI allows Android applications to run computationally intensive neural networks on the most powerful and efficient components of the mobile chip, including GPU (graphics processing unit) and NPU (dedicated neural processing unit). It was introduced in Android 8 (Oreo) and expanded significantly in Android 10 and 11 to support a richer set of AI models. Although NNAPI provides a convenient and unified interface for various hardware drivers, it is a low-level API that requires a lot of integration work. Advanced frameworks like PyTorch make these benefits available to more application developers.

NNAPI does not have a local disk model format, so we chose to encapsulate the model definition in a normal TorchScript model. Developers will prepare their models after training and then execute them on NNAPI. The saved model can then be packaged into an Android application (or delivered over the network) and then loaded and run using PyTorch Mobile's Java API or libtorch C++ API. For applications that already use PyTorch Mobile, there is no need to change the code. Developers can simply replace their CPU model with the NNAPI model.

For PyTorch developers, NNAPI that accesses and uses Android is particularly attractive to real-time models that are always online, such as computer vision on the device. These models tend to be computationally intensive, delay-sensitive and energy-intensive. Meeting these three requirements at the same time is a challenge, which makes these models the best choice to take advantage of hardware acceleration. This is one of the reasons why Facebook is interested in NNAPI. The AI model that supports virtual background experiences on portal devices is now being tested with NNAPI in Messenger applications to enable immersive 360-degree background functionality.

As shown in Table 1, using Android's NNAPI for this model on Pixel 3 can achieve high performance between single-core CPU and 2-core CPU, thereby releasing CPU for non-ML application code. In addition, NNAPI supports higher levels of performance on new devices such as Pixel 4 and Pixel 5, which can be leveraged by more complex ML models.

Table 1-immersive 360-degree background enabled model benchmark

Similarly, many real-time audio models have been introduced into mobile applications. Due to similar concurrency, latency and power requirements, features such as background noise reduction will also benefit from NNAPI-based hardware acceleration.

Switching machine learning models between frameworks or API is always tricky, and this time is no exception. Although both PyTorch and NNAPI are developed to run the same type of neural network, there are many small semantic differences that need to be bridged when switching from one to another. For example:

NNAPI uses integer offsets for quantized convolution operations, while PyTorch uses floating-point operations. PyTorch and NNAPI expect different memory ordering for weight tensors in convolution. Before PyTorch, there was an internal representation of a complex upsampling operation, which must be simplified in order to make it easier to convert to NNAPI. PyTorch and NNAPI have different NHWC tensor representations. NNAPI only supports continuous tensors, so an explicit NHWC representation is required. PyTorch supports the stride tensor, so the convention always uses NCHW, but you can choose to combine channels-last memory format to get NHWC behavior.

This is the first prototype version supported by NNAPI in PyTorch and supports the well-known linear convolution and MLP model when deployed on Android 10 and above. The upcoming PyTorch will add additional features, such as:

Additional operators are supported to unblock additional model types. Support for acceleration models based on Mask R-CNN architecture. For more information about Mobile Mask R-CNN, please visit https://research.fb.com/blog/2018/01/enabling-full-body-ar-with-mask-r-cnn2go/. Earlier Android versions, 8 (Oreo) and 9 (Pie), are supported. Models that take advantage of control flow semantics are supported. When NNAPI is available, the model can be used on the user's Android phone, otherwise it will automatically fall back to the CPU for execution.

Using the phones shown in Table 1, Table 2 shows the results of benchmarking the open source MobileNetV2 model, which is similar to that seen in the enabling 360-degree background model, with similar CPU loads and even more significant performance benefits.

Table 2-MobileNetV2 Model Datum

The above is all the content of the article "what are the prototype functions of PyTorch". Thank you for reading! I believe we all have a certain understanding, hope to share the content to help you, if you want to learn more knowledge, welcome to follow the industry information channel!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.