Shulou(Shulou.com)06/01 Report--

This article introduces what are the unexplored TensorFlow libraries, the content is very detailed, interested friends can refer to, hope to be helpful to you.

TensorFlow is an end-to-end open source machine learning platform capable of performing a range of tasks. It provides an ease of use for beginners and researchers and can be used in different applications such as, but not limited to, computer vision, natural language processing and reinforcement learning.

In the field of computer vision, most of us are familiar with the core of TensorFlow as well as TensorFlow Lite and JS. They are used to run models on mobile and edge devices and on web for the latter. However, TensorFlow also provides more mysterious libraries, which we will explain in this article.

TensorFlow Model Optimization Toolkit

Real-time models are essential for many business operations. MobileNet's speed of reasoning makes it the focus of attention, even if it means sacrificing a little accuracy.

The first thing that comes to mind when optimizing the TensorFlow model is to convert it into a TensorFlow lite service. However, this does not work well on the desktop because it is optimized for ARM neon. This question is explained (https://github.com/tensorflow/tensorflow/issues/35380), otherwise we need to further optimize the model. The model optimization toolkit can help us accomplish these tasks. According to its home page, it can be used to:

Reduce latency and reasoning costs for cloud and edge devices such as mobile devices and the Internet of things.

Deploy the model to the edge device and limit processing, memory, power consumption, network usage, and model storage space.

Support for execution and optimization of existing hardware or new dedicated accelerators.

It can be applied to trained models and can be used to further optimize the solution during training. At the time of writing this article, it provides three technologies, as well as other ongoing techniques to improve the model.

Pruning

The first method is weight pruning. It works by removing some connections between layers, thereby reducing the number of parameters and operations involved, thus optimizing the model. Unnecessary tensors are eliminated in the course of training. This helps to reduce the size of the model, and the size of the model can be further reduced by quantization after training.

Unlike pruning only during training, quantification can be carried out during training and testing. The Tensorflow Lite model is also quantized to use 8-bit integers instead of the commonly used 32-bit floating point. This improves performance and efficiency because integer operations are much faster than floating-point operations.

However, this comes at a price. Quantification is a lossy technique. This means that the information previously represented from-3e38 to 3e38 must be represented from-127to 127. More mistakes are introduced this time. In order to solve this problem, quantification can be applied during training.

Quantitative training

By applying quantification in training, we force the model to learn the differences it will cause and act accordingly. The quantization error is introduced as noise and the optimizer tries to minimize it. The model trained by this method has the same accuracy as the floating-point model. Interestingly, we can compare the Tensorflow-Lite model created in this way with the normal model.

Post-training quantification

Although it is best to apply quantification during training, sometimes this is not feasible because we may have pre-trained weights to use.

Weight clustering

It combines similar weights and replaces them with a value. It can be thought of as JPEG compression. In addition, it is also lossy because similar weights are interpolated to the same number.

The weight matrix originally stores floating-point values. These values are converted to integers, which represent cluster numbers. We then store the lookup table that contains the cluster for query. This reduces the space required because integers require less storage space.

As shown in the following example, 16 float- 32 values are assigned to 4 float-32 centroids, and layer weights are converted to integer values. The larger the weight matrix, the greater the space savings.

TensorFlow Graphics

TensorFlow graphics aims to combine computer vision and computer graphics to solve complex three-dimensional tasks.

Computer graphics workflows require 3D objects and their absolute positions in the scene, descriptions of their materials made up of lights, and cameras that generate composite renderings. On the other hand, the computer vision workflow will start with the image and try to derive its parameters.

This can be thought of as a self-encoder, and the visual system (encoder) will try to find parameters, while the graphics system (decoder) will generate an image based on these parameters and compare it with the original image. In addition, the system does not need to mark data, nor does it need to be trained in a self-supervisory manner. Some uses are:

Transform-you can perform object transformations such as rotation and translation on objects. This can be used to accurately find the position of the object through neural network learning. It is useful for robotic arms that need to accurately estimate the position of these objects.

Modeling camera-different camera internal parameters can be set to change the way the image is perceived. For example, changing the focal length of a camera changes the size of the object.

Materials-different types of materials with different types of light reflection can be used. As a result, the scene you create can accurately simulate the behavior of objects in the real world.

3D convolution and pool (point cloud and grid)-it has 3D convolution and pool layer, which allows us to semantically classify and segment 3D data.

TensorBoard 3D-3D data is becoming more and more common, which can be used to solve the problems of 3D reconstruction, point cloud segmentation, 3D object deformation and so on. With TensorBoard 3D, these results can be visualized to better understand the model.

Read further: https://blog.tensorflow.org/2019/05/introducing-tensorflow-graphics_9.html

TensorFlow Federated

This library can also be used in areas other than computer vision. With the increase of the number of mobile devices and edge devices, a large amount of data has been generated.

The goal of federated learning is to perform machine learning on distributed data, that is, on the device itself! This means that there is no need to upload large amounts of (sensitive) data to the server. It is already in use on Google keyboards.

TensorFlow Privacy

Sensitive information can be extracted from the trained ML model through privacy attacks.

On the left is an image reconstructed using only the name and model. The image on the right is the original image.

Again, like TensorFlow Federated, this is not unique to computer vision. The most commonly used technology is differential privacy.

Differential privacy a system that publicly shares information about a dataset by describing the patterns of groups in the dataset while retaining information about individuals in the dataset.

Assuming that sensitive information is not completely duplicated in the dataset, you can ensure that the model does not learn such information by using a differential privacy model.

For example, suppose you have a dataset for chatting between people. Now, the sensitive information passed in the chat can be passwords, bank account details, and so on. Therefore, if a model is created on this dataset, differential privacy will ensure that the model cannot learn these details because the amount of this information is small. Read this article on differential privacy, which also contains the code that executes it.

TensorFlow Hub

Most of you must know something about this library, so my introduction to it will be very brief.

TensorFlow Hub is a platform for publishing, discovering and reusing some machine learning modules in TensorFlow. It is correct to call it the GitHub of the TensorFlow model.

Developers can share their pre-trained models, which can then be reused by others. Through reuse, developers can use smaller data sets to train models, improve generalization ability, or just speed up training. Let's take a quick look at several different computer vision models available.

Image classification-from MobileNet to Inception to EfficientNet, there are more than a hundred models available for this task. Name any model you want and you'll probably find it there.

Object detection and segmentation-again, any model you need can be found here, especially in the collection of TensorFlow model-zoo object detectors trained on COCO data sets. Deeplab architecture plays a leading role in the field of image segmentation. There are a large number of TfLite and TensorFlow Js models available.

Image styling-different trunks of image styling, as well as a cartoon interest.

* * generate countermeasure networks-* * provide GAN models such as Big GAN and Compare GAN for training on ImageNet and Celeb datasets. There is also a borderless GAN that can be used to generate areas outside the scene captured by the camera. In addition, most of them have a Colab Notebook, so it won't be too difficult to implement them.

What are the unexplored TensorFlow libraries to share here, I hope that the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.