Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "what are the important types of neural networks in artificial intelligence". The editor shows you the operation process through practical cases, and the operation method is simple, fast and practical. I hope this article "what are the important types of neural networks in artificial intelligence" can help you solve the problem.

1. Feedforward neural network

This is the most basic type of neural network, which is largely due to technological advances, and it enables us to add more hidden layers without worrying too much about computing time. Since Geoff Hinton discovered the back propagation algorithm in 1990, it has also become popular.

This type of neural network is essentially composed of an input layer, a number of hidden layers and an output layer. There is no cycle, information only flows forward. Feedforward neural network is usually suitable for supervised learning of numerical data, although it also has its disadvantages:

It cannot be used with sequential data.

It is not suitable for image data, because the performance of the model depends heavily on features, and manually finding the features of image or text data is a very difficult exercise.

This brings us to the next two types of neural networks: convolution neural networks and cyclic neural networks.

two。 Convolution neural network (CNN)

Before CNN became popular, there were many algorithms for image classification. People used to create features from images and then input these features into some classification algorithms, such as SVM. Some algorithms also use the pixel value of the image as the feature vector. For example, you can train a SVM with 784 features, each of which is the pixel value of the 28x28 image.

Why CNN and why do they work better?

CNN can be thought of as an automatic feature extractor in an image. Although if I use an algorithm with pixel vectors, I will lose a lot of spatial interaction between pixels, and CNN effectively uses adjacent pixel information to effectively undersample the image first by convolution, and then use the prediction layer at the end.

This concept was first proposed by Yann le cun in 1998 for digital classification, where he uses a single convolution layer to predict numbers. It was later promoted by Alexnet in 2012, using multiple convolution layers to achieve state-of-the-art predictions on Imagenet. Thus, they become the preferred algorithms for image classification challenges in the future.

With the passage of time, a variety of progress has been made in this particular field. Researchers have proposed a variety of architectures for CNN, such as VGG, Resnet, Inception, Xception and so on. These architectures continue to promote the latest technological development of image classification.

By contrast, CNN is also used for object detection, which can be a problem because in addition to classifying the image, we also want to detect the bounding boxes around the various objects in the image. In the past, researchers have proposed many architectures, such as YOLO, RetinaNet, Faster RCNN, etc., to solve the problem of object detection, all of which use CNN as part of their architecture.

3. Cyclic neural network (LSTM/GRU/Attention)

What CNN means to images and what cyclic neural networks mean to text. RNN can help us learn the sequential structure of the text, where each word depends on the previous word or a word in the previous sentence.

For the simple explanation of RNN, the RNN unit is regarded as a black box, a hidden state (a vector) and a word vector are input, and an output vector and the next hidden state are given. This box has some weights that need to be adjusted using loss back propagation. In addition, the same cell is applied to all words so that weights can be shared between words in a sentence. This phenomenon is called weight sharing.

Cyclic neural network diagram

The following is an extended version of the same RNN unit, where each RNN unit runs on each word tag and passes the hidden state to the next unit. For 4-length sequences such as "the quick brown fox", the RNN unit finally gives four output vectors that can be connected and then used as part of the dense feedforward architecture, as shown below to solve the final task language modeling or classification task:

Long-term and short-term memory network (LSTM) and gated cycle unit (GRU) are a subclass of RNN, which introduce various gates to memorize information for a long time (also known as vanishing gradient problem). These gates regulate cell state by adding or deleting information.

From a very high point of view, you can think of LSTM/GRU as a game of learning long-term dependencies on RNN units. RNN/LSTM/GRU is mainly used for various language modeling tasks, and its goal is to predict the next word given the input word flow, or for tasks with sequential patterns. If you want to learn how to use RNN for text classification tasks, check out this article.

The next thing we should talk about is attention-based models, but we only talk about intuition here, because delving into these models can gain very technical knowledge (you can check out this article if you're interested). In the past, traditional methods such as TFIDF/CountVectorizer were used to find features from text by keyword extraction. Some words are more helpful than others in determining the category of the text. However, in this approach, we have lost a bit of the sequential structure of the text. Using LSTM and deep learning methods, we can deal with sequence structures, but we lose the ability to give more weight to more important words. Can we have the best of both worlds? The answer is yes. In fact, attention is what you need. In the words of the author:

"not all words have the same contribution to the meaning of a sentence. Therefore, we introduce an attention mechanism to extract words that are important to the meaning of a sentence and aggregate the representations of these information words to form a sentence vector."

4. Transformers

Transformer has become the de facto standard for any natural language processing (NLP) task, and the recently launched GPT-3 converter is by far the largest.

In the past, LSTM and GRU architectures and attention mechanisms were the most advanced methods for language modeling problems and translation systems. The main problem with these architectures is that they are circular in nature, and the running time increases with the length of the sequence. That is, these architectures take a single sentence and process each word sequentially, so as the length of the sentence increases, so does the overall run time.

Transformer, the first model architecture attention explained in this paper, is what you need to abandon this cycle and rely entirely on the attention mechanism to draw the global dependency between input and output. This makes it faster and more accurate, and becomes the preferred architecture for solving various problems in the field of NLP.

5. Generate adversarial network (GAN)

People in data science have recently seen a lot of AI-generated people, whether in papers, blogs, or videos. We have reached a stage where it is becoming more and more difficult to distinguish between real human faces and faces generated by artificial intelligence. And all this is done through GAN. GAN is likely to change the way we generate video games and special effects. Using this approach, you can create realistic textures or characters on demand, opening up a world full of possibilities.

GAN usually uses two duel neural networks to train computers to learn the properties of data sets well enough to generate convincing fakes. One of the neural networks generates false images (generator), and the other neural network tries to classify which images are fake (discriminator). These networks improve over time by competing with each other.

Perhaps it is best to think of the generator as a robber and the discriminator as a policeman. The more a robber steals, the better his ability to steal. At the same time, the police are better at catching thieves.

The losses in these neural networks mainly depend on the performance of another network:

The discriminator network loss is a function of the generator's network quality: if the discriminator is fooled by the generator's false image, its loss will be high.

The generator network loss is a function of the discriminator network quality: if the generator cannot deceive the discriminator, the loss is high.

In the training phase, we train the discriminator and generator network in turn in order to improve the performance of both. The ultimate goal is to get weights that help the generator create realistic images. Finally, we will use the generator neural network to generate high-quality false images from random noise.

6. Self-encoder

The self-encoder is a deep learning function, which is similar to the mapping from X to X, that is, input = output. They first compress the input feature into a low-dimensional representation, and then reconstruct the output from that representation.

In many places, this representation vector can be used as a model feature for dimensionality reduction.

The self-encoder is also used for anomaly detection, and we try to reconstruct our example using the self-encoder. If the reconstruction loss is too high, we can predict that the example is an exception.

This is the end of the content about "what are the important types of neural networks in artificial intelligence". Thank you for your reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.