Shulou(Shulou.com)06/02 Report--

This article introduces the knowledge of "how perceptrons are produced in machine learning". In the operation of actual cases, many people will encounter such a dilemma. Next, let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

The birth of Perceptron-- Learning from samples

The AI pioneers of neural networks have been struggling with the mapping of neurons and the way they are connected to each other. Frank Rosenblatt of Cornell University was one of the first people to imitate the architecture of the human automatic pattern recognition visual system.

He invented a seemingly simple network perceptron (perceptron), a learning algorithm that can learn how to classify patterns, such as identifying different letters in the alphabet. * * an algorithm is a step-by-step process to achieve a specific goal, just like a recipe for baking a cake.

If we understand how perceptrons learn the basic principles of pattern recognition, we are half successful in understanding how deep learning works. The goal of the perceptron is to determine whether the input pattern belongs to a category in the image (such as a cat).

The figure above explains how the input of the perceptron is converted from the input unit to the output unit through a set of weights. Weight is a measure of the impact of each input on the final decision made by the output unit, but how do we find a set of weights that can correctly classify the input?

The traditional way to solve this problem is to set the weight manually according to the analysis or specific program. This takes a lot of manpower and often depends on intuition and engineering methods. Another approach is to use an automated process of learning from samples, in the same way we know objects in the world. Many samples are needed to train the perceptron, including negative samples that do not belong to this category, especially those that are similar to the target features, for example, if the target is a cat, then the dog is a similar negative sample. These samples are passed to the perceptron one by one, and if there is a classification error, the algorithm will automatically correct the weight.

Perceptron specific algorithm

The beauty of this perceptron learning algorithm is that if such a set of weights already exists and there are a sufficient number of samples, then it must be able to find an appropriate set of weights automatically. After each sample in the training set is provided and the output is compared with the correct answer, the perceptron learns progressively. If the answer is correct, then the weight will not change. But if the answer is incorrect (0 is misjudged as 1, or 1 is misjudged as 0), the weight is adjusted slightly so that the next time it receives the same input, it will be closer to the correct answer. This gradual change is important so that the weight can receive influence from all the training samples, not just the last one.

The perceptron is a neural network with a single artificial shen element, which has an input layer and a set of connections that connect the input unit to the output unit. The goal of the perceptron is to classify the patterns provided to the input unit. The basic operation performed by the output unit is to multiply each input (xn) by its connection strength or weight (wn) and pass the sum of the product to the output unit. In the figure above, the weighted sum of input (∑ iSum 1, … , n wi xi) is compared with the threshold θ and the result is passed to the step function. If the sum exceeds the threshold, the step function outputs "1", otherwise it outputs "0". For example, the input may be the intensity of the pixels in the image, or, more commonly, features extracted from the original image, such as the outline of the object in the image. Each time you enter an image, the perceptron determines whether the image is a member of a category, such as a cat. The output can only be in one of two states. If the image is in a category, it is on, otherwise it is off. On and off correspond to 1 and 0 of the binary values, respectively.

The perceptron learning algorithm can be expressed as follows:

Geometric interpretation of how Perceptron distinguishes two categories of objects

If this explanation of perceptron learning is not clear enough, we can also use a more concise geometric method to understand how perceptrons learn to classify input. For the special case of only two input units, the input sample can be represented by points on the two-dimensional graph. Each input is a point in the graph, and the two weights in the network determine a straight line. The goal of perceptron learning is to move this line in order to clearly distinguish between positive and negative samples. In the case of three input units, the input space is three-dimensional, and the perceptron specifies a plane to separate the positive and negative training samples. In general, the same principle holds even though the dimension of the input space may be quite high and cannot be visualized.

These objects have two characteristics, such as size and brightness, which are drawn on each picture according to their respective coordinate values (x _ ray y). The two objects (plus sign and square) in the figure on the left can be separated by a straight line between them; the perceptron can learn how to make this distinction. The two objects in the other two diagrams cannot be separated by a straight line, but in the middle diagram, the two objects can be separated by a curve. On the other hand, the objects in the figure on the right must discard some samples in order to be separated into two types. If there is enough training data, the deep learning network can learn how to distinguish the types in the three graphs.

Finally, if the solution is feasible, the weight will no longer change, which means that the perceptron has correctly classified all the samples in the training set.

However, in the so-called "overfitting", there may not be enough samples, and the network only remembers specific samples and cannot extend the conclusions to new samples. To avoid overfitting, the key is to have another set of samples, called "test set", which is not used to train the network. At the end of the training, the classification performance on the test set is the true measure of whether the perceptron can be extended to new samples whose categories are unknown. Generalization is a key concept here. In real life, we rarely see the same object from the same perspective, or encounter the same scene over and over again, but if we can generalize our previous experience to a new perspective or scene, we can deal with more real-world problems.

Using perceptrons to distinguish gender

Give an example of using perceptrons to solve real-world problems. Think about how to distinguish the faces of men and women if you remove hair, jewelry and secondary sexual features, such as Adam's apple, which is more protruding in men than women.

Is this face male or female? People train perceptrons to distinguish the faces of men and women. The pixels from the facial image (upper image) are multiplied by the corresponding weight (lower image), and the sum of the product is compared with the threshold. The size of each weight is depicted as the area of pixels of different colors. The weight of positive value (white) is male, while the weight of negative value (black) tends to be female. The width of the nose, the size of the area between the nose and mouth, and the image intensity around the eye area are important for distinguishing men, while the image intensity around the mouth and cheekbones is more important for distinguishing women.

The interesting thing about the job of distinguishing male and female faces is that while we are good at making this distinction, we cannot accurately describe the differences between male and female faces. Since no single feature is decisive, this pattern recognition problem depends on combining a large number of evidence of low-level features. The advantage of perceptrons is that weight provides clues to the faces that are most helpful for gender differentiation. Surprisingly, the most prominent feature is the part between the nose and lips, and the area is larger in most men. The area around the eyes (larger for men) and the upper cheek (larger for women) also have high information value for gender classification. The perceptron weighs the evidence from all these locations to make a decision, and that's how we make the decision, although we may not be able to describe how we did it.

Extension of perceptrons

The perceptron stimulates a wonderful mathematical analysis of pattern separation in high-dimensional space. When those points exist in a space with thousands of dimensions, we cannot rely on intuition about the distance between points in the three-dimensional space in which we live. Russian mathematician Vladimir Wapnik (Vladimir Vapnik) introduced a classifier called support vector machine (Support Vector Machine) on the basis of this analysis.

It generalizes perceptrons and is widely used in machine learning. He found a way to find the plane automatically, which can separate the two categories of points as much as possible (linear). This makes generalization more tolerant to the measurement error of data points in space. Combined with the "kernel technique" (kernel trick) as a nonlinear extension, support vector machine algorithm has become an important pillar of machine learning.

This is the end of the content of "how perceptrons are produced in machine learning". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.