Shulou(Shulou.com)11/24 Report--

This article will introduce physical information neural network from the point of view of physics, from physics to artificial intelligence.

Let's take the first step from here:

We already know how the world works in physics. By using scientific methods, we can put forward a hypothesis, we explain how a specific phenomenon occurs, and then design a controllable experiment, through the experimental data to confirm or falsify this hypothesis.

It can be said that physics and the process of natural evolution are closely related. I remember that when I was in college, a professor began his report with the following sentence:

Heraclitus said: "everything moves." I believe it. But how do you exercise?

And this is the question that physics is trying to answer. How to "move" all things?

People use a specific equation, the well-known differential equation, to describe the way everything moves. Let's first try to understand what a differential equation is.

1. Physics and differential equations "differential" describe a class of things related to subtraction.

1.1 derivative the derivative of a function has a special meaning in physics.

For example, velocity represents the derivative of space to time. Let's consider the following experiment: an object moves along an one-dimensional line.

The source author means that the blue ball is moving along the x-axis. Usually, we set the initial point to 0. When the ball moves, its position changes over time, assuming that the image of the position changing over time is as follows:

As a result, the movement can be described more clearly:

a. Between time 0 and time 5, the position changes from 0 to 9: the ball moves forward.

b. From 5 to 15, the position has not changed: the ball stands still.

c. Time 15 to 17, position 9 to 3: the ball moves back.

d. Position 17 to 47 from 3 to 6: the ball is moving forward again.

When and how the ball changed its position was clearly quantified? Even the amount of position change is clear. In fact, this is the information of speed, which is described by the following expression:

Equation (1)

It seems a little complicated. But in fact, the velocity is only the position change corresponding to two very similar moments (the limit of h tends to zero in the expression) divided by the difference between the two times (h), so we use differential equations to describe it. In other words, velocity is the instantaneous change of position normalized by time interval.

If there is a sharp increase in position at a particular time, it means that the derivative is a very large positive number; when the position does not change, the derivative is zero; when the position decreases, the derivative is negative.

In the above example, the change of the position of each segment is linear, that is, the speed between 0-5, 5-15, 15-17, 17-47 remains the same.

Take tweak 0 to taper 5 as an example, for each time t, the instantaneous change of position is 9 stroke 5, and the above function can be understood with the same meaning in other periods.

1.2 numerical solution the above example is very simple, let's consider the following example:

The trajectory of the source author is difficult to model, and there is no analytical way to calculate the derivative: you can only calculate it numerically-the derivative is given by formula (1) at all points in time. By the way, I think you are completely ready to face the harsh reality:

All the differential equations in the real world are solved by numerical software.

However, the numerical solution may require thousands of iterations. Not only that, advanced methods are needed to solve differential equations, such as the famous Runge-Kutta method, which are often integrated into very complex software (such as POGO for finite element methods):

High computational cost

High economic cost

Professional knowledge is required

The time cost is high (take the finite element method as an example, it often takes several minutes to several hours)

1.3 ill-posed problems, and the above problems are not even the worst. The sad thing is that there will be ill-posed. Let me give you an example.

For example, problems that require the solution of x and y:

Isn't it easy?

X+y=4

So x+2y=x+y+y=8

4+y=8

Get yellow4.

Xerox 0

The solution of the system of equations is (x _ ()) y) = (0) ~ (4), and the problem is well-posed. Because the solution that satisfies this condition is unique, that is, the solution obtained above.

But the following questions are different:

The two equations are actually the same! Although it can be used as a solution, it can also be used as a solution. In fact, this system of equations has an infinite number of solutions. This problem is an ill-posed problem, for such a definite problem, there is more than one solution (in fact, there are infinite solutions).

If you are solving the following problem:

Equation (2)

This kind of so-called inversion problem can be used to derive the velocity diagram from the displacement map, and the above inversion problem can be used to characterize an erosion defect in the material in ultrasonic detection. Even in the most perfect experimental setting (unlimited number of sensors), this problem is ill-posed. In other words, the information cannot derive a unique and reliable velocity distribution ().

two。 Artificial intelligence and neural networks now talk about artificial intelligence (AI).

This sentence can briefly introduce artificial intelligence:

Artificial intelligence algorithms do not need specific programs to perform specific tasks.

A self-driving car can also brake when anyone walks in front of the car without math and clear training, because it has passed the "training" of millions of people.

To be exact, all artificial intelligence algorithms rely on loss function (Loss Function).

This particular function is used to represent the difference between the target value (target, the value you want to output) and the algorithm output value, which needs to be optimized to achieve the minimum value.

If you give a house with characteristics, you want to predict its cost (a well-known real estate data set problem). This is a regression problem from input space to continuous space.

If the predicted price given by the source author is 130k and the actual value is 160k, then the average absolute error MAE is defined as:

30k in this problem.

This formula indicates that our model is a neural network named to process input data and output a predictive value. In the above example, and.

If many houses have this series of projected costs, the global loss function would look something like this:

The loss function depends on a series of parameters W. The lower the loss function, the better the model.

As a result, the loss function will be constantly optimized, in a sense, as low as possible. The parameters are iterated over and over again to minimize the loss function (local minimum).

3.AI + Physics = physical Information Neural Network if you see here:

You deserve a round of applause.

You may wonder: what exactly is the relationship between artificial intelligence (neural networks) and differential equations (physics)?

Before we answer this question, we need to understand another concept-regularization (regularization).

3.1 regularization in the previous part, we learned that all machine learning algorithms are ultimately an optimization problem. It means to find the best parameter group to minimize the loss function.

However, there is a problem that this solution may be the best in the training set, but it is not suitable for the test set (over-fitting). In this case, the optimal value is only locally optimal.

Explain a little more in depth:

Assuming that the initial model is generated by two parameters, you look for the solution in the following space:

The loss value L obtained by the parameters obtained by the source author and the loss function applied in the training set is close to 0. 5%.

If this parameter combination is applied to the test set (a series of new data), the loss function becomes very large. It shows that the definition of loss function is not accurate. In fact, in this problem, the real optimal solution is as follows:

How can the author of the image source get the best point (green mark) by avoiding the error point of the red mark?

It would be nice for the source author to "find" the algorithm only in the green circle, so that he would not fall into the wrong red trap (local optimization).

This is regularization-modifying the loss function so that its spatial solution is limited, so that it is more likely to solve the global optimization rather than the local optimization.

3.2 physical information neural network = regularization! Do you remember the inversion problem represented by equation (2) above? Some scholars are trying to solve the problem.

Initially, the displacement (u) is known at some specific locations, and they want to interpolate u in the unknown area of the algorithm to get v. That is to say, given the new t, y and x, the corresponding new displacement can be obtained, thus the new velocity distribution can be obtained.

Now there are many disputes about the solution of this equation, because this problem is not well-posed, that is to say, even if people find a solution, they are not sure whether it is unique or not. Not only that, there are many physical limitations that cannot be solved.

They hope to generate displacement by limiting the displacement to satisfy the wave function equation (2), which is combined with the loss function:

Satisfy:

:

It represents the square deviation between the predicted displacement and the target displacement, and the value should be as close to 0 as possible.

What does that mean? In short, it represents a regularization process:

They are helping the algorithm find the optimal solution by limiting the range of loss functions (excluding solutions that do not conform to differential equations).

How do neural networks obtain "physical information"? It's just regularization through differential equations.

4. At the end of the article, I would like to say:

The physical information neural network is just a neural network that regularizes the loss function through differential equations.

After a few minutes of reading, this article can be summarized into several aspects:

What is the loss function (part 2)

What is regularization (part 3.1)

What is the differential equation (part 1)

What is the physical information (part 3.2)

I hope you have some understanding of physical information neural network.

This article comes from the official account of Wechat: Institute of Physics, Chinese Academy of Sciences (ID:cas-iop), author: Piero Paialunga translator: zhenni revision: Crunc

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.