Shulou(Shulou.com)06/03 Report--

How to solve the problem of finding the extreme value of multivariate function, I believe that many inexperienced people are at a loss about it. Therefore, this paper summarizes the causes and solutions of the problem. Through this article, I hope you can solve this problem.

Today, we will discuss the problem of finding the extreme value of multivariate function, which will be used to calculate the parameters by Newton iterative method in Logistic regression, so it is necessary to study it clearly.

Think about it, how do we do the problem of finding the extreme value of a unary function? For example, for a concave function, first find the first derivative, and get

Because the derivative at the extreme value must be zero, but the point where the derivative equals zero does not necessarily have an extreme value, for example. So further judgment is needed, yes.

The function continues to take the second derivative, because the second derivative holds at the stationary point, so

The minimum value is obtained at the point, and the meaning of the second derivative here is to judge the local concavity and convexity of the function.

The method of finding the extreme value in a multivariate function is similar, except that a matrix, called Hessian matrix, is introduced to judge the concavity and convexity.

If the real-valued multivariate function is second-order continuously derivable in the definition domain, then we find its extreme value and first find the partial derivative for all, that is,

An equation is obtained as follows

The stationary point can be solved by this equation, and the stationary point is an one-dimensional vector of length. But we only get this stationary point, which is actually here.

There are three cases of stationary points, namely: local maximum, local minimum and non-extreme value.

So the next thing to do is to determine which of the three residences belong to. So the Hessian matrix is introduced, that is to say, it is used to

Judge the concavity and convexity of multivariate functions.

Hessian matrix is a square matrix composed of second-order partial derivatives of a multivariate function, which describes the local curvature of the function and is often used in Newton iterative method to solve optimization problems.

For example, for the above multivariate function, if its second partial derivative exists, then the Hessian matrix is as follows

If the function is continuously derivable in the definition domain, then the Hessian matrix is a symmetric matrix in the definition domain, because if the function is connected

Then there is no difference in the order of derivation of the second partial derivative, that is,

With the Hessian matrix, we can judge the three extremes mentioned above. The conclusions are as follows.

(1) if it is a positive definite matrix, then there is a local minimum at the critical point.

(2) if it is a negative definite matrix, then there is a local maximum at the critical point.

(3) if it is an indefinite matrix, then there is no extreme value at the critical point.

Let's go on to learn how to judge whether a matrix is positive definite, negative definite, or indefinite.

One of the most commonly used methods is the sequential principal subtype. A real symmetric matrix is a positive definite matrix if and only if all the sequential principal subforms are greater than zero.

Because this method involves the calculation of determinant, it is more troublesome! There is another method for real quadratic matrices, which is described as follows

The real quadratic form matrix is positive definite quadratic form if and only if the eigenvalues of the matrix are all greater than zero. A sufficient and essential piece of negative definite quadratic form

The eigenvalues of the matrix are all less than zero, otherwise they are indefinite.

After reading the above, have you mastered how to solve the problem of finding the extreme value of multivariate functions? If you want to learn more skills or want to know more about it, you are welcome to follow the industry information channel, thank you for reading!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.