Shulou(Shulou.com)05/31 Report--

This article mainly explains "what is the error representation method in PCA". Interested friends may wish to have a look at it. The method introduced in this paper is simple, fast and practical. Next, let the editor take you to learn "what is the error representation in PCA?"

Given n m-dimensional samples X ­(1), X (2), … X (n), suppose our goal is to reduce the n samples from m-dimension to k-dimension, and to ensure that this dimensionality reduction operation will not incur a great cost (loss of important information) as far as possible. In other words, we want to project n sample points from m-dimensional space to k-dimensional space. For each sample point, we can use the following expression to represent the projection process:

Z=ATX (1)

Where X is the m-dimensional sample point, Z is the k-dimensional sample point obtained after projection, and An is an m * k matrix.

In retrospect, if principal component analysis (PCA) is used to reduce dimensionality, we first find the mean of the sample:

Then get the scatter matrix (scatter matrix):

(2)

Example: in order to more intuitively understand the meaning of equation (1) geometrically, we take a set of two-dimensional data as an example, in which we use PCA's method to reduce the set of two-dimensional arrays to one dimension. The eigenvectors stored in matrix An are actually reduced to a new coordinate axis, and in this case, we get a new one-dimensional coordinate axis. As shown in figure 1, the red cross point in the figure represents the point on which the 2D sample point is projected vertically onto this new axis. For each sample point X in two-dimensional space, as long as we substitute it (1), we can calculate its reduced-dimensional expression (in this case, a 1-dimensional vector, that is, a value):

Fig. 1 expression of 10 sample points in two-dimensional space

The value calculated by equation (3) is actually the distance of these projection points from the origin. Therefore, we can draw a number axis to represent the new axis, and then mark their positions on the number axis according to the values calculated by equation (3), as shown in figure 2.

(4)

To understand equation (4), we first need to understand AATX (I). Looking back at what I just said, the result of calculating ATX (I) is actually the representation of the sample points in low-dimensional space (see figure 2). Relatively speaking, X (I) is the expression of sample points in high-dimensional space. However, we know that points in different latitudes cannot be compared. for example, a point in two-dimensional space (x1, x2) cannot be compared with a point in one-dimensional space (y1). Because they have different latitudes (they exist in a different world, they are not in the same world).

In order to compare the sample points of two different latitudes, we need to put them in the same latitude space. A reasonable approach is to project points in low-dimensional space to high-dimensional space and assume that the value of high latitude is 0. The work done by AATX (I) is to project the reduced sample points onto the high-dimensional space. In the example just given, ATX (I) is the cross point on figure 2, while AATX (I) is actually the cross point on the straight line (new axis) in figure 1.

It is worth noting that these forks in figures 2 and 1 correspond one to one, and their distance from the origin is constant in both high-dimensional and low-dimensional space (carefully observe the distance between the fork in figures 1 and 2 from the origin). We can still prove this theoretically around this example. First, suppose that the reduced-dimensional expression of one of the sample points X is Z = [s1x1+s2x2], then the reverse projection of it from low-dimensional to high-dimensional (in this case, from 1-dimensional to 2-dimensional) is:

Since s has been unitized, that is, | s | | = s12+s22=1, so | | Xapproax | | = (s1x1+s2x2) 2 = | Z | |, the certificate is completed.

Prove ②:

First of all, to get the general expression of the hyperplane, and to get the general expression of the hyperplane, it is necessary to calculate the normal vector n corresponding to the hyperplane. In this case, the normal vector satisfies nTs=0, where sT= [S1, S2]. We can get n = [- s2/s1, 1], then the general expression of the hyperplane is (- s2/s1) x ­1+x2=0. Substituting XapproxT= [S1 (s1x1+s2x2), S2 (s1x1+s2x2)] into (- s2/s1) x ­1+x2, we get (- s2/s1) * S1 (s1x1+s2x2) + S2 (s1x1+s2x2) =-S2 (s1x1+s2x2) + S2 (s1x1+s2x2) = 0. It is said that for any Xapprox, it is on the hyperplane.

In the return formula (4), L calculates the sum of the distances of each sample point projected from the high-dimensional space to the low-dimensional space.

At this point, I believe you have a deeper understanding of "what is the error representation method in PCA". You might as well do it in practice. Here is the website, more related content can enter the relevant channels to inquire, follow us, continue to learn!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.