Shulou(Shulou.com)06/03 Report--

Unsupervised image classification is a challenging research topic in the field of image classification. this paper introduces the development of unsupervised image classification algorithms for your reference.

Author | Guo Bingyang

Editor | there are three words.

1 introduction

In recent years, deep learning has made unprecedented progress in the field of image recognition, which can be attributed to the expansion of data set capacity and the improvement of computing resources.

At this stage, the task of image classification is largely achieved by supervised learning, that is, each sample has its corresponding label, through the depth neural network to constantly learn the corresponding features of each label, and finally achieve classification. In this case, the capacity of the data set and the quality of the label often play a decisive role in the performance of the model.

If you think of the neural network as an F1 car running on the track, the data set is the energy that constantly provides it with power. If there is no high-quality data set as the basis, it is impossible to drive the neural network for training.

High-quality data sets will naturally bring difficulties in tagging. According to statistics, it takes about 2 to 3 seconds to mark a single object category in a single image, but in practical applications, data sets often contain tens of thousands of images. The whole labeling process becomes extremely long. Especially when it comes to fine-grained classification and multi-label classification, the labeling cost will increase exponentially with the number of targets and the difficulty of identification.

Under this background, the research on unsupervised image classification is becoming more and more hot, which can be divided into two directions: data set transformation and clustering analysis. This paper will introduce the research status of unsupervised image classification in two directions. For your reference.

2 algorithm based on dataset transformation

Dataset transformation is to construct a new representation of the existing dataset so that the features it contains are easier to understand, that is, the so-called dimensionality reduction and noise reduction. The main methods are PCA and T-SNE.

1 、 PCA

PCA algorithm, namely principal component analysis algorithm, is a typical method of rotating data sets in the field of machine learning, and the rotated features are not statistically related. Through the rotation of the data set, a subset can be constructed according to the importance of the new features to the interpretation data, thus constructing a new representation of the data set.

2 、 T-SNE

As a widely used data analysis algorithm in recent years, the main idea of T-SNE is to find the two-dimensional representation of the data and keep the distance between the data points as much as possible, and then try to make the closer points in the original feature space closer, and the distant points in the original feature space farther away. It focuses on the points that are close to them, rather than those that are far away.

In principle, the complexity of the above two methods of data set transformation is high, and the goal of the algorithm is too clear, so that there is no secondary information in the abstract low-dimensional data, and these secondary information may be the main factor to distinguish the data from a higher level. Therefore, these two algorithms are mostly used in the data preprocessing stage before network training to provide corresponding prior knowledge for subsequent operations.

3 algorithm based on cluster analysis

Clustering analysis is one of the most commonly used methods in unsupervised learning, and it has been widely used in the field of machine learning. many researchers have tried to combine mature clustering algorithms with deep learning to achieve more efficient learning strategies.

The current research results can be divided into two types: unsupervised classification based on K-means algorithm and unsupervised classification based on information invariance.

3.1.Unsupervised classification based on K-means algorithm

DCN network [1] proposes a self-encoder (auto-encoder) method to learn and cluster the network at the same time. The encoder learns the potential features of the input data, maps the high-dimensional features to the low-dimensional subspace and inputs them to the K-means clustering model for clustering, while the decoder restores the features to reconstitute the original data, which is conducive to network learning of more important features and ignore some unimportant features.

Deep Cluster [2] also combines the two branch tasks of clustering and classification based on the inspiration of DCN network, and realizes the clustering of network features through K-means algorithm. Its innovation is to take the clustering results as pseudo-tags, update the parameters of the network, and further let the network predict these pseudo-tags. These two processes are carried out in turn, and quite good prediction results are obtained.

On the other hand, Associative Deep Clustering puts forward a scheme of clustering directly using the deep neural network, instead of sending the feature graph extracted from the network into the clustering model for additional training. The main inspiration is to let the network learn the structure type of the data in the deeper layer, so as to refine the centroid of a certain kind of data to complete clustering.

The main idea of the unsupervised classification algorithm based on K-means is to embed the clustering module into the feature layer extracted by the neural network or directly let the network learn the structural features of the data. Although this method has achieved some results, it lacks the corresponding semantic filtering process, which can not guarantee that the features used are meaningful, which will affect the final classification results to a certain extent.

3.2. Unsupervised classification based on information invariance

As mentioned in the previous section, the combination of traditional clustering algorithms and deep learning is not a natural match, and it is often necessary to rely on the dataset transformation operation described in the previous section to provide more prior knowledge.

Therefore, the relevant researchers focus their attention on the neural network itself and expect to improve the network structure in order to make better use of image feature information to achieve clustering. Information invariant network (Invarient Information Clustering CNN [3]) is a typical example.

IIC-CNN builds two input branches by making minor changes to CNN. In order to be unsupervised, the model converts each picture x once (translation, rotation, or crop) to get another image x'. Input the branches of the converted image as the auxiliary layer for clustering training directly. At the same time, in order to make the model better identify the same category of objects, IIC-CNN uses the objective function of mutual information maximization to make the clustering effect of the network better as far as possible.

Compared with the traditional clustering algorithm, deep learning and clustering fusion algorithm, this method has made great progress, and has made remarkable achievements in the comparative training of many data sets, which is also a very meaningful research content.

4 Summary

At the present stage, the research of unsupervised image classification based on deep learning is still in the stage of development, coupled with the difficulty of the problem, the research results are less than other directions, and experiments are only carried out on some simple data sets. it is not really applied to the actual scene on a large scale.

Therefore, in order to better make unsupervised image classification widely used, we must explore the advantages of traditional algorithms, closely combined with the characteristics of neural network, put forward more and more creative ideas, in order to achieve greater breakthroughs.

reference

[1] Yang B, Fu X, Sidiropoulos N D, et al. Towards k-means-friendly spaces: Simultaneous deep learning and clustering[C] / / Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. Org, 2017: 3861-3870.

[2] Caron M, Bojanowski P, Joulin A, et al. Deep clustering for unsupervised learning of visual [C] / / Proceedings of the European Conference on Computer Vision (ECCV). 2018: 132,149.

[3] Ji X, Henriques J F, Vedaldi A. Invariant information distillation for unsupervised image segmentation and clustering [J]. ArXiv preprint arXiv:1807.06653, 2018.

Https://www.toutiao.com/i6727839857348968972/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.