Shulou(Shulou.com)06/01 Report--

Today, I will talk to you about how to analyze the AutoML algorithm, many people may not know much about it. In order to make you understand better, the editor has summarized the following content for you. I hope you can get something according to this article.

What is AutoML?

As the name implies, Auto:Automated is automatic; ML:Machine Learning machine learning. So AutoML is automatic machine learning.

For algorithm engineers of machine learning, it is a very complex task to design machine learning models suitable for specific problems. It is necessary to choose the corresponding neural network architecture, training process, regularization method, super-parameter and so on, which have a great impact on the final performance and need to be tried constantly. Therefore, the deep learning algorithm engineer is also called the tuning (refining) parameter (Dan) engineer.

The goal of AutoML is to use an automated, data-driven approach to make these decisions. As long as the user provides data, through sufficient computing power, the system automatically determines the best scheme. Experts in various fields no longer need to worry about learning various machine learning algorithms.

In the field of AutoML, what scholars pay most attention to is NAS (Neural Architecture Search, Network structure search), and there are many corresponding algorithms. More than 200 related papers have been listed on automl.org:

Https://www.automl.org/automl/literature-on-neural-architecture-search/

The corresponding algorithms can be roughly divided into three categories:

Discrete search algorithm based on RL (Reinforcement Learning): NASNet,ENAS

Discrete search algorithm based on evolutionary algorithm (such as genetic algorithm, ant colony algorithm, simulated annealing, etc.): CARS,EENA

Continuous differentiable search algorithm based on gradient descent: DARTS,FBNet

This paper mainly introduces the first kind of discrete search algorithm based on RL.

NAS algorithm based on RL

The best of these algorithms is NASNet proposed by Google Brain in Learning Transferable Architectures for Scalable Image Recognition in 2018, and the process is shown in the following figure. The whole structure consists of controller and validator. The iterative steps of the algorithm are as follows:

Controller is responsible for sampling child networks.

The sampling results are given to validator for evaluation accuracy.

Evaluate accuracy as reward to train controller

Return to the first step to cycle until the end condition is reached (the number of samples is reached or accuracy is reached)

Finally spent 2000 GPU hours search to get a series of structures, excellent performance, in the same number of parameters / computation, accuracy can reach the best level at that time, surpassing a series of excellent networks such as Inception,ResNet,MobileNet,SENet.

The final searched network structure:

Where normal cell and reductioncell are:

After reading the above, do you have any further understanding of how to analyze the AutoML algorithm? If you want to know more knowledge or related content, please follow the industry information channel, thank you for your support.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.