Shulou(Shulou.com)06/03 Report--

Machine learning is in a dominant position. Especially in pattern recognition, machine learning is the preferred method. Tangible examples of its applications include fraud detection, image recognition, predictive maintenance and train delay prediction systems. We usually encounter these three main problems (but not the only ones) in the process of day-to-day machine learning (ML) and seeking to deploy the knowledge gained.

Data quality-data from multiple sources over multiple time ranges may be difficult to organize into clean and coherent data sets that will benefit most from machine learning. Typical problems include data loss, data value inconsistency, autocorrelation and so on.

Business relevance-while many of the technologies that support the machine learning revolution are moving faster than ever before, many applications today do not take business value into account.

Operational model-once the model has gone through the build and adjustment cycle, it is critical to deploy the results of the machine learning process to the wider business. This is a difficult bridge to cross, because predictive modelers are not usually IT solution experts, and vice versa.

There is also a whole set of algorithm toolkits behind machine learning, and each algorithm can be adjusted using so-called hyperparameters to achieve higher accuracy. For example, for the popular k-nearest neighbor algorithm, k refers to the number of neighbors we want to consider. In a neural network, this will cover the entire architecture of the network.

One of the key tasks that data scientists do today is to find the right algorithm for a given problem and "set" it correctly. But in fact, the scope of the mission is much larger. Data scientists must understand the business perspective of the problem, solve the data situation, prepare the data appropriately, and obtain models that can help with the assessment. This is usually a circular process that follows the Cross-Industry Standard data Mining process (CRISP-DM) [1].

Accordingly, projects in the field of machine learning are complex and require time for multiple people to qualify in a range of areas (business, IT, data science). In addition, it is often not clear what the outcome will be: therefore, in this sense, such a project is risky.

Correlation of AutoML (http://www.o9qh.com)

To this day, data science projects cannot be automated. However, in some cases, some steps of the project can be automated: this is the reason behind the concept of automatic machine learning (AutoML). For example, AutoML can help you choose an algorithm. Data scientists usually compare the results of several algorithms to the problem and choose an algorithm considering a number of factors (such as quality, complexity / duration, robustness). Another aspect that can be automated in some cases is the setting of hyperparameters: many algorithms can be adjusted by parameters and their quality optimized relative to a particular problem.

AutoML is a resource that accelerates data science projects that automate those components or individual steps, thereby increasing productivity. For example, AutoML is very useful in algorithm evaluation. As a result, many libraries and tools use AutoML as a supplementary feature. Noteworthy examples include auto-sklearn (in the Python community) or DataRobot, which specializes in AutoML. The following example, taken from RapidMiner, shows how to use an assistant to compare different algorithms and quickly find the best algorithm for a particular problem [2]:

Nevertheless, AutoML should not be understood as an one-size-fits-all solution that fully automates data science projects and does not require data scientists. In this sense, unfortunately, it is not the Holy Grail.

Like other professional areas, automation is first of all a tedious technical task, in which highly skilled professionals would otherwise spend most of their time systematically trying certain parameter sets and then comparing the results-a job best left to the machine.

What remains is a large number of challenges that mankind still needs to solve. This starts with an understanding of the real problem itself and covers a wide variety of time-consuming tasks from data engineering to deployment. AutoML is a useful tool, but it is not the Holy Grail yet.

More: (http://www.o9qh.com)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.