Shulou(Shulou.com)06/02 Report--

2020-01-18 11:52 introduction: the first Chinese President of AAAI ~

According to Zhou Zhihua Weibo of Nanjing University, Professor Yang Qiang, chief artificial intelligence officer of WeBank, is the president of AAAI 2021, the international artificial intelligence conference. Professor Yang Qiang's appointment is also the second chairman of the AAAI conference and the first time for a Chinese.

Professor Zhou Zhihua mentioned on Weibo that before 2020, there will be only the Chairman of the procedure Committee (Program Committe e chair) and no President of the General Assembly (General chair) in AAAI. Due to the recent sharp increase in the number of contributions, in order to enable the Chairman of the procedural Committee to focus on controlling the academic quality of contributions, the Chairman of the General Assembly will not be set up until 2020.

Professor Yang Qiang is an international expert in artificial intelligence and has made many contributions in academia and industry, especially in recent years, he has made important contributions to the development of artificial intelligence and data mining in China. Professor Yang Qiang is not only the initiator and leader in the field of international artificial intelligence transfer learning (transfer learning), but also one of the sponsors and leaders of international federal learning (Federated Learning).

He was elected as AAAI Fellow in July 2013, which made him the first Chinese to receive this honor, and then he was elected as a member of the Executive Committee of AAAI in May 2016. he was the first Chinese Executive Committee of AAAI, and he was elected President of the Council of the International Federation of artificial Intelligence (IJCAI) in August 2017. he is the first Chinese scientist to serve as Chairman of the IJCAI Council.

Thus it can be seen that it is only natural to be the chairman of AAAI 2021.

Coincidentally, not long ago, the AI Science and Technology Review of Lei Feng (official account: Lei Feng) happened to listen to a report made by Professor Yang Qiang. On January 11, at the "Annual meeting and Cognitive Intelligence Summit Forum of Tsinghua-Chinese Academy of Engineering knowledge Intelligence Joint Research Center", Professor Yang Qiang made a report on "several Frontier issues of Machine Learning", aiming at the limitations of artificial intelligence algorithms. Several cutting-edge issues at the machine learning level are summarized. In the report, Professor Yang pointed out that most links in the machine learning process can be designed automatically; in the face of the dilemma of small data sets, transfer learning is a good solution; today, with more and more attention to privacy protection, federal learning can achieve the use of multi-party data for training, but also can well protect each party's data privacy.

AI Science and Technology Review takes this opportunity to share Professor Yang Qiang's views with you as follows-Lei Feng's AI Technology Review has been edited and revised, which has not been confirmed by Professor Yang Qiang himself.

1. How to scale machine learning?

Although the development of artificial intelligence is very hot now, artificial intelligence is facing great challenges, first of all, the challenge of talents, training a talent of artificial intelligence, including training in school, training in practice, it will take nearly a decade before and after. So can artificial intelligence technology itself be used to design artificial intelligence? In the specific application of artificial intelligence, can some links be carried out by artificial intelligence? In other words, can AI's algorithm be designed automatically? To solve this problem, we need to comprehensively consider the AI algorithm, especially every link of the machine learning algorithm, which link is suitable for automation and which link can not be automated.

The whole process of machine learning includes problem definition, data collection, feature engineering, model training and testing, application, and then feedback the application results to the first step. So the process of this cycle is very complicated, and there are many links that require artificial intelligence experts, which is why an artificial intelligence landing project is very expensive. So which step can be solved by automation? For example, starting from the definition problem, the definition problem is not only the problem of learning from the past knowledge, but also can consciously put forward new ideas, and even some ideas have no past experience to refer to.

Therefore, we conclude that it is difficult to introduce automation in the process of defining problems, but the later links, such as data collection, data aggregation, and then the formation of feature engineering, including model training, can actually be automated.

In the concept of mathematical model, the purpose of machine learning is to make the difference between training data and model getting smaller and smaller. The whole process is not only an optimization process, but also a probability process. When we are looking for the model, we are actually looking in the configured parameter space. But the number of parameters, especially in deep learning, is very large, and the dimensions can reach hundreds of millions. These parameters are generally adjusted by machine learning experts, so if you use a machine, is it better?

The second link is the performance evaluation, that is, the difference between the evaluation model and the training data, which can also be partially solved by the machine, although the definition of the difference itself is still solved by the human. That is, it is up to mathematicians to define the difference between a model and training data. Therefore, some recent analysis, whether the basis of artificial intelligence should be mathematics, is reflected in: mathematicians' definition of distance, that is, a variety of definitions of distance between transformation spaces in different spaces. So how to solve it efficiently and find the best configuration in this space is an optimization problem, so it can be summed up as: data preprocessing, feature processing and model training. These aspects can form some search spaces, and optimization functions can be formed in such spaces. For example, if there are three spaces on the left, then these three spaces are included in the performance space on the right.

Automated feature engineering already has a very good platform, for example, the fourth paradigm company launched the AutoCross platform, which will automatically combine and screen the features of different dimensions, and finally come up with the optimal combination. At the same time, it is difficult to find an optimal network structure in automated machine learning, which is also the search problem of topological space.

The upper-right corner of the image above shows a deep learning topology, that is, the connection from one neuron to another, which is ever-changing and influential. So how to find the best topology? This is a difficult problem, and it is now trying to solve it in an automated way, specifically referring to the concept of reinforcement learning, as shown on the left side of the figure above. First of all, in the process of deep learning, the right side shows how to constantly find a better matching value, that is, a matching parameter in the network topology space, and then return to the design to get feedback. This is like AlphaGo playing chess, except that the chessboard is defined as a network connection.

2. What if there is not enough AI data?

All the existing algorithms for deep learning have an important assumption: there is enough data. If the data is not enough, it can be solved by transfer learning.

The specific method is: first look at the red model in the picture above, assuming that the model we want to train is the target model in the picture above, which requires a lot of data. If you assume that the data is limited, you need to look for the blue field on the left side of the image above. The blue area is characterized by a large amount of data, a very reliable model, and a very good model. So transfer learning is: migrate from the blue mature model above to the red domain. This is similar to human analogical learning.

The result is: if the original data is very large, migrate it to a small data, the effect will be very good. And when the data quantity and data quality of the original data continue to improve, the effect of transfer learning is also improving. Then it can be landed on the ground to a deep learning transfer learning. In the image above, red can be migrated to blue, and red already has a flow from left to right, with input on the left and output on the right, which can be well classified.

But the part that is particularly relevant to the domain, especially the specific part, exists at the upper end of the depth model, that is, the part of the output on the right, which tries to keep it out of the migration. This involves a migration strategy, similar to the annealing model. Over time, we gradually push the focus of this migration to the bottom, so that the migration effect of the following blue model becomes better and better.

This kind of migration is now widely implemented in the industry, for example, in the risk control of auto finance, large car loans are often very few, so we need to transfer the learning solution, specifically through small loans, a large amount of data, through the migration between the two data to achieve good results.

For example, urban computing, assuming that a good traffic travel prediction model has been obtained in a city, then it can be migrated to a new city, then good results can be obtained without collecting a lot of data in this new city. The automated transfer learning mentioned above is the use of machine learning to learn transfer learning strategies. The main points of the strategy are: the choice of the original field, the choice of transfer learning algorithm. Then find the best optimization solution in the optimization space on the right, turn this problem into a mathematical problem, and then you can solve it.

3. How to protect users' privacy?

More and more voices tell us that we should also do moral artificial intelligence when doing artificial intelligence. First of all, we must protect people's interests, the greatest embodiment of people's interests is privacy. Big data can improve efficiency on the one hand and involve the privacy of many people on the other. Now a variety of laws and regulations also appear frequently, such as the European GDPR protection law, China also has a corresponding very strict personal privacy protection law, and spread in games, finance, the Internet in all aspects. The situation in the past is: different institutions aggregate data into a big data company, while giving full play to the power of big data company, so that there are enough samples and dimensions; the disadvantage is that privacy will be exposed.

Is there any other way to build the model with high quality? Federal learning (Federated learning) is a new approach. Suppose there are two data owners, Party An and Party B, side An is the upper matrix, and Party B is the matrix below, and there may be data overlap between them. The aim now is to make Party A can't see Party B and Party B can't see Party A. at the same time, it is necessary to establish a common model, which uses the data of both parties. For example: suppose a farmer is raising a sheep and he needs to collect grass from all over the country to the farm to feed the sheep, similar to aggregating the data to a central server. But assuming that the grass cannot be moved to other places, what we can do now is to lead the sheep to graze everywhere. That is, let the model first come to the A side for training, and then bring the model to the B side for training. after a few times, the model grows, and the data does not have to flow out of the local area. This is the idea of federation learning.

Federal learning requires a lot of cross-domain knowledge in the computer field, such as multi-party computing, privacy encryption, encryption technology, mathematics, distributed machine learning, distributed computing. How do you do it exactly? For example, it is necessary to transfer the learning model between two fields, from A to B, so that A does not see B's data while B does not see A's data. This can be done by encrypting the parameters and weights of the data through logical regression, and then transporting the encrypted package to B, then the encrypted package will participate in the model training on the B side, and then encrypt the model to A, so repeatedly, and the model has matured. There are two points in the above process: the first does not disclose any aspect of the data, and the second achieves the same effect as the training with the sum of the data on both sides.

The recommendation system widely used in e-commerce and video will generate a large amount of data, and different referrals have different data. In order to protect privacy, we can't roughly upload all the data generated by clicks on mobile phones, so we should adopt the approach of federal learning. The specific practices are as follows:

First of all, realize that the data has two parts, one is to describe the user, which is the U1~UN on the right side of the figure above. The other part is the description of the product, which is described by the matrix in linear algebra. After decomposing the matrix, then realize that although there are different users, the product matrix itself is common, and this matrix can be learned by federation learning. The specific learning process is as follows:

In the process of iteration, each part is contributing the eigenvalues of the part they have learned, and then sending it to the server, in which we should pay special attention to the red encryption steps on the right side of the image above: encrypting with modules when uploading and downloading, while packaging the parameter packages of the product matrix, so that each side can not see each other's data, and the model is growing.

At the same time, we can transfer and learn this process, that is, when the dimensions of the users and parameters of the two data sides overlap little, we can take a step back and map it to a subspace for learning, so we can get a very robust learning effect. The result of the practice in the specific movie recommendation data set is shown in the above figure, the right side represents the error rate, and the error rate decreases sharply with the number of training.

At the same time, in terms of news recommendation, the recommendation engine of federal learning and transfer learning is used in Caixin news recommendation. The technology is now open source and on Linux Foundation, and there has been a breakthrough recently.

4. How does AI achieve anti-fraud?

The landing application of AI in financial institutions is particularly concerned about how to achieve anti-fraud. The development of artificial intelligence has a technology: Deepfake, can simulate a completely virtual person, between the virtual and the real not only people can not see, now many artificial intelligence algorithms can not be distinguished.

At present, AI fraud mainly focuses on the following points: the first is the fraud of the data; the second is the fraud of the model; the third is the fraud of the results. For the above three aspects, researchers have carried out targeted research, for example, for the training process, on the basis of considering the original training objectives, while considering adding antagonistic samples to enhance the robustness of the model. Not only one model should be considered, but also several models should be considered to classify the samples from different angles. Suppose the bad guys use two models, we will use four models, and suppose the bad guys also learn to use four models, we will use eight models.

5. Summary

Finally, to sum up, the success of artificial intelligence lies in: first, it can automate a link, such as the automated machine learning we just talked about. The second distributed big data, that is, how to make different data owners cooperate on the premise of protecting privacy. The third is the improvement of high-performance computing power, which I do not have much research, Tsinghua University has done a lot of research in this area, I will not repeat it. Thank you!

Https://www.leiphone.com/news/202001/f4XPv3dB7HnpZe4y.html

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.