Shulou(Shulou.com)06/02 Report--

Last week, one of the biggest things in the AI circle was the Turing Award, which was finally awarded to the Big three of deep learning.

The story about Geoffrey Hinton and his two students, Yoshua Bengio and Yann LeCun, was naturally completed within a few days after the news came out.

Even though AI practitioners and enthusiasts are basically familiar with the deeds of these three, their heroic complex of "holding high the flame and passing through the darkest times" can easily move people far away from the AI world.

The experience of the "Big three" and deep learning is in fact a story of starting from the academic edge, proving itself in the industrial carnival, and then going back to conquer the mainstream academia. The ups and downs are more Hollywood than Hollywood blockbusters.

However, perhaps the biggest difference between real life and movies is that movies can come to an abrupt end at the climax, leaving the audience with meaningful splendor. But life has to go on, life has to go on, and AI has to develop. Deep learning will not come to a successful end with the arrival of the Turing Award.

If the Big three do not believe in the stubbornness of deep learning all over the world, and finally get the Turing Award and the praise of "Tai Dou", then what we should pay more attention to today is that at a time when deep learning is at its peak, what are the Big three busy with? Is it true that what they are busy now reflects the future of AI?

We might as well change the angle, not to talk about their acclaimed years, but to talk about the present of the Big three.

It should be noted that it is not intended to sort out every paper written by the trio and the team, but to read out some of the underlying rules of in-depth learning about future trends and congenital deficiencies from their choices at the time of the AI outbreak.

Yann LeCun: production and Learning migrant Birds and Lu Xun in AI Circle

Among the Big three, Yann LeCun is the most frequently active in the public eye.

Why do many people who don't know Hinton also know Yann LeCun? There are many explanations for this unsolved mystery in the AI world.

Unreliable explanations include that the name Yann LeCun is very suitable to be written as Yang Likun, so it is easy to remember. For example, his trademark smile can easily make people think that AI is not so boring.

The reliable explanation is that Yann LeCun is the most deeply involved in the industrial world among the three, and was even regarded as a representative of AI scientists to the business world.

In 2013, Yann LeCun suddenly joined Facebook was a very explosive thing. People don't understand why Facebook wants AI scientists or what Yann LeCun is doing on social media.

After coming to Facebook, Yann LeCun founded FAIR. This is not a law firm, but a Facebook artificial intelligence lab.

It must be admitted that Yann LeCun, as the head of FAIR, has made a lot of achievements in those years. For example, it greatly improves the automation operation ability of Facebook and improves the intelligent degree of Facebook lifeline such as advertising. On the other hand, the idealistic and scholarly Yann LeCun has turned FAIR into one of the most utopian laboratories in Silicon Valley.

Experts at Facebook,FAIR focus on those forward-looking technologies and how AI will benefit all mankind in the future. With the acquiescence and support of Zuckerberg, the prosperity of FAIR was even thought to be the engineer culture of Silicon Valley to turn the gun to the scientist culture.

In addition, Yann LeCun's charisma has become a powerful tool for Facebook to quickly recruit top scientists. In five years, FAIR has expanded to six offices and has nearly 100 researchers. Along with Wu Enda and Li Feifei, Yann LeCun is also known as one of the three AI stars from school to enterprise.

However, with the continuous expansion of the team size, Yann LeCun, as an ideal scientist, has gradually exposed the deficiency of team management ability. Coupled with the ups and downs of Facebook in 2017, after the overall business reform, the line of business began to ask FAIR for more direct technologies that help improve business quality or liquidity-which is not what Yann LeCun wants.

Yann LeCun, who remained a faculty member at NYU, announced in January 2018 that he had resigned as head of FAIR to become chief scientist behind the scenes. And this has also been interpreted as a signal: it seems that, like Wu Enda and Li Feifei, top scientists are really not so easy to maintain long-term stability in the positions of senior executives of big companies.

In addition to the exploration of the industry, another person of Yann LeCun can be called Lu Xun in the AI circle-twitter when the road is uneven, never rest when you should be scolded.

Many people feel that Yann LeCun is a bit too "amazing", but in fact, the bigger background is that there is endless controversy about what deep learning is, how it will develop in the future, and whether AI is useful or not.

And as one of the biggest stars in the AI industry, Yann LeCun seems to have an obligation to get to the bottom of it.

Let's review three things that got Yann LeCun to start the opposing mode:

1. For the famous humanoid robot, Sophia, the world's first "robot citizen". Yann LeCun angrily denounced it as a "complete scam" and "complete nonsense". In Yann LeCun's view, the so-called Sophia is just a model for playing recordings, and its "object worship" makes people think that AI is juggling. Real AI has a long way to go to reach the IQ of babies and even animals.

2. Musk, Silicon Valley Iron Man, his hobby for the past two years is to talk everywhere that AI is going to destroy human beings. Yann LeCun thinks this statement is very irresponsible. In his view, Musk talked to some optimistic scientists, went home and read some imaginative books, and then came up with the idea that the Terminator was about to come true. In addition, it is well known that Musk always wanted to save mankind, which eventually led to the voices we heard. In Yann LeCun's view, scientists know that a strong AI is unlikely to happen in a few years, and Musk's remarks are spreading panic.

3. Last year, Filip Piekniewski, an expert in computer vision, published "Xiongwen" such as "Deep Learning is dead", singing that the deep learning bubble is about to burst. Yann LeCun immediately opened the retaliation mode, saying bluntly that the author was "very ignorant", pointing out that the author first did not see the reality of academic and industrial circles, and then used some irrelevant evidence to prove AI's conclusion that "winter is coming." For example, when the author mentioned AI pills, one big evidence is that AI scientists sent out less twitter-angry Yann LeCun quickly sent out a bunch of twitter.

To sum up, we will find that there are mainly three kinds of voices in Yann LeCun: fake AI, AI threat theory and AI winter theory.

In fact, looking around, are these three arguments also scattered around us?

Yoshua Bengio: defend Ivory Tower, defend Canada

Said the most high-profile, then talk about the lowest-key Yoshua Bengio.

He keeps a low profile because Yoshua Bengio is the least willing of the Big three to explore the industrial world. In the face of the inevitable temptation, he did not join the tech giants we are familiar with, but chose to stay in the ivory tower of the University of Montreal and enjoy the good mountains and waters of Canada.

However, as the deep learning that he invented became more and more popular, Yoshua Bengio found that things were not simple. Tech giants and investors are starting to rob schools like crazy. A Ph.D. in deep learning became hot until Yoshua Bengio discovered that his ungraduated PhD had been carved up by hungry technology companies. The scientist is determined to protect the purity of the ivory tower.

His solution is to start his own company.

Ahem, in fact, under the persuasion of several partners, Yoshua Bengio decided to jointly establish a new type of industry-university integration institution. We know that the AI world has the famous OpenAI, which focuses on open source projects with all commercial goals, so that scientists can realize their dreams in the enterprise.

Element AI, co-founded by Yoshua Bengio and its partners, is doing the opposite. It aims to allow AI scientists to directly participate in commercial projects and earn returns, while retaining faculty positions-such as spending only a few hours a week working on Element AI, and working on projects together to earn extra money.

This kind of online part-time model for AI scientists can effectively solve a problem: startups and traditional companies simply can't compete with tech giants for AI talent, but they really need AI talent to help. Such a model has the best of both worlds.

Soon, Element AI received investment from Microsoft, and now, in addition to customizing AI solutions in various industries, it has also begun joint research with large companies, as well as investment and technical assistance in high-quality AI projects. For customers, Yoshua Bengio himself is the gold-lettered signboard of business cooperation.

Another focus of Yoshua Bengio is the Montreal Institute of Learning algorithms (MILA) on campus. MILA and Element AI make up the dual-core drive of Montreal's AI industry. Today, the rapid development of AI in Canada and the Silicon Valley of Montreal, known as the AI era, are closely related to the work of Yoshua Bengio.

Well, at least for now, the guardian of the ivory tower, the star of Canada's AI, has done a good job.

Yoshua Bengio is also known for acting as an advocate in the field of AI's social responsibility and public welfare. For example, he took the lead in opposing Google's military program, calling for an end to the weaponization of AI. And actively promote attention to discrimination and unfairness in AI.

If you put three labels on Yoshua Bengio's work in recent years, they are: academic, commonweal, Canadian.

Geoffrey Hinton: skeptics, still skeptical

Compared with the two students in their 50s, Hinton, a 72-year-old teacher, seems to be free to enjoy the title of "AI Godfather", guide students and plan biographies.

However, this is not the case. Geoffrey Hinton is still working at a high intensity today. Plagued by lumbar disc disease, he even had to stand up to complete all his research. Compared with the two students and most of their successful peers, Hinton is more like the one working on the front line of AI.

To put it simply, that stubborn and bossy young man has now become a stubborn and bossy old man.

In an interview, Geoffrey Hinton was asked why he had survived decades of neglect, and his answer was very cool and very Hinton. He said:

"they are all wrong."

To this day, Hinton still thinks it's possible that everyone is wrong, including himself.

In 1986, Hinton published "Learning representations by back-propagation errors", which is one of the masterpieces of Hinton's life, which marks the introduction of back propagation algorithm into deep learning.

However, in the past two years, Hinton has repeatedly said that back propagation may have this huge defect. He has not only tried a variety of ways to break through it, but also put a lot of related research together to write a paper comparing how to get rid of the pattern of back propagation-until now, he has not surpassed himself, but it does not mean that he cannot do so in the future.

Geoffrey Hinton is an out-and-out skeptic, and that hasn't changed because he became a hero.

In the industrial world, Hinton's main job is Google's brain. In the past two years, Hinton and the team have been behind the simplification and upgrading of TensorFlow and the expansion of AI capabilities in Google's brain.

As the existence of "godfather", Hinton is paid more attention to the subversive views constantly put forward in the academic field of AI. It just so happens that this is another person who is willing to subvert the work of himself and others.

At the end of 2017, Hinton published a plan called capsule Network Capsule Networks, which is widely believed to rewrite the trajectory of deep learning.

The capsule network is aimed at the operation mode of convolution neural network. In the traditional deep learning algorithm, each layer of neural network must do the same convolution operation. The capsule network believes that different neurons can carry different attributes, just as different regions of the brain are responsible for different tasks.

This subversive scheme, which sparsely activates deep learning, has been proved to be innovative in the field of image recognition. Many people believe that capsule network will become the key technology that AI can explain and AI is endowed with common sense in the future.

In recent years, another subversion brought about by Hinton is the continuous work in the field of dark knowledge extraction dark knowledge extraction. Generally speaking, deep learning to obtain abstract features is based on huge data operations. As a result, AI has to consume a lot of data and computing power to complete the training repeatedly. On the other hand, dark knowledge extraction, or knowledge distillation, is committed to enabling agents to extract hidden knowledge, retain part of the knowledge into sub-deep learning systems, and finally achieve agents to get rid of huge computing power and data cravings. touch the relatively innate "intelligence".

As you can see, Hinton is still as hard as it is today. A lot of things that seem like common sense in the AI world, the father of AI doesn't believe it at all and is challenging it again and again.

Has AI come to an end yet? Is deep learning the final solution? The old man never trusted the judgment of most people in his life.

Today of the Big three: deep learning, from 1 to many

Let's say that the years when the Big three held torches high and no one was around was the era of deep learning from 0 to 1.

So today's deep learning craze around the world undoubtedly marks the beginning of deep learning from 1 to N. However, judging from the work of the Big three today, it has only reached 1.

I don't know if you have noticed that the main focus of the Big three today is precisely corresponding to the congenital deficiencies and acquired problems of this AI renaissance represented by deep learning. By forcibly merging the work of the Big three, we can see several directions:

1. Is AI empty talk or fact? To solve this problem, we must throw deep learning into the industrial melting pot and test what deep learning can do in computing, data and application scenarios. And this is one of the logic that a large number of AI scientists must go to enterprise.

2. After AI becomes popular, all kinds of messy things will come out and fly, and someone will need to pull the train back to the track. That's why Yann LeCun opened fire on social media. Sophia's tricks, Musk's AI threat theory, and "the cold winter of AI is coming again" must have a large number of supporters in China today. It is not difficult to see that the problem is quite acute.

3. The matching and balance of talents in AI. One of the characteristics of this round of AI revival is the high degree of industry-university integration, which leads to academic talents being able to communicate directly with applications. However, there are also problems in China that how academic talents maintain their academic pursuit in the face of industrial temptation, and how the industry can get help from AI talents under the competition of giants.

4. Deep learning is a double-edged sword, which leads to the problems of militarization, discrimination and injustice, factor security and so on. The social responsibility of AI is a contradiction that Brooks no delay.

5. Deep learning is not the end. Back propagation, multi-layer neural networks and other technical models constitute the "AI" that we are used to. However, there are still a lot of problems in deep learning, such as black box, poor transfer ability, high consumption and so on. Should we regard today's AI as a fundamentalist tenet, or continue to challenge and find a higher end? Hinton, the "father of deep learning", is really a good leader in this respect.

Of course, the achievements are great, and there are still a lot of problems. Today, the Big three are still working, and they are responsible for their creation.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.