Shulou(Shulou.com)06/02 Report--

2019-11-18 10:53:03

Machine Heart report

Machine Heart editorial Department

In 2012, Alex Krizhevsky, a student of Turing Award winner Geoffrey Hinton, one of the Big three in deep learning, proposed AlexNet and won the championship by a significant margin in that year's ILSVRC (ImageNet large-scale Vision Challenge), far surpassing the second place. This achievement has aroused great concern in the academic and industry, and computer vision has gradually entered an era dominated by deep learning. But such an epoch-making study has also been questioned recently.

Recently, some netizens claimed on reddit that DanNet (also a CUDA-based convolution neural network) proposed by Dan Ciresan of the Jurgen Schmidhuber team completed four image recognition challenges before AlexNet.

Strong evidence that DanNet predates AlexNet

The poster showed the following evidence in reddit:

1. The poster saw the clue from the references in chapter 19 of "Deep Learning: Our Miraculous Year 1990-1991" published by Jurgen.

two。 The poster said that before AlexNet (winner of the 2012 ImageNet competition), the Romanian postdoctoral Dan Ciresan of the Jurgen team had won four important computer vision competitions between May 15, 2011 and September 10, 2012, using a CUDA CNN called DanNet.

The red-framed IDSIA, the team of Dan Ciresan and others, won four important computer vision competitions in Chinese writing, traffic signs, brain segmentation and cancer detection between May 15, 2011 and September 10, 2012.

3. The poster has seen news reports that AlexNet started the deep learning revolution in 2012, but in fact, according to Jurgen, DanNet became the first method to win the superhuman visual pattern recognition competition in 2011, and also won the medical imaging competition with a larger image than AlexNet.

Dan Ciresan won the superhuman visual pattern recognition competition for the first time.

Dan Ciresan won the mitotic detection competition for histological images of breast cancer.

4. DanNet's most frequently cited paper, "Multi-column Deep Neural Networks for Image Classification" (July of CVPR,2012), was 5 months earlier than "ImageNet Classification with Deep Convolutional Neural Networks", which introduced AlexNet (December of NIPS,2012), but earlier papers on DanNet appeared at IJCAI 2011 and IJCNN 2011.

Dan Ciresan's paper.

Alex Krizhevsky's paper.

5. To be fair, AlexNet quoted DanNet and admitted that the two were similar, but AlexNet did not mention that DanNet had won four computer vision challenges before it

6. ResNet beat AlexNet in the ImageNet competition in 2015, but ResNet was actually a special case of the earlier Highway networks, which was also first proposed by Jurgen Labs. In "the first feasible forward communication network with more than 100 layers", Jurgen credits their leading research to their students Rupesh Kumar Srivastava and Klaus Greff.

Jurgen believes that Microsoft's ResNet is a special case of Highway Nets proposed by his team.

7. In the fifth chapter of the Jurgen article, he details "the Origin of GAN", while in the fourth chapter, he introduces the successful LSTM in 2009. The above is familiar to everyone, but most people may not know that the Jurgen team was the first to launch CNN on CUDA and was successful.

These are some of the documentary evidence that the poster believes that DanNet preceded AlexNet.

The world owes Schmidhuber a Turing Award?

So it seems that many of the concepts of deep learning today were played by J ü rgen, the father of LSTM 30 years ago? The lively discussion on Reddit stems from an article published by J ü rgen Schmidhuber in October detailing much of the research he and his team did nearly 30 years ago (between 1990 and 1991). According to him, the research ideas have laid the foundation for many cutting-edge studies of deep learning today, including LSTM, meta-learning, forgetting door mechanism, attention and reinforcement learning, etc.

The impression of J ü rgen Schmidhuber is usually "the father of LSTM". He is from Germany and is now the head of the Dalle Molle Institute of artificial Intelligence in Switzerland. He is a famous scholar in the field of artificial intelligence. In addition to LSTM, he has always believed that the GAN model, which has developed rapidly in recent years, is a variant of the PM model proposed as early as 1992. In addition, he has made great contributions in areas such as speech recognition.

In March this year, after the Turing Award, the highest honor in the computer field, was awarded to the three giants of deep learning, Geoffrey Hinton, Yoshua Bengio and Yann LeCun, the discussion of "academia owes J ü rgen a Turing Award" became lively.

Zhou Zhihua, a famous scholar of artificial intelligence and dean of the School of artificial Intelligence of Nanjing University, also said: "Hinton undoubtedly ranks first in terms of contribution to deep learning, and both LeCun and Schmidhuber have made great contributions. But HLB is always tied together, and S and HLB are not right. To win the prize, there must be nominations and votes, and popularity is also important. But it doesn't matter, the textbook-level contribution of LSTM is enough to calm down. "

J ü rgen himself has always complained that his academic status has been "unfairly treated". In recent years, he has always promoted his creative research on various occasions, even at the expense of public confrontation with other famous scholars. (you may still remember J ü rgen's fight against Ian Goodfellow, the "father of GAN" at the artificial intelligence summit NIPS 2016). Therefore, people's views on J ü rgen are also polarized.

Today's discussion is no exception. When DanNet was ahead of AlexNet at the beginning of the discussion, the dominant point of view was as follows:

All right, J ü rgen is our Creator.

But on the whole, the ridicule of personal character still has to give way to reason, and people finally think that J ü rgen Schmidhuber has indeed suffered a loss of reputation.

Face up to his contribution, netizens said:

Although we are all gloating, J ü rgen does deserve the Turing Prize. Outside of LSTM, many of his studies are impressive.

In the field of computer science, there are many scholars with perverse personalities, and maverick behavior is always unpopular. But I always wonder that people use this reason to judge their academic contributions.

I think the current North American-centric CS academic system completely suppresses similar contributions from other research institutions around the world.

Reference link:

Https://www.reddit.com/r/MachineLearning/comments/dwnuwh/d_dannet_the_cuda_cnn_of_dan_ciresan_in_jurgen/http://blog.itpub.net/31077337/viewspace-2158712/

Https://www.toutiao.com/i6760474106769965575/

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.