Shulou(Shulou.com)06/01 Report--

CaffeOnSpark's example analysis, aiming at this problem, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and feasible way.

Another technology giant has unveiled its know-how artificial intelligence to the world. On February 25th, Yahoo released the source code of CaffeOnSpark artificial intelligence, which can be used or modified by every academic researcher or large company.

Little is known about Yahoo's technological prowess. In fact, open source Hadoop is incubated by Yahoo; Facebook, Twitter, and many other companies are using this data processing platform. So Yahoo has unique strengths in artificial intelligence, because data is as important as algorithms when training artificial intelligence systems. And Yahoo holds one of the most interesting databases in the world: the photo community site Flickr.

Like many new open source artificial intelligence projects, CafffeOnSpark is based on deep learning. Deep learning is a branch of artificial intelligence, which is famous for its recognition ability of human voice, photos and videos. Yahoo uses this technology to identify different content in photos to improve search results on Flickr. Unlike other sites, Flickr's image search does not rely on image descriptions or keywords entered by users. Yahoo teaches computers how to recognize certain features of photos, such as specific colors, or even objects or animals.

In recent months, Google has opened the source code of the deep learning network TensorFlow, Microsoft has opened a similar network CNTK,Facebook to share artificial intelligence hardware design, and Chinese search engine giant Baidu has also released deep learning training software.

These open source technology projects all have their own emphasis and bias. The goal of Yahoo, for example, is to run deep learning on existing systems to avoid transferring data from one place to another. Andy Fent, vice president of architecture at Yahoo, explained to us that training deep learning systems to recognize pictures requires a huge amount of data information. You have to put a large number of examples into the algorithm, the more the better, for example, if it is a cat, the system needs enough graphs to "learn" to identify common features between cats and to distinguish cats from other animals.

There are thousands of pictures on Flickr, enough to train artificial intelligence. But the development team did not want to transfer these images from the Flickr server to another server running deep learning software. So they invented a way to run deep learning software on existing architectures.

In terms of name, CaffeOnSpark combines two existing technologies: the deep learning framework Caffe and the large-scale data processing system Spark. What Yahoo does is find a way to run Caffe at the Spark level. Once the method is found, Caffe can run not only on Spark, but also both on Hadoop. The development of Yahoo will not only allow artificial intelligence developers to use simpler and familiar tools and save the hassle of transmitting data, but also make it easier for deep learning to process the content of several servers at the same time. Feng also deliberately told us that Google's TensorFlow can't do this yet, and Yahoo is one step ahead.

So much for the answers to the sample analysis questions about CaffeOnSpark. I hope the above content can be of some help to you. If you still have a lot of doubts to solve, you can follow the industry information channel for more related knowledge.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.