Shulou(Shulou.com)11/24 Report--

The computing power of the AI model doubles every 100 days, far exceeding the 18-24 months of Moore's Law. Can intelligent computing break this computational dilemma?

Artificial intelligence is a "financial struggle" industry, without high-performance computing equipment, let alone the development of basic models, even fine-tuning models can not be done.

However, if we only rely on hardware and the current development speed of computing performance, sooner or later, we will not be able to meet the expanding demand, so we still need supporting software to coordinate the overall computing power. At this time, we need to use "intelligent computing" technology.

Recently, as many as 12 domestic and foreign research institutions, such as Zhijiang Laboratory, Chinese Academy of Engineering, University of National Defense Science and Technology, Zhejiang University, and so on, jointly published a paper, which conducted a comprehensive survey in the field of intelligent computing for the first time. It covers the theoretical basis, the technological integration of intelligence and computing, important applications, challenges and future prospects.

Paper link: https://spj.science.org/ doi / 10.34133 / icomputing.0006 this is also the first review article that formally puts forward the definition of intelligent computing and its unified theoretical framework. The full text is structured as follows.

AI ushered in the era of large-scale computing. Human society is moving from an information society to an intelligent society. Computing has become a key factor in standardizing and promoting social development. In the new era of digital civilization of the Internet of everything, traditional data computing is far from being able to meet the growing demand for a higher level of intelligence.

People's increasing interest in intelligent computing, coupled with the development of computing science, the intelligent perception of the physical world and the understanding of the cognitive mechanism of human consciousness, have jointly improved the intelligence level of computing and accelerated the discovery and creation of knowledge.

In recent years, with the rapid development of computing and information technology, artificial intelligence (AI) has been established as the frontier field of human exploration of machine intelligence due to the unprecedented popularity and success of deep learning. On this basis, a series of breakthrough research results have been achieved, including:

Yann LeCun's convolutional neural network (CNN); Yoshua Bengio's contribution in the field of causal reasoning in deep learning; Geoffrey Hinton, one of the pioneers of artificial intelligence, proposed the deep confidence network (Deep Brief Network) model and back propagation optimization algorithm in 2006.

J urgen Schmidhuber proposed the widely used cyclic neural network (RNN) and long-term and short-term memory (LSTM), which are successfully used to deal with sequence data, such as voice, video and time series data.

In March 2016, DeepMind launched the artificial intelligence go program AlphaGo against the world's top human go master Lee se-dol, which attracted unprecedented attention all over the world. This epoch-making man-machine war ended in an overwhelming victory of artificial intelligence and became a catalyst to push the wave of artificial intelligence to a new height.

Another important promoter of artificial intelligence is the emergence of large-scale pre-training models, which have been widely used in natural language and image processing, dealing with a variety of applications with the help of transfer learning.

For example, GPT-3 has proved that a large model with high structural complexity and a large number of parameters can improve the performance of deep learning. Inspired by GPT-3, a large number of large-scale deep learning models have emerged.

Intelligence and computing power is one of the important factors supporting intelligent computing.

In view of the astronomical data sources, heterogeneous hardware configurations and ever-changing computing requirements in the information society, intelligent computing mainly meets the computing power requirements of intelligent tasks through vertical and horizontal architecture.

Vertical architecture (vertical architectures) is characterized by a homogeneous computing infrastructure, mainly through the application of intelligent methods to improve resource utilization efficiency to improve computing power.

In contrast, horizontal architecture (horizontal architecture) coordinates and arranges heterogeneous and wide area (wide-area) computing resources to maximize the efficiency of collaborative computing.

For example, in April 2020, in response to the computing needs of global COVID-19 research, Folding@home teamed up with 400000 computing volunteers to achieve the computing power of 2.5Exaflops, which is more powerful than any other supercomputer in the world.

Although great success has been achieved in intelligence and computing, these two areas still face some difficulties.

The challenge of intelligence using deep learning artificial intelligence is still unsolved in terms of interpretability, versatility, evolutionability and autonomy.

At present, compared with human intelligence, most artificial intelligence technologies can only play a weak role, and only play a role in specific fields or tasks. There is still a long way to go to achieve powerful and universal artificial intelligence.

Finally, there are also major theoretical and technical challenges to upgrade from data-based intelligence to more diversified forms of intelligence, including perceptual intelligence, cognitive intelligence, autonomous intelligence and man-machine fusion intelligence.

The challenge of digital computing has brought unprecedented growth in applications, connections, terminals and users, as well as the amount of data generated, which requires huge computing power.

For example, the computing power required for artificial intelligence doubles every 100 days, that is, it is expected to increase more than 1 million times over the next five years.

As Moore's Law gradually fails, it becomes challenging to keep up with such fast-growing computing power requirements.

Moore's Law: the number of transistors that can be contained in integrated circuits doubles every two years. The processing of large-scale tasks in an intelligent society depends on the effective combination of specific computing resources. The traditional hardware model can not well adapt to intelligent algorithms, which greatly limits the development of software.

What is intelligent computing? Up to now, there is still no widely accepted definition of intelligent computing.

Some researchers regard intelligent computing as a combination of artificial intelligence and computing technology, but this view limits the definition of intelligent computing to the field of artificial intelligence. at the same time, it ignores the inherent limitations of artificial intelligence and the important role of ternary interaction among human beings, machines and things.

Another school regards intelligent computing (intelligent computing) as computational intelligence (computational intelligence), imitates human or biological intelligence to achieve the optimal algorithm to solve specific problems, and regards intelligent computing as a kind of algorithm innovation.

In this paper, researchers put forward a new definition of intelligent computing from the point of view of solving complex scientific and social problems, taking into account the three basic spaces of the world, that is, the increasingly close integration of human social space, physical space and information space.

Intelligent Computing's definition of intelligent computing refers to the field of new computing theories and methods, architecture systems and technical capabilities in the era of digital civilization that supports the interconnection of the world. According to the specific actual needs, intelligent computing completes the computing task with the minimum cost, matches enough computing power, calls the best algorithm, and can get the best results.

The new definition of intelligent computing is aimed at the fast-growing computing demand in the integration of human society, physical world and information space. Intelligent computing is people-oriented, pursuing high computing power, high energy efficiency, high intelligence and high security.

Its goal is to provide universal, efficient, secure, autonomous, reliable and transparent computing services to support large-scale and complex computing tasks. Figure 1 shows the overall theoretical framework of intelligent computing, which embodies various computing examples that support human-physical-information integration.

First of all, intelligent computing is neither a substitute for existing computers, cloud computing, edge computing and other computing technologies, such as neural morphological computing, optoelectronic computing and quantum computing, nor is it a simple integration. On the contrary, it is a form of computing that solves practical problems by systematically and comprehensively optimizing existing computing methods and resources according to task requirements.

By contrast, existing major computing disciplines, such as supercomputing, cloud computing, and edge computing, belong to different areas: supercomputing aims to achieve high computing power, cloud computing emphasizes the convenience of cross-platform / devices, while edge computing pursues quality of service and transmission efficiency.

Smart computing dynamically coordinates data storage, communication and computing among edge computing, cloud computing and supercomputing, and builds a variety of cross-domain intelligent computing systems to support end-to-end cloud collaboration, cloud-to-cloud collaboration and supercomputing interconnection.

Intelligent computing should make full use of existing computing technologies, and more importantly, promote the formation of new intelligent computing theories, architectures, algorithms and systems.

Secondly, the concept of intelligent computing is proposed to solve the problems in the development of human-physical-information space fusion in the future.

With the development of the application of information technology in big data era, the boundary between physical space, digital space and human society has become more and more blurred.

The human world has evolved into a new space, which is characterized by the close integration of human beings, machines and things. Social systems, information systems and physical environment constitute a large system of dynamic coupling, in which people, machines and things integrate and interact in a highly complex way, which promotes the development and innovation of new computing technologies and application scenarios in the future.

Reference:

Https://spj.science.org/doi/10.34133/icomputing.0006

This article comes from the official account of Wechat: Xin Zhiyuan (ID:AI_era)

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.