Shulou(Shulou.com)11/24 Report--

Human beings are highly dependent on the perception of sound, even higher than visual perception from some angles. Helen Keller said: blind isolates people from things, and deafness isolates people from people. It can be said that sound is the most important means of human interaction. People are highly dependent on sound from birth to death. Ears are sensing devices that never turn off all their lives. They remain sharp even in sleep, receiving all the sounds in the environment and transmitting them to the brain all the time. Therefore, in the era of artificial intelligence, sound will also be one of the most important means of human-computer interaction.

It is precisely because human beings are highly dependent on sound perception that the pursuit of beautiful sound is never-ending. From the earliest phonograph to the tape recorder, from CD to MP3, from Bluetooth speakers to TWS headphones, people's pursuit of high-quality audio experience never stops, and the wishes of enthusiasts yesterday often become the standard for ordinary people tomorrow. Accompanied by the continuous demand for portable, miniaturized and long standby audio products.

History has proved that although audio products have a long history, they often lead the fashion in every era. From the early phonograph to the tape recorder in the 1980s became a fashion at home and on the street, from the three turns and one sound in the 1970s to the eight big pieces as a standard for marriage in the 1980s and 1990s, there are historical traces of audio products leading the fashion. Apple is one of the most innovative companies in the world, changing people's lives with disruptive innovations, including twice successfully leading the fashion with audio products, one with iPod leading the new experience of MP3 portable audio, and then Airpods leading the TWS headset trend.

In recent years, smart speakers marked by Amazon Alexa are all the rage, bringing a new experience of man-machine voice interaction. The new generation of ChatGPT based on voice interaction gives everyone endless room to imagine the future AI experience. I believe that in the AI era, AI audio products will still lead the fashion of the AI era, the earliest to take root. And the chip, is to make these fashion-leading audio products become the core of reality, audio AI will once again drive chip leaping innovation.

On November 10, 2023, Dr. Zhou Zhengyu, Chairman and CEO of Torch Core Technology Co., Ltd., was invited to attend the 2023rd Annual meeting of China Integrated Circuit Design Industry (ICCAD2023). Combined with the development trend of audio field and the upsurge of AI era, he shared how portable products create high computing power in the AI era, and delivered a keynote speech: "renovated voice vitality: AI-driven audio chip innovation".

Building high computing power under the premise of low power consumption of ● is the core foundation of portable AI audio SoC.

Dr. Zhou Zhengyu pointed out: in the AI era, for audio wearable or audio portable products, the challenge and opportunity to enhance the AI experience is how to create as much computing power as possible per milliwatt of power consumption, rather than simply pursuing the absolute value of computing power. For battery-powered portable audio or wearable products, the core demand for successful AI is to create large computing power at low power consumption in order to achieve a better AI experience.

Computing power and power consumption are contradictory unity, which inevitably requires greater power consumption, and the increase of power consumption has become an obstacle to the improvement of computing power. Dr. Zhou Zhengyu mentioned, "to achieve an order of magnitude increase in computing power per unit of mW, we must not only expect and rely on advanced technology, but also innovate in computing architecture and chip circuit implementation."

Take the two most typical wearable products: TWS headphones and smartwatch as an example, based on 4.2V lithium battery power, the typical full function SoC average working current is generally 3-5mA In other words, the overall power consumption budget of SoC, the core of portable audio or wearable products, is less than 15-20mW. If there is no revolutionary change in battery technology, portable audio or wearable products SoC should create more AI computing power with a power budget of 10mW or less.

Through the systematic analysis and induction of the computing power required by the well-known AI models and algorithms in different fields, the computing power of AI audio model is basically below 1TOPS, and the typical computing power of 200-500GOPS can provide a good audio AI experience. Therefore, the challenge we face is how to build 200-500GOPS AI computing power at less power consumption than 10mW. The computing power of 200-500GOPS does not seem to be much of a challenge, and almost all NPU IP can be achieved, but it is extremely challenging to achieve it within a power budget below 10mW.

Taking the published data of NPU Weekly based on traditional computing architecture as an example, its energy-efficiency ratio under 28nm is about 2TOPS / W, that is, 200-500GOPS requires 100-250mW power consumption, which is more than 10-25 times higher than the power budget below 10mW. Even with 7nm, Zhouyi can achieve 10TOPS / W, but the power consumption is still 20-50mW, which is only a reasonable order of magnitude and is still 2-5 times higher. In other words, even if the advanced 7nm process is used, it can only provide much lower computing power than the target value under the power consumption limit.

In other words, to achieve the goal of "creating 200-500GOPS audio AI computing power under the power budget of 10mW", the traditional von Neumann computing architecture must rely on a more advanced process than 7nm, such as 5nm or 3nm, and this still assumes that traditional NPU can achieve 100% computing efficiency, not limited by "memory wall" and "power wall".

The von Neumann computing architecture is characterized by the complete separation of storage and computing. Because the design of the processor is mainly to improve the computing speed, storage pays more attention to capacity improvement and cost optimization, and the performance mismatch between "memory" and "computing" leads to low memory access bandwidth, time extension, high power consumption and other problems, the main power consumption and the data transfer between memory and computing bottleneck. It is commonly referred to as "storage wall" and "power wall". The more intensive the access to memory, the faster the speed, the more serious the problem of "wall" is, and the more difficult it is to improve the calculation power.

In order to break through the "memory wall" and "power consumption wall" of von Neumann architecture, Compute-In-Memory (CIM) is a potential technology path. In the process of chip design, we no longer distinguish between memory unit and computing unit, realize the integration of deposit and calculation, realize the calculation on the memory unit, directly eliminate the boundary of "memory" and "calculation", and improve the computing energy efficiency by an order of magnitude. Such an extreme nearest neighbor layout basically completely eliminates data movement delay and power consumption, and is the key technology to solve the problem of memory wall and power wall under the traditional von Neumann architecture.

At present, the development of CIM based on several of the most popular storage media, such as Flash, DRAM, SRAM and other emerging Memory has its own advantages and disadvantages and reasonable application fields. The biggest problem of CIM based on Flash is that the writing speed is slow and the number of writes is limited. After writing many times, the chip is broken. At the same time, it needs to use special technology. Although it is mature, it is not suitable to integrate with other circuits of SoC, so it is not an ideal choice for CIM technology. Although there is no limit on the number of writes of CIM based on DRAM, the relative energy efficiency ratio is the lowest among the four media, which is not suitable for low-power computing. At the same time, DRAM is also a special process that cannot be integrated in SoC, but it has the advantage of high density, so it is suitable for cloud computing and servers to create super computing power. Although the emerging storage media such as RRAM and MRAM have many advantages in theory and may surprise everyone in the future, the current process is extremely immature and has not yet reached the stage of mass production.

The biggest disadvantage of SRAM-based CIM is that low density is not suitable for super-large computing power (such as AI applications with dozens of TOPS). However, audio AI applications discussed above do not need super computing power (only 0.2-0.5TOPS), which effectively avoids the weakness of relatively low density of SRAM. 、

Therefore, in the application of building the computing power of audio products at low power consumption, SRAM-based CIM has very significant technical advantages including:

1. High energy efficiency, low power consumption, fast reading and writing speed, suitable for low power consumption and high performance devices.

2. There is no limit on the number of writes, which is suitable for self-learning or adaptive AI models and algorithms that are adjusted repeatedly. It is also convenient to support adaptive adjustment and time-sharing processing of multiple neural network algorithms that need to switch models frequently.

3. The process is mature and can be mass produced on a large scale. Standard CMOS process is mature, stable and universal, and all FAB can be produced on a large scale.

4. The process is leading and suitable for integration. It is easy to use the most advanced process nodes, and it is convenient to realize single-core SoC integration in any process node.

Dr. Zhou Zhengyu concluded: for low-power audio SoC, SRAM-based CIM is currently the first choice to build low-power audio AI computing.

● provides audio AI computing power in a portable or wearable product with a very low power budget.

There are two mainstream implementation methods of CIM circuit based on SRAM, one is based on pure analog design, the other is based on analog-digital mixed design. The main difference between them is that the operation unit of analog CIM uses ADC and analog multiplier and adder to realize operation unit, while the operation unit of analog-digital mixed CIM uses custom-design to fuse memory unit and logic operation unit to realize digital. Dr. Zhou Zhengyu proposed that torch core technology chose the technical path of SRAM in-memory computing (Mixed-Mode SRAM based CIM, referred to as MMSCIM) based on analog-digital hybrid circuits, which has the advantages of both analog and digital circuits, and of course the barriers to design are relatively high.

Compared to the design idea of simulating CIM, MMSCIM has several obvious advantages:

The accuracy is lossless, the simulation will be disturbed by circuit noise and environmental factors, the calculated results are not completely consistent, and the accuracy is lost.

Digital operation unit has high reliability and high production consistency, which is the inherent advantage of digitalization.

Easy to upgrade process and design conversion between different FAB.

It is easy to improve speed and optimize performance / power consumption / area (PPA).

The reading and calculation of data are synchronized in SRAM, and the energy efficiency ratio is higher.

The sparsity of the adaptive model is easy to improve the energy efficiency.

Based on the realized Testchip test and estimation results, MMSCIM can reach 7.8TOPS / W under 22nm process, and NPU;MMSCIM, which is close to the traditional architecture implemented by 7nm advanced process, is expected to reach 15.6TOPS / W under 16nm, which is higher than NPU under 7nm advanced process. However, the power consumption below 10mW is still not enough to create 200-500GOPS computing power, so technology needs to continue to innovate.

Dr. Zhou Zhengyu proposed that using the sparsity of the AI matrix to calculate the energy efficiency ratio will be an important breakthrough. Most of the AI models of audio algorithms have the characteristics of matrix sparsity, that is, many model parameters are zero, so they do not have to operate if they encounter zero, so as to save power consumption. Traditional NPU can perform Skip-Zero techniques through special logic circuit design to reduce power consumption. However, this kind of Skip-Zero logic circuit is relatively easy to implement for one-dimensional AI operator, but faced with 2D operator, it is more challenging to implement, and it needs to pay additional logic circuit cost and power consumption, which makes the energy efficiency improved by Skip Zero discounted.

On the other hand, MMSCIM has the characteristic of adaptive sparse matrix, and the multiplication unit does not consume power when it encounters zero input. No matter 1D or 2D operator, it can naturally achieve the effect of Skip-Zero without the help of additional logic circuit, which makes MMSCIM technology naturally achieve a better energy efficiency ratio.

Through simulation analysis, MMSCIM can achieve the energy efficiency ratio of 24.5TOPS / W-70.38TOPS / W in 22nm when the matrix sparsity is in the range of 50% and 80%, and the corresponding 10mW power consumption can create 245GOPS-704GOPS computing power. Under the implementation of 12nm, when the matrix sparsity is in the range of 20% 46.9TOPS 50%, the energy-efficiency ratio reaches 23.5-46.9TOPS / W, and the computing power of the corresponding 10mW can reach 235GOPS-469GOPS.

Therefore, SRAM based in-memory computing (MMSCIM) based on analog-digital hybrid design, under the blessing of sparse matrix, can achieve the goal of providing audio AI computing power for portable audio or wearable products under a very low power budget, that is, "to build 200-500GOPS audio AI computing power under the 10mW power budget", and can achieve rapid mass production. The energy efficiency ratio of 22nm MMSCIM is expected to exceed the NPU,12nm of 7nm's traditional von Neumann architecture by a large margin than the NPU of 7nm's traditional architecture.

● Torch Core Technology will launch the latest high-end AI audio chip based on MMSCIM

Dr. Zhou Zhengyu finally shared and concluded that audio AI will once again drive the innovation of chip technology, especially the innovation of SoC technology, and the main basis of this innovation is how to provide large computing power on the premise of low power consumption on end-side portable products. This is the challenge faced by wearable and portable SoC and terminal products in the AI era. Only persistent innovation can help us break through the dilemma. It also brings a huge market opportunity for domestic end-side AI audio chips.

For a long time, Torch Core Technology is committed to creating low-latency and high sound quality technology based on CPU+DSP dual-core heterogeneous audio processing architecture. Torch Core Technology will comply with the development trend of artificial intelligence, start with high-end audio chips, integrate low-power AI acceleration engine, and gradually upgrade to CPU+DSP+NPU (based MMSCIM) tri-core heterogeneous AI SoC architecture, providing more computing power for portable products. It will soon bring high-quality improvement to AI noise reduction, voice separation, voice isolation and other applications, and will also be widely used in intelligent audio, intelligent office, intelligent education, intelligent accompany and other market fields.

Torch Core Technology will launch the latest generation of MMSCIM-based high-end AI audio chip ATS286X, which is expected to start Sample in 2024.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.