Shulou(Shulou.com)12/24 Report--

CTOnews.com December 16 news, Apple recently released DeepPCR machine learning algorithm, by parallel processing of conventional sequential operations, can accelerate the reasoning and training of neural networks.

Neural networks have been able to handle complex tasks such as text or image composition, segmentation and classification. However, due to the computational demands, it may take days or weeks for the neural network to train and feed back results.

Parallel technology is widely used in neural network processing, which can accelerate training and reasoning speed.

While some operations in neural networks are still done sequentially, diffusion models generate outputs through a series of denoising stages and pass them forward and backward layer by layer, and as the number of steps increases, sequential execution of these processes becomes computationally expensive and can lead to computational bottlenecks.

In order to solve this problem, Apple's research team introduced DeepPCR algorithm, which further accelerated the training and reasoning of neural networks.

The team employed a parallel loop reduction (PCR) algorithm to retrieve the solution, reducing the computational cost of the sequential process from O (L) to O (log2 L), reducing complexity and increasing runtime speed.

The team said that after deploying DeepPCR algorithms in multilayer perceptrons, forward and backward passes were parallelized, achieving up to 30 times the forward transfer speed and up to 200 times the backward transfer speed.

CTOnews.com attaches the following main conclusions of DeepPCR algorithm:

DeepPCR is an innovative method for parallelizing sequential processes in neural network training and reasoning. Its main feature is that it can reduce the computational complexity from O (L) to O (log2 L), where L is the sequence length.

DeepPCR has been used to parallelize forward and backward propagation in multilayer perceptrons (MLPs), and extensive analysis of the performance of the technique has also been performed to determine the high performance state of the method while taking into account basic design parameters.

DeepPCR has been used to speed up deep ResNet training on MNIST and the generation of diffusion models trained on MNIST, CIFAR-10 and CelebA datasets. The results show that although DeepPCR shows significant acceleration, it improves the data recovery speed of ResNet training by 7× and the diffusion model creation speed by 11×.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.