Shulou(Shulou.com)06/03 Report--

This article mainly explains "What are the new features of MindSpore". The explanation in this article is simple and clear, easy to learn and understand. Please follow the ideas of Xiaobian to study and learn "What are the new features of MindSpore" together!

In deep learning, when the scale of data sets and parameters becomes larger and larger, the time and hardware resources required for training will increase, and finally it will become a bottleneck restricting training. Distributed parallel training is an important optimization method for training, which can reduce the requirements for memory and computing performance. The MindSpore dynamic graph mode supports data parallelism. By dividing the data according to batch dimensions and distributing the data to each calculation unit for model training, the training time is shortened.

Based on ResNet50 v1.5+ImageNet dataset test, MindSpore dynamic graph mode distributed performance can reach 1.6 times of PyTorch typical distributed scenario, static graph mode distributed performance can also reach 2 times of TensorFlow typical distributed scenario.

Data preprocessing accelerates Dvpp data as the foundation for machine learning. In the network reasoning scenario, we need to preprocess the data for different data, filter out the core information and put it into our trained model for reasoning and prediction. In practical application scenarios, we often need to reason about a large amount of raw data, such as real-time video streams. Therefore, we introduce Dvpp module in ascending inference platform to accelerate the data preprocessing process for network inference.

Dvpp data preprocessing module provides C++ interface, provides image decoding, scaling, center matting, standardization and other functions. In the design of Dvpp module, considering the overall ease of use, its functions overlap with MindData's existing CPU operators. We unify its API and distinguish it by setting the running equipment through inference execution interface. Users can choose the best operator according to their own hardware environment. The Dvpp data preprocessing process is shown in the figure below:

We test the performance gains of Dvpp series operators on an ascending inference server. The server has 128 CPU cores clocked at 2.6 GHz and 128Gb of memory space. In the experiment, we select the yoloV3 network and select 40504 images from the coco2017 inference dataset for inference, and finally get the image with model input size [416, 416].

We use Dvpp operator and CPU operator to preprocess data respectively, and get the following performance comparison:

It can be seen that Dvpp series operators have obvious performance advantages compared with CPU operators when processing a large amount of data. In this experiment, the performance of processing 40504 pictures is improved by 129%.

II. Innovative Molecular Simulation Library (SPONGE), from MindSpore version of Community Molecular Dynamics Working Group SPONGE is a molecular simulation library jointly developed by Gao Yiqin Research Group of Peking University and Shenzhen Bay Laboratory and Huawei MindSpore team in Molecular Dynamics Working Group (MM WG) in the community, with high performance and modularization.

Why do we need SPONGE?

Molecular dynamics simulation is a computer simulation method that uses Newton's law approximation to describe microscopic atomic and molecular scale evolution. It can be used for both basic scientific research and industrial practical applications. In the field of basic science, molecular dynamics helps researchers to study the physicochemical properties of systems from the microscopic perspective.

In industrial production, it can use large-scale computing capabilities to assist drug molecule design and protein target search [1, 2]. Due to the limitation of time and space scale of simulation, the application range of traditional molecular dynamics software is limited greatly. Researchers are also constantly developing new force field models [3,4], sampling methods [5,6], and attempts to incorporate emerging artificial intelligence [7,8] to further expand the applicability of molecular dynamics simulations.

As a result, a new generation of molecular dynamics software needs to be put on the agenda. It should be modular in nature, allowing scientists to efficiently create and build structures that can validate their theoretical models. At the same time, it also needs to balance the efficiency of traditional simulation methods and be compatible with their use in traditional fields. In addition, in order to realize the natural fusion of molecular simulation + machine learning, it should also have the form of embedding artificial intelligence framework. SPONGE is a brand new, completely autonomous molecular simulation software created based on these ideas.

SPONGE natively supports SITS and optimizes the computational process to make it more efficient to simulate biological systems using SITS methods than previous approaches to biomolecular enhancement sampling using SITS methods on traditional molecular simulation software [9]. For polarized systems, traditional molecular simulation uses methods such as combination of quantitative calculation to solve problems such as charge floating [10]. Even using machine learning to reduce computation costs wastes a lot of time on program data transfer problems. SPONGE, on the other hand, uses modular features to support direct communication with machine learning programs on memory, greatly reducing overall computing time.

Figure 1: Na[CpG], Lys Biomolecular Simulation in Combination with SITS Methods

Figure 2: Machine learning + molecular simulation method can simulate polarization system faster and more accurately. Figure 2:[C1MIm]Cl ionic liquid simulation

MindSpore + SPONGE

Based on MindSpore's automatic parallelism and graph-calculation fusion, SPONGE can efficiently complete the traditional molecular simulation process. SPONGE uses MindSpore's automatic differentiation feature to combine AI methods such as neural networks with traditional molecular simulations.

SPONGE Modular Design Structure Diagram

SPONGE, which is open sourced with MindSpore 1.2, has the following advantages:

1. Fully modular molecular simulation. Modular construction of molecular simulation algorithms facilitates rapid implementation of theories and algorithms by domain developers, and provides a friendly open source community environment for external developers to contribute submodules.

2. Full-flow realization of artificial intelligence algorithm combining traditional molecular simulation with MindSpore. In MindSpore, developers can easily apply AI methods to molecular simulations. SPONGE will be further integrated with MindSpore to form a new generation of end-to-end differentiable molecular simulation software, realizing the natural fusion of artificial intelligence and molecular simulation.

Short-term outlook: In the subsequent version update, the MetaITS module and finite element calculation module that have been verified theoretically will be added one after another. These modules will help SPONGE to better engage in phase transition and metal surface-related simulations. At the same time, MindSpore SPONGE modules gradually support automatic differentiation and automatic parallelism, providing more friendly support for cohesive machine learning schemes.

Long-term perspective: Expand SPONGE's various feature modules so that it can describe most microscopic systems with high computational and sampling efficiency. For specific industrial needs, such as drug screening or crystal form prediction, SPONGE will be used to derive complete flow-based computing solutions that can meet the needs of large-scale parallel computing. Under MindSpore framework, SPONGE has meta-optimization function to achieve more accurate and faster force field fitting.

MindQuantum is a quantum machine learning framework developed in conjunction with MindSpore and HiQ to support the training and reasoning of multiple quantum neural networks. Thanks to Huawei HiQ team's quantum computing simulator and MindSpore's high-performance automatic differentiation capability, MindQuantum can efficiently handle quantum machine learning, quantum chemistry simulation and quantum optimization, and its performance reaches TOP1(Benchmark) in the industry, providing an efficient platform for researchers, teachers and students to quickly design and verify quantum machine learning algorithms.

MindQuantum vs TF Quantum/Paddle Quantum Performance Comparison

Multi-hop Knowledge Reasoning Question Answering (TPRR) TPRR is a general model for solving open-domain multi-hop problems proposed by Huawei Poisson Lab and Huawei MindSpore team. Compared with traditional Q & A, which only needs to retrieve answers from a single document, multi-hop knowledge reasoning Q & A needs to obtain final answers from multiple supporting documents and return the reasoning chain from question to answer. TPRR is based on MindSpore's mixed precision property and can efficiently complete the multi-hop question-answering reasoning process.

Full Path Modeling:

The TPRR model models conditional probabilities of all inference paths in each link of the multi-hop problem inference chain, and the model performs knowledge inference from a global perspective.

Dynamic sample selection:

TPRR model adopts dynamic sample modeling method, and improves the ability of multi-hop question answering through stronger contrast learning.

The algorithm flow chart is as follows:

The TPRR model topped the international authoritative multi-hop quiz list HotpotQA evaluation. The list chart is as follows:

MindConverter is a script migration tool designed to help algorithm engineers quickly migrate stock models developed based on the three-party framework to the MindSpore ecosystem. Based on the TensorFlow PB or ONNX model file provided by the user, the tool generates a readable MindSpore Python model definition script (.py) and corresponding model weights (.ckpt) by parsing the computational graph of the model.

One-click migration:

MindConverter CLI command can be used to migrate the model to MindSpore model definition script and corresponding weight file, saving model retraining and model definition script development time;

100% mobility:

When MindConverter has operator mapping across frames, the post-migration script can be used directly for inference to achieve 100% mobility.

List of supported models:

The tool currently supports typical models in the field of computer vision, natural language processing BERT pre-trained model scripts and weight transfer. For a detailed list of models, see README.

BERT model definition migration results display (partial code):

4. Reliability robustness evaluation tool helps OCR service achieve the first AI C4 robustness standard. MindSpore robustness test tool MindArmour provides efficient robustness evaluation scheme based on black and white box against samples (20 + method) and natural disturbance (10 + method) to help customers evaluate the robustness of models and identify vulnerable points of models.

OCR refers to the use of optical equipment to capture images and recognize text, reduce labor costs and quickly improve work efficiency; if an attacker makes imperceptible modifications to the text to be recognized, and the model cannot recognize or process it correctly, the accuracy of OCR service for Optical Character Recognition will decrease, and the user will not know the reason behind the problem. The evaluation team used MindArmour to evaluate the robustness of OCR service, and found that some models in OCR service had poor defense ability against natural disturbance and anti-sample, for example, the accuracy rate of text box detection model was less than 66% under check noise, PGD, PSO (particle swarm optimization) and other attack algorithms; Based on this, the model development team was guided to improve the robustness of the model and OCR service by means of anti-sample detection, data enhancement training and other technologies, so that the recognition accuracy of the model for malicious samples reached 95+%.

Thank you for your reading. The above is the content of "What are the new features of MindSpore?" After studying this article, I believe everyone has a deeper understanding of what new features MindSpore has. The specific use situation still needs to be verified by practice. Here is, Xiaobian will push more articles related to knowledge points for everyone, welcome to pay attention!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.