Shulou(Shulou.com)06/02 Report--

Press conference portal

Product details

Cloud native technology, pay attention to user experience, improve R & D efficiency

Environment building is an important part of the algorithm research and development process, in addition to hardware selection, software environment installation and configuration, subsequent upgrades often take a lot of time. With the help of native cloud technologies such as Aliyun ECS,Docker and Kubernetes, DSW can help users build the environment in 3 minutes. Users can choose all the resource specifications provided by Aliyun ECS, including CPU and heterogeneous computing GPU, according to the needs and cost of the algorithm. Support not only prepaid, but also post-payment. DSW also configures more than 10 typical software environment configurations for different AI scenarios, including different versions of mainstream training frameworks such as Tensorflow and PyTorch for users to choose from. As a highly open development environment, DSW opens sudo permissions to users and supports the installation of any third-party library.

In order to meet different levels and development habits of algorithm users, combined with visualization, interactive programming and command line input, DSW provides three kinds of programming entry: WebIde is suitable for projects with high engineering requirements; JupyterLab is suitable for fast POC testing; Terminal entry can be used for fast execution of Shell commands, running procedures and simple editing.

DSW has also developed and pre-installed a variety of JupyterLab and WebIDE plug-ins, such as the visualization tool Tensorboard, which is widely loved by deep learning developers, users can open it through Launcher,Commands in DSW, and you can even use the% tensorboard magic command to open it directly in Notebook and other ways to use Tensorboard. You can not only support local files, but also open the training log stored in OSS,ODPS. In view of the fact that the algorithm students often use Python, a Python plug-in is installed in the WebIDE of DSW, which can be debugged online in the browser directly and the tracking program runs step by step. Users can also install any plug-ins they need on their own.

DSW supports a variety of data sources to read and write, including NAS, OSS, cloud disk and MaxCompute, especially the built-in dswmagic magic command allows users to use SQL statements to read and write MaxCompute table data in ipynb files, preset SQL editor supports syntax highlighting, intelligent prompts, automatic completion and other functions, but also supports running Sql scripts with variable replacement function. Query results are automatically displayed in the friendliest graphics.

In order to save resource costs, users can take advantage of the no charge feature of downtime, shut down and save the environment when the instance is not in use, and quickly restore it when needed. In addition, DSW supports user-defined image installation and can create instances based on previously saved or customized environments.

DSW has supported more than 100 teams of Tianchi competitions on the public cloud, and has also undertaken competitions within Ali Group. After repeated training, it has been proved that it is not only suitable for individual and team research and development, but also supports large-scale algorithm competitions and education and training.

Security and stability are issues of concern to users. The computing, storage and network resources used by DSW are purchased by users' own accounts and deployed to their own vpc, which makes it easy to communicate with other users' data. It is completely isolated between users and has very good security. Based on Aliyun ECS and CCS, the stability is guaranteed.

Integrate the capability components of PAI to speed up the landing of business

As a member of the PAI family, DSW not only completes the stand-alone development training function, but also has some basic capabilities of PAI built in. For example, users can directly use the PAI visual algorithm package EasyVision for image classification training evaluation and prediction; automatically adjust the parameter AutoML to automatically tune the algorithm hyperparameters; and users can even unconsciously enjoy the operator optimization in the training process provided by the PAI compilation optimization algorithm component TAO. Finally, DSW also provides CommonIO components for algorithms to read MaxCompute table data directly, and supports standard interfaces such as TableRecordDataSet and TableReader,TableWriter, which makes it convenient for training programs to be submitted directly to the distributed training cluster of PAI.

It can be predicted that in the near future, more PAI algorithm packages will be built into the DSW base image. DSW also relies on PAI SDK to develop key AI production processes such as data reading, processing, model training, model management and online services, providing users with one-stop services such as Pipeline building, scheduling and management.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.