Shulou(Shulou.com)06/01 Report--

This article will explain in detail how to use antd.sketchapp to generate training data in UI2Code. The content of the article is of high quality, so the editor shares it for you as a reference. I hope you will have a certain understanding of the relevant knowledge after reading this article.

Ant Design recently released antd.sketchapp:

Using the skpm construction tool, based on React Sketch.app and Ant Design, antd.sketchapp is implemented, which can easily convert the front-end code of Ant Design into sketch files. Similarly, we can implement a MD.sketchapp based on Material Design. The greater significance of antd.sketchapp is that it is a set of tools for designing system updates, which can quickly update the existing atoms of Sketch, so that the front-end components based on Ant Design can be modified based on the characteristics of our company, and continue to iterate, so as to ensure that business designers are up to date at specific times, while the front end is also up-to-date.

Code2UI is relatively simple to implement, but UI2Code is a bit difficult. The basic work includes MD's sketch plug-in, and designers can easily generate Material Design's UI components in sketch.

Airbnb is also trying to use UI2Code:

Airbnb launched sketching interfaces.

Https://airbnb.design/sketching-interfaces/

The method of CNN image classification is used.

With regard to the work and attempts related to UI2Code, there are two hot projects:

Pix2code

Https://github.com/tonybeltramelli/pix2code

Turning design mockups into code with deep learning

Https://github.com/emilwallner/Screenshot-to-code-in-Keras

The main deep learning models are CNN and LSTM. Because this article is based on Ant Design, there are many agreed rules, after all, it is a design language, unlike pure HTML or Bootstrap-based UI framework, it is too free, so we can simplify and try UI2Code based on CNN classification. The general idea of CNN, through the traditional algorithm, save the screenshot, and the corresponding pressing time. In this way, the IMG and the corresponding Label can be formed. Through the training of CNN convolution neural network, it can be used to input new IMG to predict the pressing time.

CNN needs a lot of training data, so we can try to use antd.sketchapp to generate a large number of paired data of Code and UI images. Code is abstracted into appropriate data forms, such as [Header, Content, Footer], and Header is expressed in the form of [Button, Text, Button], which is combined as a two-dimensional matrix, like this [[1,0,1], [1,1,1]]. Here we need to do a lot of experimental and abstract work on how to describe the code in a matrix way. The idea is similar to the previously introduced Snippet Grammar (parameter-based descriptive language) used in building generation, and strictly speaking belongs to domain-specific languages (DSL, domain-specific language).

DSL

A specialized computer language designed for a specific task.

A computer language specially designed to solve a particular type of task.

Martin Fowler believes that:

DSL is first and foremost a tool that helps users abstract parts from a system. So DSL is useful when you realize that you need a component, or when you already have a component and you want to simplify the way you operate it. DSL not only improves the readability of the code, but also allows developers to communicate better with domain experts.

A few examples of DSL

1 regular expression

Through some agreed symbols and combination rules, write regular expressions, through the regular expression interpretation engine to achieve string matching function.

2 JSON dynamically generate UI interface

Jasonette

Create your own native iOS app with nothing but JSON.Then send it over the Internet.

Dynamically generate the UI interface of IOS by configuring JSON.

Https://github.com/Jasonette/JASONETTE-iOS

Tmall Tangram

Android&iOS supports consistent rendering of one piece of data. Through the description of json data, we can combine the commonly used page structure and give the dynamic ability of the layout structure in the page.

Http://pingguohe.net/2017/12/07/Tangram-2.html

3 CSS in the front end of web

This has written CSS has an intuitive feeling, we can only according to the rules, write descriptive CSS language, to achieve the purpose of controlling the page style.

(4) facing the design problems

The design metadata of UI interface, such as length and width, color, text layout, internal and external margin, etc., are used as the input of the program with a kind of agreed and concise language rules (that is, DSL). The program automatically analyzes and processes, thus generating the available code of the real UI interface. The files generated by Sketch software are actually in JSON format and can be regarded as a kind of DSL in the design field.

Therefore, in the training data set, we need to design a set of DSL, use Sketch.app to generate a large number of paired data, and transform DSL into Label data used in CNN training.

On how to use antd.sketchapp in UI2Code to generate training data is shared here, I hope the above content can be of some help to you, can learn more knowledge. If you think the article is good, you can share it for more people to see.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.