Shulou(Shulou.com)06/02 Report--

background

With Flutter's continuous understanding of existing businesses, the focus of idle fish Serverless infrastructure also tends to dart ecology, first packaging dart containers onto servers, realizing the unification of dart programming languages, realizing programming framework integration (nexus, story) on unified containers, and back-end domain service integration. Based on the dart ecosystem, the front-end FaaS is actually not efficient in R & D delivery. The main problems faced in the R & D stage are:

Programming language is not unified: Although the programming language itself is not the biggest obstacle, it does add a lot of barriers to front-end developers, and more importantly, the ecology, environment and system behind the language is a high wall.

The project is separated from the complex environment behind it: there is a project on the end side and an independent project on the FaaS side. Behind them, there is their own set of tool chains for construction, debugging, integration/release; in addition, FaaS has its own supporting environment, runtime and framework as support. Developers facing such a complex FaaS R & D environment and dual R & D workflow are unable to achieve efficient delivery.

Programming Language Integration

Typescript, as a superset of Javascript, makes up for Javascript's static type checking and extends many OOP syntax features, making TS and dart very similar in syntax features, providing possibility and convenience for later conversion. To implement language-level conversion, there is always a small compiler behind it, but fortunately Typescript officially provides a syntax parser, through which we can easily get a reliable AST, so we only need to implement a dart generator. The generator can be roughly divided into four levels of work:

Basic syntax conversion

Native method difference conversion

Business Framework Bridging

Dependency library and header file bridge

Basic syntax conversion

This part is very easy to understand, is the most basic syntax level conversion, with the simplest example to look at.

Native method difference conversion

The two languages also have big differences in built-in native methods, for example: you can see that the following array instance methods are inconsistent in both language systems, and there are many native methods that are inconsistent in addition to array insertion. Of course, there is no need to be too scared by this unimaginable number, most of the time: 90% of the scenes will only use those 10% of the methods, and 10% of the conversions will cover 90% of the scenes.

// ts

list2.push(10)

// dart

list2.add(10)

To implement differential transformation of system methods, we must first identify which class the method comes from, for example list2.push(10) I can't just check push, because any class/object can implement a push method. We have to recognize that list2.push's push belongs to Array.push. Don't forget ts.TypeChecker, the largest type checker in the entire typescript compiler, which can help us solve this problem very well. The general idea is as follows:

Business Framework Bridging

After completing the above two capabilities conversion, it is not a big problem to conventionally write a section of logic for conversion; however, it is impossible to write naked business, business needs framework, and needs to communicate and deal with containers with the help of framework. A framework is needed for business abstraction to better organize and manage business logic. Let's look at an example:

DartMtopResult result = awaitHsfServices.request(moduleName, parameter);

The above code is used to make internal service requests on the dart side. From the code, we can obtain three parts of information:

There is a class for HsfServices

HsfServices has a request method that returns the result synchronously, taking two parameters

Finally returns the data structure of DartMtopResult

Let's look at the implementation of request and the declaration of DartMtopResult again:

// DartMtopResult.dart

classDartMtopResult implements xxxx {

T data;

bool success;

String errMsg;

String errCode;

// more code hidden

}

// HsfServices.dart

classHsfServices{

// more code hidden

staticFuture request(String moduletName, String parameter) async{

// more code hidden

}

// more code hidden

}

That's enough. For example, if I wanted to write ts code on the typescript side that could make requests with HsfServices.request without reporting errors, what would I do? Say something like this:

// HsfServices.d.ts

export declare classHsfServices{

static request(moduletName: string, parameter: string): Promise;

}

// DartMtopResult.d.ts

export declare classDartMtopResult {

data: T;

success: boolean;

errMsg: string;

errCode: string;

}

// business.ts

import{HsfServices} from"HsfServices.d.ts"

import{DartMtopResult} from"DartMtopResult.d.ts"

const result: DartMtopResult= awaitHsfServices.request('recycleGet', parameter);

It is very simple to write business logic normally and do not report errors. But you'd say this code doesn't work, no, but I don't need it to work, I just need to convert it to dart and run it in dart runtime. The general idea of bridging is as follows:

Dependency library and header file bridge

This part of the work is derived from the business framework bridge, let's use an example to illustrate the cause of the problem. TS source code is as follows:// business.tsimport {HsfServices} from "@ali/faas-hsf"import {DartMtopResult} from "@ali/faas-mtop-result"const result: DartMtopResult = await HsfServices.request ('recycleGet', parameter);dart source code as follows:// business.dartimport 'package:hsf_services/hsf_services.dart';import 'package:dart_mtop_result/dart_mtop_result.dart';DartMtopResult result = await HsfServices.request (moduleName, parameter); You can see that in addition to the above logic, the request part needs to be converted to dart, and the service reference header file needs to be bridged. The introduction of the header file is usually installed by the pub dependency package (pubspec.yaml), which means that the converter needs to obtain the pub package and the introduction header file on the dart side corresponding to @ali/faas-hsf. Our solution goes something like this: specify the mapping by placing the faas.yaml file in the @ali/faas-hsf module.

@ali/faas-hsf

|--lib/

|--faas.yaml

|--package.json

// faas.yaml

faas_pub:

#Dart side dependency packages mapped

hsf_services: ^1.1.7

#Mapping introduced header file

index: hsf_services.dart

In the process of R & D, the mapping relationship between them is extracted automatically through engineering scaffold: header file mapping and dependency package mapping. The header mapping is ultimately passed to the converter, while the dependency mapping is passed to the automatically maintained dart project (see below). The general idea is as shown in the following figure:

R & D engineering integration programming language integration is only the first step of the whole FaaS integration R & D, and only after the programming language is unified, the ecology (npm), tool chain (build) and engineering behind it can be integrated. Let's look at the status quo: developers are facing two separate projects, two different sets of environment and ecology. As I said at the beginning of this article, the programming language itself is not the biggest obstacle, but the environment and ecology behind the language are a high wall. After we unified the programming language, integration of R & D and engineering became feasible.

As shown in the figure above, the complexity of FaaS projects is that the whole project needs to run in a local container, because the container has to provide runtime, corresponding tool chain, framework dependencies, etc. for the project. Therefore, the local container itself is essential, and what we can do is to make the developer feel the existence of the container as much as possible; in addition, we must do a certain fusion of the logic of the two projects, which can be roughly abstracted into four parts: R & D code level fusion Code level fusion includes two parts: business logic fusion and business logic dependent programming framework fusion. Faas_src stores the ts version of business logic, package.json stores the programming framework on which business logic depends (we introduced the business framework bridge above, which is finally reflected in the end-side dependency package) └── Home│ └── index.ts├── package.json├── src│ ├── components│ └── pages│ └── Home│ ├── index.css│ ── index.js

This part is mainly responsible for compiling ts into dart in real time by docking converter and synchronizing it to FaaS project in black box. There are two parts in the real-time compilation process: one part is the logical compilation of pure ts into dart, and the other part is the synchronous installation of dependency packages, where faas_pub.yaml is extracted and generated by scaffolding by detecting faas dependency packages in end-side package.json, and does not require manual maintenance.

The build flow of the tandem debugging phase goes from saving each change to successfully launching a faas function request on the browser, roughly through these steps: monitoring the flow of changes, compiling code, and product deployment, concatenated by a unified end-side scaffold.

effect

After the integration of programming languages, we not only provide developers with a familiar technology stack, but also provide possibilities for later engineering integration; after engineering integration, we solve the engineering fragmentation for developers, solve the complex FaaS local operating environment behind it, and bring R & D experience basically consistent with the original R & D model.

There are still many ways to build subsequent integration, debugging, publishing, rollback, etc.; in addition, FaaS still runs on the backend, and finally communicates with the end-side through network protocols, so there must be two data structure declarations in the two codes, and two sets of packet unpacking logic; this provides a good room for the integration and automation of the data structure.

The idle fish technology team is not only the creator of Alibaba Group's idle trading community, but also the leader and innovator of new technologies for mobile and high-concurrency big data applications. We worked closely with the Google Flutter/Dart team, contributing several high-star projects and a lot of PR to the community. We are actively exploring innovative applications of deep learning and vision technologies in interactive, transactional, and community scenarios. The FaaS platform jointly built by Idle Fish Technology and the middleware team of the Group supports high-concurrency access scenarios of tens of millions of users every day.

Now! Client/server java/architecture/front-end/quality engineer for social + campus recruitment, base Hangzhou Alibaba Xixi Park, together to do creative space community products, do in-depth top-level open source projects, together to expand the technical boundary to achieve the ultimate!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.