Shulou(Shulou.com)06/02 Report--

This article introduces the relevant knowledge of "how to connect Flutter and Web ecology". In the operation of actual cases, many people will encounter such a dilemma, so let the editor lead you to learn how to deal with these situations. I hope you can read it carefully and be able to achieve something!

First of all, let's discuss why we want to connect Flutter to Web ecology.

Flutter is now a hot cross-platform technology, which can run a set of code on Android, iOS, PC, IoT and browsers. It is considered to be the next generation of cross-platform technology. Compared with Weex and React Native can solve the problem of multi-platform consistency, the native rendering performance is similar, the upper layer is not as thick as JS encapsulation layer, the overall performance will be slightly better.

But the first question that most people who are eager to learn Flutter ask is: why does Flutter use Dart? A new language means new learning costs. Doesn't JS smell good? doesn't JS smell good? and TypeScript! In fact, Flutter abandons not only JS, but also HTML and CSS, and designs a better decoupled Widget system. Flutter abandons the whole Web and strives to create a new ecology, but this ecology cannot reuse the code and solutions of Web ecology. In particular, all previous cross-platform solutions such as Hybrid, React Native and Weex are integrated with Web ecology, which makes Flutter seem a little out of place and deters most front-end developers.

Here's what I've sorted out, the cost of using Flutter for front-end developers:

Because the development model of Flutter is similar to the front-end framework (that is, copied React), the cost of learning the framework is not high, slightly higher is the cost of learning the Dart language, and you have to learn how to assemble UI with Widget. Although many layout Widget designs are very similar to CSS, the flexibility is still much worse. If you want to use it in a real project, you need to transform the entire tool chain to develop from a "Native First" perspective. The link between developing Flutter and developing native applications is quite similar, which is quite different from developing front-end pages. The highest ecological cost is the front-end ecological accumulation, whether the code or technical solutions are difficult to reuse, which is the most painful point, ecology is also the weakest link of Flutter.

No matter why Flutter abandons Web ecology for the sake of advanced technical concepts or commercial selfishness, the real problem is that the largest group of UI developers is the front end, and the richest ecology is Web ecology. I think Web technology is also the most efficient way to develop UI. If you can use Web technology stack to develop in the upper layer and use Flutter to achieve cross-platform rendering at the bottom, can't you take into account development efficiency, performance and cross-platform consistency? It can also reuse a large number of technology accumulation of Web technology stack.

Perhaps these reasons are not sufficient, let's continue the analysis according to this hypothesis for a while, and then re-discuss whether it should be docked or not.

The docking of Flutter and Web ecology involves two aspects:

From Web to Flutter. Is to use the Web technology stack to develop, and then connect to the Flutter to achieve cross-platform rendering. For Web, it is to solve the problem of performance and cross-platform consistency, and for Flutter, it is to solve the problem of ecological reuse.

From Flutter to Web. This is the official implementation of Web support for Flutter, which compiles the App that has been developed with Dart into HTML/JS/CSS and runs it on the browser, which can be used for downgrading and outbound scenarios.

How to realize from Web to Flutter?

First take a look at the architecture diagram of Flutter to see where you can start.

Flutter can be divided into two parts: Framework and Engine, and the Engine part is relatively stable at the bottom, so it is best not to move. What needs to be changed is the Framework implemented with Dart. If you want to dock with Web ecology, the JS engine must be introduced, as to whether to retain Dart VM remains to be discussed. At the top of the figure, the front end of the two UI libraries, Material and Cupertino, is not needed, and the front end has its own. The key is whether to replace Widget with HTML/CSS to write UI, or to keep Widget but change the language to JS. Different solutions are also different.

There are many solutions that can be interfaced, and there are many attempts in the industry. I have summarized the following three ways:

TS Modification: replace Dart VM with JS engine and re-implement Flutter Framework with JS/TS (or compile directly with dart2js).

JS docking: introduce JS engine while retaining Dart VM, and use front-end framework to dock Flutter Framework.

C++ magic change: replace Dart VM with JS engine and re-implement Flutter Framework with C++.

TS magic modification

TS magic modification is to completely abandon Dart VM, using TypeScript to re-implement the Flutter Framework written in Dart.

Why TS and not JS? This is because TS is a big hit, and it is backward compatible with JS, and now almost all fashionable frameworks are rewritten in TS.

The starting point of this scheme is "if you can change the Dart of Flutter into JS", the easiest way to think of is to translate Dart into TS, or to compile the code into js directly with dart2js, but the compiled code contains a lot of dart:ui and other library encapsulation, the generated package is also quite large, and it is difficult to customize the interface that needs to be exported. It is better to rewrite it with TS, the tool chain is more familiar, and you can add some customization.

In theory, most of the functions of Flutter are still supported after translation, and various npm packages can be reused and dynamically, but without the ability of AOT, the execution performance of JS language should be inferior to that of Dart. And the layout operations of all nodes take place in JS, so the bottom layer only needs to provide basic graphics capabilities, just like writing a UI framework based on Canvas API, and its performance may not be as high as that of the existing front-end frameworks.

In addition, the biggest problem is how to be consistent with the official Flutter. If it is translated from v1.13 now, will it be updated synchronously after the official upgrade to v1.15? This process is unskilled, requires continuous investment, and is disgusting to do.

You also need to consider whether the upper layer should write UI in Widget or HTML+CSS, which is familiar to the front end. If you still use Widget, most of the front-end components will not be used, and UI will have to be rewritten. Anyway, if you want to rewrite, the cost has not come down, so use Dart to rewrite it. Using the official original Flutter directly also avoids having to translate the Dart code every time you update. Therefore, since you have chosen to dock the front-end ecology, it is necessary to dock CSS, otherwise it will not be of sufficient value. However, the docking of CSS and Widget is also a tedious process, and there is a problem of completeness.

JS docking

The way to translate the code is not elegant enough, so just keep the Dart and connect the JS/CSS to the Widget.

Of course, this method only uses Flutter as the underlying rendering engine, while the upper layer maintains the writing of the front-end framework, and only connects the rendering part to the Flutter. Many existing front-end frameworks abstract the underlying rendering capabilities, which can be connected to different rendering engines, such as Vue/Rax, which supports both browsers and Weex. In the same way, another Flutter can be supported.

This method is compatible with the front-end framework better, but the link is too long, the business code calls the front-end framework interface to render, and after a meal of operation, the rendering instruction is sent to Flutter Framework based on communication, which involves a cross-language conversion from JS to C++ and then to Dart, and then to the corresponding Widget tree after receiving the rendering instruction. The conversion from CSS to Widget is still very complicated. And Widget itself can have a status, itself is a responsive update, when the update will regenerate the widget and diff, if the front-end update UI, the front-end framework in the js diff once vdom, after passing to the Flutter again diff widget.

If you want to bypass Widget to directly interface with the Rendering layer in the figure, you can bypass widget diff, but you have to change the rendering link of Flutter Framework. Since you want to change Flutter Framework, why not just use TS to change it? you can also bypass the communication from JS to Dart, and return to the first solution.

To sum up, the advantages of this scheme are: easy to implement, can maximize the retention of front-end development experience, and its disadvantages are: long rendering link, high communication cost, responsive logic conflict, incomplete CSS to Widget and so on.

C++ magic change

If you want to kill Dart VM, you need to re-implement Framework developed with Dart in other languages. You can use JS/TS, of course, you can use C++. The hardest way is to use C++ to re-implement Flutter's Framework, and then connect to the JS engine to expose the C++ interface to the JS environment through binding. The upper application is still developed with JS.

After sinking the Framework layer to C++, you will not only have better performance, but also support more languages. Originally Flutter Framework is on Dart VM, must rely on Dart VM to run, so it has a strong dependence on Dart; after re-implementation with C++, JS engine is on the C++ version of Framework, the framework itself does not rely on JS engine, but also can interface with various other languages, such as docking JVM can support Java and Kotlin, docking back to Dart VM can continue to support Dart.

This solution can enhance performance and maintain consistency with Flutter, but the cost of transformation and maintenance is quite high. The development efficiency of C++ is certainly not as efficient as that of Dart. How to follow up after a rapid iteration of Flutter is a big problem. If the follow-up is not timely or the implementation is inconsistent, it is likely to split up. The conversion from CSS to Widget is also a problem that we have to face.

Comparison of several schemes

Draw the above schemes in the same picture like this:

The solid line part of the figure represents cross-language communication, too often will affect performance, and the dotted line part represents other docking possibilities.

Flutter Engine does not need to move from bottom to top, and this layer is the key to cross-platform. Framework is available in three languages, JS/TS, Dart and Clippers. C++ version has the best performance and Dart version has the lowest cost. Then you need to deal with the problems of HTML/CSS and Widget up, either directly with a front-end framework or in the C++ layer (otherwise there are too many binding interfaces to be exposed and communication is too frequent).

How to realize from Flutter to Web?

This function has been officially implemented. App developed with Dart can be compiled into Web App and run on the browser. The official documents are mainly about usage and API. Here, I will briefly analyze the internal specific implementation.

Realization principle

Combined with the architecture diagram of Flutter, it is necessary to transform the upper Framework to realize the Web to Flutter, and the bottom Engine to realize the Flutter to Web.

The core dependence of Framework on Engine is dart:ui, which is implemented in Engine, which abstracts the interface of drawing UI layer, and the bottom layer connects with the implementation of skia, revealing the interface of Dart language. From this point of view, the docking method is relatively simple:

Use dart2js to compile Framework into JS code.

Browser-based API reimplements dart:ui, that is, dart:web_ui.

There is no problem with compiling Dart to JS, performance may have a slight impact, the functionality can be completely retained, the key is the implementation of dart:web_ui. In native Engine, dart:ui relies on SkCanvas to draw through skia, which is a very low-level graphics interface, which only defines the underlying capabilities such as line drawing, polygon drawing, mapping and so on. It is very challenging to implement this set of interfaces with browser interfaces. As can be seen in the image above, Engine for Web is based on DOM and Canvas. The underlying layer defines two graphical interfaces, DomCanvas and BitmapCanvas, which will render the incoming layer tree into browser Element tree, but the nodes only contain styles such as position, transform, opacity, and so on. Only a small subset of CSS is used, and some more complex rendering is directly implemented in 2D.

Existing problems

I compiled a rather complex demo and tried it, and the performance was not ideal, the sliding was not smooth, and sometimes the pictures flashed. The generated js code has 1.1MB (after minify, not gzip), and the node level is relatively deep. I estimate that the front-end writing of this page will not exceed 300KB, and the number of nodes can be reduced by more than half.

In addition, take a look at the issue of the Flutter warehouse, filter out the platfrom-web related, you can see a large number of: invalid text editing, can not find the cursor, ListView can not scroll on ios, abnormal checkbox/button behavior, Android scrolling stutter picture flicker, font invalid, some models of video can not be played, text selected can not be copied, unable to debug …... It feels like flutter for web is in a quagmire, reminiscent of the front end's nightmare of dealing with all kinds of browser compatibility.

The core reasons for these performance and compatibility issues are that browsers do not expose enough underlying capabilities, and that browsers handle gestures, user input and ways and Flutter are very different.

The implementation of Flutter Engine requires the underlying graphical interface and system capabilities, although it provides a similar graphical interface, if all implemented in canvas, it is difficult to deal with accessibility, text selection, gestures, forms and other problems, there will be a lot of compatibility problems. Therefore, the Canvas + DOM hybrid method is used in the real solution, the encapsulation level is too high, and the rendering link is too long. It is as if after a fierce operation in Flutter Framework, the nodes are generated, the layout is calculated, and the drawing attributes are dealt with, only a canvas is drawn, and then it is handed over to the browser to generate Element again, calculate the layout again, draw again, and finally give it to the underlying graphics library to draw it.

For example, when scrolling a long page, only a CSS (overflow:scroll) in the browser can make the elements scrollable. The monitoring of gestures, the scrolling of the page and the scrolling animation are all native to the browser, without interacting with JS or even re-layout and paint, just compositing. As shown in the figure above, in Flutter, Animation and Gesture are implemented in Dart, and compiled by JS. The browser itself does not know whether this element can be rolled, but keeps sending touchmove events. JS calculates node offset based on event attributes, then calculates animation, then applies transform or new position to the node, and then the browser performs the complete rendering process again.

Optimization scheme

The problems of performance and compatibility still need to be solved. There are two official attempts to solve the long-term optimization solution of issue in the short term:

Use CSS Painting API to draw.

This is a new standard that is still in the state of proposal, and you can use JS to implement some drawing functions and customize the CSS properties.

It hasn't been implemented yet, so you need to wait for the browser to support CSS Houdini first.

Use the WebAssembly version of Skia to draw. Https://skia.org/user/modules/canvaskit

This gives full play to the performance advantages of wasm and maintains the consistency of skia functionality. However, at present, wasm may not have a performance advantage in the browser environment, so we will not discuss it here.

It has been partially implemented. Refer to the configuration here to enable the function: https://github.com/flutter/flutter/issues/41062#issuecomment-533952994

Both of these schemes want to make more use of the underlying capabilities of the browser. Only when the browser exposes more underlying capabilities can we better implement the Web Engine of Flutter. However, this will take a long time, and we will not be able to participate. If we want to use flutter for web at this stage, we still have to maintain the existing architecture and participate in solving the issue, giving priority to ensuring the function, and then optimizing the performance.

A more adaptable architecture

If it is idealized, can we make the ecological integration of Flutter and Web better from an architectural point of view?

Review the official architecture diagram at the beginning of the article, above is Framework (Dart), below is Engine (C++), syncopated at the Foundation level, the interaction between the two sides is geometric information. If this architecture is maintained, the segmentation level is divided higher, as shown in the following figure, which is divided into Widgets and Rendering layers, which is theoretically imperceptible to Flutter developers, because the upper development language and Widget interfaces are unchanged.

Syncopated in this layer, the interaction between Framework and Engine is no longer geometry but node information, the combination of Widget, setState responsive updates, Widget diff are still in Dart, after the deployment of RenderObject layout, drawing, cutting, animation are all in C++, not only have better performance, but also can have a better combination with Engine.

In other words, the original design of Engine is retained, and this part of the sinking is logically divided into Renderer, resulting in the following three-tier structure:

Each layer divided in this way has a clear positioning:

Framework: development framework. Provide developers with programmable API, achieve a responsive development model, and provide fine-grained Widget for developers to package and combine freely.

Renderer: rendering engine. Specializing in layout, drawing, animation, gesture processing, this part of the function is relatively independent, can be decoupled from the development framework, and does not have to be bound to a specific language.

Engine: graphics engine. Achieve a consistent graphical interface across platforms, synthesize the input layer and draw it to the screen, and handle the access and adaptation of the platform force.

This segmentation not only has performance advantages, but also makes the rendering engine get rid of the dependence on Dart, and can support multiple languages and development modes. You can use Dart to write code to Dart VM, JS to JS engine, and Java to JVM, but no matter how you write it, the underlying rendering ability is the same, a unified layout algorithm, animation and gesture processing behavior is also the same.

Under such a framework, it is easier to interface with Web ecology. Dart and Widget are unwanted at the front end, hoping to be replaced with JS and CSS, but want the underlying cross-platform consistent rendering engine, so just start docking at the Renderer layer, bypass all the unwanted and retain all the desired ones.

It's also a little easier to implement Flutter for Web. When interfacing at the Engine layer, it always suffers from the lack of underlying capabilities revealed by the browser, which is easier if it is based on Renderer. The JS/CSS/DOM/Canvas-based capability encapsulates a set of Rendering interfaces that can be called by Widget, which can make the rendering link shorter, but still have to deal with the compatibility between Widget and DOM/CSS.

Discuss it again: why docking?

The technical analysis has been completed, and it takes a lot of cost to complete the docking of Flutter ecology and Web ecology, so before you really decide to do it, you should discuss why you want to do the docking. Do you want to dock or not?

First of all, the official positioning of Flutter by Google is a problem. At the beginning of Flutter design, it does not consider the ecology of Web, or even deliberately avoids it, and advocates a development way that is closer to the original. The reason why I said not to dock at the beginning is also very simple: the two technological design concepts are different, not in the same direction, the ecology is impassable, the technical solution is impassable, and forced integration is likely to deprive each other of their advantages. But there are a lot of teams in the industry trying to show that the demand exists, and if Google resists this direction, it will be difficult to do so. However, now that officials have supported Flutter for Web, they have taken a step towards Web ecology, and it is also possible to further integrate with Web in the future.

In addition, there is the problem of cross-platform technology itself. After 20 or 30 years of development, the browser has been a very powerful cross-platform product, almost synonymous with Web, which is unbeatable. But it is also bloated, with a lot of historical baggage, poor performance and experience, and poor integration with Native, especially on mobile and IoT platforms. Although the hardware performance is constantly improving, it is shared by all software, and the performance and experience of browsers will always be worse than that of Native, which is likely to be room for new business and new scenarios. If you take a look at the new business scenarios that have emerged in recent years, many of them have become popular by making use of the new capabilities provided by Native, such as AI/AR/ video / LVB, etc. Are there any business models hatched because of the new Web API?

This is the end of the content of "how to connect Flutter and Web ecology". Thank you for your reading. If you want to know more about the industry, you can follow the website, the editor will output more high-quality practical articles for you!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.