Shulou(Shulou.com)06/02 Report--

This article mainly explains "how to understand the abstraction of construction". The content of the explanation is simple and clear, and it is easy to learn and understand. let's follow the editor's train of thought to study and learn how to understand the abstraction of construction.

Introduction 1: starting from the compilation of Java

Most programmers are from hello, world! Start your own life of copying and pasting. The same is true for programmers who are new to Java:

Javac HelloWorld.java

When we rely on other packages, we need to add classpath at compile time and run time to add dependencies. Therefore, the corresponding run command is as follows:

Java-classpath.: libs/joda-time-2.10.6.jar HelloWorld

In this way, we can get the expected results:

Hello, World Millisecond time: in.getMillis (): 1599284014762

If we need to package into jar, we need a more complicated process:

Jar cvfm hello.jar manifest.txt HelloWorld.class libs/*

Several key elements are involved in this process:

Tool chain. That is, java and javac, and the corresponding Runtime and so on.

The construction process. That is, I will first execute javac to compile, and then use the java command to start the application.

Rely on management. That is, the problem of getting the location of our joda-time-2.10.6.jar, as well as the process of adding it when packing.

Source code configuration. That is, class and java in the conversion process.

Input and output in the process.

Introduction 2: tasks and their input and output

For an artifact build, we tend to split it into a series of tasks, each with its own input and output. When the input changes, you need to change the corresponding output. Then we just need to choreograph the task:

Exports.build = series (clean, parallel (cssTranspile, series (jsTranspile, jsBundle)), parallel (cssMinify, jsMinify), publish)

What is shown above is which tasks can be performed in parallel and which tasks need to be executed sequentially-- which can also be thought of as task dependencies.

Of course, another task is the watch task, which is used only at development time, not at build time. The following is an example of file monitoring for the Gulp build tool in Node.js:

Function javascript (cb) {/ / body omitted cb ();} function scss (cb) {/ / body omitted cb ();} watch ('src/*.scss', scss); watch (' src/*.js', series (javascript))

Under the combination of the two, we will see the concept of incremental tasks: compiling only the modified parts to improve build efficiency. The one who has done a good job in this area is Gradle. Take a look at an official example, InputChanges:

Abstract class IncrementalReverseTask extends DefaultTask {@ Incremental @ InputDirectory abstract DirectoryProperty getInputDir () @ OutputDirectory abstract DirectoryProperty getOutputDir () @ TaskAction void execute (InputChanges inputChanges) {inputChanges.getFileChanges (inputDir). Each {change-> if (change.fileType = = FileType.DIRECTORY) return def targetFile = outputDir.file (change.normalizedPath). Get (). AsFile if (change.changeType = = ChangeType.REMOVED) {targetFile.delete ()} else {targetFile.text = change.file.text.reverse ()}

Similarly, it also requires us to monitor the corresponding inputs and outputs. With a slight difference, Gradle indexes the file, providing only the changed parts at a time, so that we can deal with them according to our actual needs.

Incremental build related resources:

Tup is a file-based build system for Linux, OSX, and Windows. It enters a list of changes to the file and a directed acyclic graph (DAG), and then processes the DAG to execute the appropriate commands needed to update the dependent file.

Ninja is a small build system focused on speed, similar to GNU Make.

SCons is an open source build system written in the Python language, similar to GNU Make.

Introduction 3: optional dependency management (hell)

I have written a series of articles about the management slots of dependency, such as: 11 strategies for managing dependency, dependency twins: low-cost dependency security solutions.

Purely from the construction of this matter, the management of dependency is dispensable. The main reason for this is that historically programming languages have not considered this problem. Therefore, building a system is a headache in the ancient CumberCraft + language. Of course, the new Golang also lacks good design.

Fortunately, for dependency management, the process is not complicated:

Package naming and version mechanism

Package management server

Dependency management at build and run time

Packet conflict processing

……

Abstraction of construction

Well, with the above series of basics, we can look at different construction systems, for the abstraction of the same concept, after integrating Bazel, Gradle, Cargo, NPM, etc., there is a basic level of abstraction:

Workspace (workspace). A workspace is an integration of one or more software packages that can share dependencies, output directory configurations, and so on. Typical examples are Gradle settings.gradle in Java, Cargo.toml of Cargo in Rust, and so on.

Warehouse. The repository can be mapped to the repository of Git, representing a piece of software that can be built independently.

Bag. The project structure of the smallest executable unit.

Package layout. Corresponding to different languages and building systems, it is used to define the location and structure of the code.

Products. That is, the product of the build, which may be a reusable software package or a runnable application.

Mission. Define the rules that are built and execute them.

FAQ

Why is there no project? In the field of business and technology, there is some ambiguity in our definition of project. In order to reduce ambiguity, we use workspace + warehouse to solve this problem. The workspace can be seen as a complete business project. On the other hand, the warehouse is a single code base, which may be a library or a complete project containing the library.

The best solution available is Bazel.

Work area

A workspace is an integration of one or more software packages that can share dependencies, output directory configurations, and so on. Typical examples are Gradle settings.gradle in Java, Cargo.toml of Cargo in Rust, and so on.

We can think of it as the final product, such as the executable file generated by the APK,Rust generated by Android. In the process, the shared packages are generated to support part of the project.

First look at the directory of CMakeLists.txt, where we define the project at the root node of the workspace, and add projectA and projectB.

Cmake_minimum_required (VERSION 3.2.2) project (globalProject) add_subdirectory (projectA) add_subdirectory (projectB)

To generate the final build artifact. Similarly, there is workspace in Rust:

[workspace] members = ["adder",]

Or the workspace in the front-end Yarn:

{"private": true, "workspaces": ["workspace-a", "workspace-b"]}

They all do the same thing.

Warehouse

The re-extraction of this concept comes from Bazel. A repository is a collection of packages that we can think of as the boundaries of a team and, in a sense, a code repository. For a large project, its code sources are varied, from other teams within the organization and from other teams outside the organization. Each independent part is a warehouse.

It is worth noting that from the end product point of view, the output of each team is a warehouse, but within the team, they are the workspace.

Let's take a look at a multi-project build example of Gradle (Android project):

. ├── README.md ├── library_a ├── app │ ├── build.gradle │ └── src ├── build.gradle ├── local.properties ├── settings.gradle └── third-partys ├──... ├── build.gradle └── settings.gradle

In terms of directory structure, this is a workspace, while in the workspace, it contains some three-party code repositories (third-partys), as well as its own library library_a and application app.

Therefore, the various projects of library_a and third-partys here are considered warehouses.

Bag

A package is a collection of code that can be large or small. The main reason is that when we build, we may output multiple packages from a warehouse (even the smallest Gradle project), such as src/main and src/test in the Java project.

So in build tools such as bazel, custom packages are supported:

Src/my/app/BUILD src/my/app/app.cc src/my/app/data/input.txt src/my/app/tests/BUILD src/my/app/tests/test.cc

For a package, we often need to define a series of related information, such as package name, dependency information, entry, and so on. For example, for Java build in Bazel:

Java_binary (name = "ProjectRunner", srcs = ["src/main/java/com/phodal/ProjectRunner.java"], main_class = "com.phodal.ProjectRunner", deps = [": greeter"],)

This has implemented the abstraction of information for different packages. By the way, take a look at an example of MANIFEST in the Java package:

Main-Class: HelloWorld Class-Path: libs/joda-time-2.10.6.jar

We can know the connection.

Package definition

During the packaging phase, we defined the package in a simple form-- because it wasn't that important and we didn't care. And when we decide to release this package to the Internet, we need to define the package. Some of the necessary information is:

Name

Version

Authors

License

Description

……

This information is used to display in the package management center and provide users with package-related information and so on. Different forms are used in different languages, Rust uses custom toml, and XML is used in repositories such as Maven:

.........

Similarly, similar fields are used in NPM's package.json: name, verison, and so on.

In these programming languages, this thing is designed to be too simple, such as requirements.txt used in Python's pip to manage dependencies and setup.py to configure when you want to release the package. Therefore, if your application is not released, there will be no package name.

Package layout

When the build tool is designed, it designs the default package hierarchy, which is called package layout (package layout). The build tool uses this layout to obtain the required input source and configuration information. It also contains some default configurations, such as src/main points to the directory of the source code, and src/test points to the test code (which will not be added to the artifact)

├── build.gradle └── src ├── main └── test

For users, they can also extend this layout to meet their needs, such as SourceSets in Gradle:

SourceSets {main {output.resourcesDir = file ('out/bin') java.outputDir = file (' out/bin')}}

The same is true for other languages. However, for some languages, there is not such a strong correlation, such as in Golang, there are no such strong constraints. It's just that it used to be the default value, but now it needs to be manually configured by the developer.

Product

The product is the final construction product. Similarly, there are different ways of naming in different languages. It is called artifacts in Gradle and targets in Rust. Products are mainly related to the circulation of various documents and their circulation rules.

To take a simple example, a jar file must contain a MANIFEST.MF to configure information about applications, extensions, and classloaders. The relevant documents will be organized in the form of META-INF.

Therefore, in the whole process of product creation, it is the process of copying the corresponding files, making corresponding transformations, such as java-> .class, then copying them to the corresponding directory, and finally packaging them together.

Task: rule engine + DSL

In many of the examples we have seen above, you create your own DSL and then use it to build. Only in this way can users get the greatest convenience. This is a rather complex process, which is equivalent to designing a platform-independent, language-independent DSL. And there are many ways to evolve:

Internal DSL abstracted using API. Such as in Webpack, Gulp and so on.

Self-made external DSL language. Such as Groovy and multilingual Bazel used by Gradle.

The rules engine itself is a set of DSL about tasks. Take a look at an example of Gradle:

Task copyReportsDirForArchiving2 (type: Copy) {from ("$buildDir") {include "reports/**"} into "$buildDir/toArchive"}

What it does is copy. The corresponding Gradle packaging example is also a fairly simple DSL abstraction:

Task packageDistribution (type: Zip) {archiveFileName = "my-distribution.zip" destinationDirectory = file ("$buildDir/dist") from "$buildDir/toArchive"}

Gradle uses external DSL. Let's take a look at the packaging example of Webpack:

Module.exports = {entry:'. / path/to/my/entry/file.js', output: {filename: 'my-first-webpack.bundle.js', path: path.resolve (_ _ dirname,' dist')}, module: {rules: [{test: /\. (js | jsx) $/, use: 'babel-loader'}]}, plugins: [new webpack.ProgressPlugin (), new HtmlWebpackPlugin ({template:' / src/index.html'})]}

Rules here is a simple rule engine (using regular expressions to match)

Each of the two modes has its own advantages and disadvantages. In complex scenarios, it is easier to use DSL + custom scripts.

PS: it seems that I am free. I should also write a rule engine.

Built extension

For mainstream build systems, they all support different forms of extension support:

External DSL extension

Plug-in interface programming

In-project programming language extension

Out-of-project programming language extension

Thank you for your reading. the above is the content of "how to understand the abstraction of construction". After the study of this article, I believe you have a deeper understanding of the problem of how to understand the abstraction of construction. the specific use of the situation also needs to be verified by practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.