Shulou(Shulou.com)06/01 Report--

How to use genrule to change from makefile to bazel, this article introduces the corresponding analysis and answer in detail, hoping to help more partners who want to solve this problem to find a more simple and easy way.

0x01 background

How the existing makefile build project switches to the bazel build system.

Bazel provides a wealth of extension methods, and of course supports the transition from current makefile to bazel builds.

Explain its characteristics again:

Multilingual support, and support for extension to any language build.

Extended DSL is a starlyke language, a subset of Python, and is easy to use. This is not available in cmake and other build systems.

Caching is supported.

Support for distributed construction.

Support for minimized builds.

In a large system, a person may only be responsible for one of the small components, which may depend on other components. When this component needs to update the test, you only want to build the component that the component depends on and the component itself, and other unrelated components can not be built, so that you can use the build process more quickly. Bazel supports this approach.

The problem with 0x02 makefile

As mentioned above, makefile is good at dealing with small-scale software, and when the scale increases, makefile has the following problems.

Dependencies between components are difficult to manage. Makefile only supports putting all the source code in one directory, and then by the top-level.

This dependency can only be known by experienced people. Newcomers can only make trial and error when they want to minimize compilation, and then build it manually when they find that there are dependencies that have not been built.

Incremental builds are difficult to control.

This is related to point 1, because dependencies cannot be built automatically, so incremental builds are not available.

The build speed is not enough to do caching.

Makefile itself does not support caching, which requires extra work to implement caching on its own.

Problems that need to be solved in 0x03

Although bazel is compatible with the build of makefile, it is not directly supported. You need to have some understanding of how bazel is extended. There are also the following problems that need to be solved.

Input and output need to be defined in bazel rules

Output in makefile is generally undefined. For example, in the following makefile. There is no clearly defined output in this makefile.

All: gcc hello.c

Or something like this. In makefile, the rule is to execute a make.sh script.

Install: bash-x make.sh

Generally, makefile needs to be executed under the current directory.

The inputs and outputs defined in makefile are based on the current directory, while bazel is executed at the top level, and its working directory is the directory where the top-level WORKSPACE is located. This is unfriendly to makefile.

For these two problems, we did not find a good solution by looking at the issue and documentation of bazel. The respondent basically recommends you to use the extension of the corresponding language, which is unrealistic for many projects. The reasons are:

Extensions to the corresponding language are unlikely to be supported seamlessly.

You probably need to modify the built code yourself. For example, I came across a c language project, using cc_external_rule will not work.

The extra cost of study.

The extension of this corresponding language also takes time to learn, and it is also a hidden cost.

So multilingual build projects that are interested in bazel want to transition in such an ideal way.

The existing makefile project can make a painless transition.

New development or large-scale refactoring projects can be directly on bazel.

Through the summary, the following transition schemes are given, which can basically meet the needs of the next makefile project.

0x04 transition plan

Main points of the scheme:

Use the sandboxie environment variable of bazel to retain the environment variable of the build time of makefile.

These environment variables may be installation environment variables such as INSTALLROOT, or compiler link environment variables such as LD_LIBRARY_PATH.

Using genrule rules, you can quickly change to the directory where makefile is located and execute make.

For example projects, please see gitee. Https://gitee.com/ul1n/bazel-demo/tree/makefile contains two directories, src and lib. Each has its own makefile. Where src depends on lib.

The first point in the scenario can be easily passed through the bazel build option-- action_env. The second point of implementation is a bit tortuous, but it is not complicated. Here is a detailed explanation.

By familiarizing ourselves with bazel documentation, we know that genrule can implement any build process. Here we want the following genrule. Only basic attributes are defined, in which cmd can be changed from cd to BUILD_PATH,BUILD_PATH automatically, and different subdirectories will be changed to the relative path of the directory to which they belong. At the same time, a pseudo-target can be generated to meet the requirements of genrule.

Genrule (name= "hello", srcs= ["srcs"], cmd= "cd $(BUILD_PATH) & & make", outs= ["test"])

So how to automatically control the environment variables of BUILD_PATH? Bazel does not support passing environment variables directly to the context of genrule. In other words, environment variables in cmd need to be declared in advance. This can be done by looking at some official examples.

Note: the syntax of the bazel involved in the BUILD file can be familiar with with reference to the official documentation.

Define a def.bzl file. This bzl is an environment variable that can introduce a BUILD_PATH into the BUILD file that references it.

Def _ var_providing_rule_impl (ctx): build_path = ctx.build_file_path loc, _ = build_path.rsplit ('/', 2) return [platform_common.TemplateVariableInfo ({"BUILD_PATH": loc}),] var_providing_rule = rule (implementation = _ var_providing_rule_impl, attrs = {"var_value": attr.string ()})

Then add a BUILD file in the lib directory. This document is very general-purpose and can be placed in any makefile directory. Except for other dependencies that need to be added in the srcs property of genrule.

Load ("/ /: def.bzl", "var_providing_rule") var_providing_rule (name= "set_build_path", var_value= "") filegroup (name= "srcs", srcs=glob ([* * "])) genrule (name=" default ", srcs= [" srcs "], cmd=" cd $(BUILD_PATH) & make & & cd-& & touch $(RULEDIR) / out.txt ", visibility= [" / / visibility:public "] Outs= ["out.txt"], toolchains= [": set_build_path"])

Add the following BUILD file to the src directory, which is basically the same as in lib, except that srcs has more dependence on the rules in lib.

Load ("/ /: def.bzl", "var_providing_rule") var_providing_rule (name= "set_build_path", var_value= "") filegroup (name= "srcs", srcs=glob ([* * "])) genrule (name=" default ", srcs= [" srcs "," / / lib:default "], cmd=" cd $(BUILD_PATH) & make & & cd-& & touch $(RULEDIR) / out.txt " Outs= ["out.txt"], toolchains= [": set_build_path"])

Here is a description of the cmd parameter.

1 change to the directory where the BUILD file is located. 2 execute make. This can also be replaced with the required command, such as make install. 3 change to the previous directory, which is where the WORKSPACE file is located. 4 create a pseudo-output out.txt. RULEDIR is the directory of regular products agreed upon by bazel. Is a relative path, which is why you switched back to the directory in step 3. The purpose of using & & connection is to exit execution immediately in the event of failure.

Once the addition is complete, you can execute the build process with the following command.

Bazel build-sandbox_writable_path=$INSTALLROOT-action_env=C_INCLUDE_PATH=$INSTALLROOT/include-action_env=INSTALLROOT=$INSTALLROOT / / src:default

The-- sandbox_writable_path option is to open permissions and ensure the directory creation operations that need to be performed in the makefile. The other two environment variables are to ensure that the original makefile can be built successfully. When the execution is complete, the app executable is generated in the tmp/bin/ directory of the current directory. Generate dynamic link libraries under tmp/lib. In this way, as long as the appropriate environment variables are configured, the original build process can be executed perfectly.

0x05 summary

This implementation may not be the best, but it is by far the most straightforward. Of course, I also hope that students who know a better way to let me know. At present, there are still the following problems in this implementation.

The product is fake and not sound (perceptible) to bazel. Unable to cache the production of INSTALLROOT.

The producer is imperceptible, so if the INSTALLROOT is deleted, the bazel build may not do anything because it is unaware of the existence of the INSTALLROOT.

The answers to these two questions are as follows: if it is a large module that needs to be cached, you can produce the real production and copy it to RULEDIR at the same time. But the deletion of INSTALLROOT cannot be guaranteed by technical means.

This is the answer to the question about how to use genrule to change from makefile to bazel. I hope the above content can be of some help to you. If you still have a lot of doubts to be solved, you can follow the industry information channel to learn more about it.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.