Shulou(Shulou.com)06/03 Report--

Preface

First of all, we introduce several important concepts that appear in this article:

Function computing (Function Compute): function computing is an event-driven service. Through function calculation, users do not need to manage the operation of the server, but only need to write code and upload. The function calculation prepares the computing resources and runs the user code in an elastic way, while the user only needs to pay according to the resources consumed by the actual code. Function to calculate more information.

Fun: Fun is a tool for supporting Serverless application deployment, which can help you easily manage resources such as function computing, API gateways, log services, and so on. It assists you in developing, building, and deploying operations through a resource configuration file (template.yml). More documentation references for Fun.

NAS: Ali Cloud File Storage NAS is a distributed file system with shared access, flexible extension, high reliability and high performance. In the scenario of function calculation, because of the limitation of code package, NAS can be used to store some files that do not change frequently, such as data model, static resources, and so on. Reference.

ROS: Ali Cloud Resource orchestration Service (ROS) helps you simplify the management of cloud computing resources. You can follow the template specification defined by ROS to define the collection of required cloud computing resources and their dependencies in the template. ROS automatically completes the creation and configuration of all resources to achieve automated deployment and operation and maintenance. Refer to more documentation.

Note: the techniques described in this article require a Fun version greater than or equal to 3.4.0.

Basically all faas platforms add code package restrictions in order to optimize the cold start of the function. Ali Cloud function Computing (FC) is no exception. FC requires that the size of the compressed code package does not exceed 50MB. Because the user's function code may require a large number of dependent libraries, the code package can easily reach the threshold set by the function calculation.

Traditional solution

Based on the rigid requirements of the platform, there are still many ways to solve the problem that the size of the uploaded code package for function calculation is limited. Here are three common solutions. Remarks: including not limited to the following options.

1. Submit a work order

Submit a function to calculate the ticket, and the backend staff will open the upload limit whitelist for your account. However, there are still code package restrictions on submitting work orders. In the time of deployment, the upload time of the code package is long, and the cold start time of the function is increased, which affects the performance of the function. PS: cold start can be completely removed by using reservation mode, which is beyond the scope of this article and will not be discussed here.

two。 Using OSS to realize

To classify the code package, except for the project code and a small number of dependent libraries that can be uploaded to the function calculation when the function is created, the user uploads part of the dependent library to OSS in advance and starts to load the dependency from the OSS when the function is triggered. The loading operation of this kind of dependency can define the cold start of the application layer. When the loading dependency ends, the cold start of the application layer ends and the processing logic of the function begins to execute. The cost of cold startup in the application layer often leads to burr, which affects the performance of the function. You can put the code logic that loads the package from OSS in the initializer function. It can not only solve the limitation of function calculation on the upload code packet, but also does not affect the performance of the function. But additional actions are required by the user. Please refer to API initializer to solve the problem that the size of uploaded code package is limited by function calculation.

3. Based on file system Nas

We store some large and immutable resources through NAS. This makes it applicable even in scenarios where there is a large dependency. NAS not only helps function calculation to solve the problem of large dependency, but also increases the difficulty of function calculation because of its own complex configuration. How to manage Nas resources, upload local resources to Nas and Nas-related configuration of service level refer to the correct posture calculated by the development function-- use Fun NAS to manage NAS resources.

The purpose of this article is to introduce the fourth way-- to use the Fun tool to solve the limitation of function calculation on uploading code packets, so that the deployment steps are simple and straightforward, and you don't need to care about additional configuration. At the same time, it also shows the smooth experience of Fun tools on big dependency scenarios, Let's go!

Note: runtime that has been supported by fc is nodejs6,nodejs8,nodejs10,python2.7,python3,java8,php7.2,dotnetcore2.1,custom. Currently, based on all runtime supported by fc, it is supported except php7.2 and dotnetcore2.1 in large dependency scenarios.

Environmental preparation

The Fun installation tutorial can download the binary version of Fun directly here, and you can use it after decompression. Execute fun-- version to check that Fun is installed successfully.

$fun-version3.4.0

What are the criteria for determining the big dependence of Fun?

Fun install is a subcommand of the fun tool, which is used to install pip,apt dependencies, etc., and provides two forms: command line interface and Funfile description file. For dependencies on fun install installations, large dependencies are automatically handled when fun deploy is deployed.

When Fun detects that the packaged code is compressed and exceeds the limit (50m), large dependencies and code are separated according to the corresponding runtime. Fun divides large dependency directories into system dependencies and language dependencies. The local path that the system depends on is .fun / root, and the language dependency is obtained according to the function runtime. The mapping of the large dependency directory corresponding to each runtime is as follows:

Language (runtime) highly dependent directory (directory) nodejs6node_modulesnodejs8node_modulesnodejs10node_modulespython2.7.fun/pythonpython3.fun/pythonjava8.fun/build/artifactscustom/

The custom execution environment custom large dependency directory is /, which can be understood as a collection of other runtime large dependency directories. For example, the function runtime is custom. If node_modules or .fun / python exists in the directory, Fun will consider them as big dependencies during the deployment wizard and will deal with them respectively.

Fun deploy support for large dependencies

Fun, the command-line tool for function evaluation, now natively supports this large dependency deployment without any additional operations. Just execute the fun deploy:

$fun deploy

The overall process is shown in the following figure:

Cdn.nlark.com/yuque/0/2020/png/502931/1579052914872-bf485f36-5771-4075-a764-5d8497624536.png ">

Fun deploy automatically completes the deployment of dependencies. When it is detected that the packaged function directory exceeds the limit of the platform, it will go to the configuration wizard to help users configure automatically. The above figure can be understood as: Fun uses the built-in NAS (Ali Cloud File Storage) solution to help users create, configure, and upload NAS to NAS. At run time, function calculations can be automatically read from NAS to functional dependencies.

Big dependency wizard

The screenshot of the large dependency wizard is as follows:

When Fun deploys the current function, it detects that the compressed (.zip) dependency exceeds 50m and prompts the configuration wizard (subsequent log omits.). Just enter enter or yes. Finally, Fun will automatically complete the configuration and successfully deploy the resources to the function calculation.

Experience upgrade

After the completion of the fun deploy big dependency wizard, the function will be deployed to function computation and provide services to the outside world. At this time, the big dependency is separated from the code through NAS, and the deployment speed will be very fast because there is no big dependency when packaging the local code directory when deploying again.

A practical case of using fun deploy for large dependency deployment is recommended here, which demonstrates the smooth experience of Fun tools in large dependency scenarios-- the rapid development of a distributed Puppeteer web screenshot service.

Fun package support for large dependencies

Fun Package is a function used to upload local resources such as code, compiled artifacts, static resources, and so on to OSS. Scenarios that use fun Package are usually scenarios where you want to deploy only through a template file. For example, after local development is complete, the local resources that the template depends on can be uploaded to OSS through fun package, so that it can be done simply through a template file in text format, whether deployed on other servers or using ROS. It is recommended to read the introduction to Fun Package features.

Non-large dependency scenario

The process is shown in the following figure:

Through Fun Package, you can upload the local resources contained in the template file to OSS with one click, and complete the packaging operation of the resources.

Template file difference

Comparing the packaged template file (template.packaged.yml) with the original file, you can see that the difference is only in scenarios where local resources are used, such as:

-CodeUri:'. /'+ CodeUri: 'oss://bucket/PackageDemo/function/39ce6e9109a23d313bc267b1a5211273'

Large dependency scenario

The process is shown in the following figure:

When you encounter the large size of the packaged function, you will also enter the large dependency wizard, Fun's built-in Ros solution to help you complete the automatic configuration. At the same time, Fun will separate the big dependencies and the code and upload them to OSS. The purpose of doing so will be mentioned below.

Template file difference

The template.packaged.yml template files generated after packaging in large dependency scenarios are different from those in non-large dependency scenarios. In addition to the above differences in CodeUri, many resource descriptions will be added as prerequisites for deployment using Ros, such as NasCpFunc. Only one is introduced here, and the rest will not be discussed in detail.

NasCpFunc: Type: 'Aliyun::Serverless::Function' Properties: Handler: index.cpFromOssToNasHandler Runtime: nodejs8 CodeUri:' oss://ellison-hongkong/9e610f5540e21ace83d5b742241da6aa' MemorySize: 3072 Timeout: 300

NasCpFunc is a built-in resource function for users in large dependency scenarios, which can be understood as an auxiliary function. When deployed in Ros mode, helper functions are executed automatically. It is used to download big dependencies (.zip) from OSS and extract them to Nas. This is why when the above fun pakckage is packaged, the big dependencies and the code should be separated and uploaded to OSS.

Fun Packge + Ros deployment practice

Serverless practice-use Rendertron to build Headless Chrome rendering solution using Rendertron + function calculation to quickly build a Headless Chrome rendering solution that can be directly used in production, in order to help websites better SEO. Based on the article, we will upgrade the one-click deployment experience in the article. You can refer to the steps in the above article, where dependent installation, project compilation, and so on do not require additional operations.

The code of Rendertron project is too dependent. Based on the support of Fun tools for large dependent projects, the original Fun deploy deployment is now transformed into Fun Packge + Ros deployment. Fun package automatically handles large dependencies uploaded to OSS,Ros deployment, decompresses large dependencies from OSS to Nas, and the resources described in the template are automatically created. Based on function calculation, the service architecture of the project is as follows:

1. Fun pakcage

Follow the steps in the RenderTron article and execute the fun package command before one-click deployment:

Fun package-oss-bucket aliyun-ellison

The-- oss-bucket name here is the Bucket name of the OSS that you have read and write access to. The full log is as follows:

2. Ros deployment

ROS transforms the template syntax of function evaluation into the syntax supported by ROS through Transform macros. This means that ROS also supports the syntax rules described in the Fun specification document. At the same time, the resources supported by ROS can also be declared in the Fun template file, such as RAM, function workflow, and so on.

In terms of experience, resources must be "clouded" because of ROS deployment. That is, there is no way to use local code resources directly. You must first upload the resource to oss through the fun package command.

It can be seen that we have completed this step, whether it is a large dependency scenario or a non-large dependency scenario. After the fun package is packaged, the subsequent deployment operations only need to be based entirely on the packaged template file (template.packaged.yml). No longer rely on local code and other resources, you can simplify the difficulty of deployment.

Finally, the resources are deployed through ROS. It is recommended to read the correct posture calculated by the development function-- use ROS for resource orchestration.

Fun deploy-use-ros-stack-name bucket-name

-- stack-name represents the environment to be deployed. Multiple development environments, such as test, staging, and prod, can be established based on the name.

3. Verification

It can be verified in the way mentioned in the above RenderTron article, and I won't repeat it here.

Combing FAQ with frequently asked questions

1. How does Fun deal with large dependencies that are referenced by two functions under the same service at the same time?

Scenarios where Fun supports large dependencies are functional-level, that is, when you package a function and find that the limit is exceeded, you will enter the wizard. When two functions are in the same runtime and codeUri,Fun, the second function is automatically configured at the end of the first wizard, ensuring that both functions are deployed successfully and available after deployment.

two。 After the automatic configuration of large dependencies, if I add new dependencies locally, will the dependencies be automatically updated to Nas when I deploy?

No, I won't. If you add a new dependency, such as a new dependency library in the node_modules directory, you need to execute fun nas sync in the directory where the template.yml template file is located to synchronize the local nas resources to the nas service. If you change the code, you just need to redeploy using fun deploy. Because large dependencies and code are separated through NAS, dependencies usually do not need to change frequently, so the frequency of calls is relatively low, while the deployment speed of fun deploy is very fast due to the absence of large dependencies.

3. Why is the directory that the java8 environment Fun supports big dependencies .fun / build/artifacts?

In many scenarios, compiled languages actually have a certain distance from the source code to the deliverables, such as java. After writing the java code, we should also consider how to compile and package. The responsibility of Fun build is to complete the build process from source code to delivery, and it is recommended to read "the correct posture of development function calculation-- using Fun Build to build functions." Fun build will copy the compiled and packaged delivery product to the .fun / build/artifacts directory, and when it detects that the code size exceeds the limit during deployment, it will naturally go to the .fun / build/artifacts to find the corresponding serviceName/functionName directory and upload all jar packages to Nas. Therefore, the support of java8 for Fun heavily dependent deployment is based on the Fun build scenario. Fun will integrate more solutions in the future, please look forward to it!

Summary

Through the built-in NAS (Ali Cloud File Storage) solution, Fun can help users create, configure and upload NAS to NAS with one click. At run time, function calculations can be automatically read from NAS to functional dependencies. At the same time, it also shows the smooth experience of Fun tools in large dependency scenarios.

If you encounter some problems in the process of using Fun, you can mention issue on github, or join our nail group 11721331 for feedback.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.