Shulou(Shulou.com)05/31 Report--

This article shows you how to quickly build serverless network disk service, the content is concise and easy to understand, it will definitely brighten your eyes. I hope you can get something through the detailed introduction of this article.

Preface

Function computing supports the rapid construction of enterprise and developer software systems in a serverless architecture, with its fully managed event trigger mode, super-elastic scaling and low-cost billing method, compared with the traditional server architecture. Obvious advantages have been achieved in development performance and operation and maintenance mode. In addition, it also provides the trigger function between various services, as the adhesive of various products in the cloud, suitable for a variety of application scenarios.

Will lead you to use function calculation to quickly build a fully functional network disk service.

Network disk service introduction function

For a detailed introduction of the network disk service function, please refer to the article network disk service system and the corresponding project repo.

The focus of this article is not to introduce the features of the network disk, so please experience it by yourself through the network disk demo that has been built. I will not repeat it here.

Serverless architecture of network disk system

There are many functions of network disk service. Here we take the function of uploading files as an example to show the implementation architecture of the network disk.

The module of network disk service and its resource dependence

The module implementation of the network disk service is divided into three parts, as shown in the following figure:

Apis: provides the API interface function of the network disk, which is the core of the serverless architecture implementation. The FC+API Gateway architecture pattern is used to implement the principal function, as shown in the serverless architecture diagram above.

Site: provides the management console of the network disk instance, as well as the corresponding network disk promotion, usage documents and api documents.

UI: provides a client for users to use and maintain the network disk.

Here, the two sub-modules Site and UI are implemented in static website mode, which can be directly implemented using the website feature of OSS bucket. In order to simplify the whole deployment process and automate the resource orchestration process, the deployment methods of Site and UI are also integrated into FC for unified maintenance, as described in the detailed deployment process below.

The backend dependent resources of the network disk service feature are listed as follows:

Fun introduction

Fun, the abbreviation of have Fun with Serverless, is a tool for Serverless application development, which can help users define resources such as function calculation, API gateway, log service and so on. It is designed to be compatible with Aliyun ROS (although so far ROS does not support function calculation. However, fun as a subset of ROS is our goal.

Using fun can solve the problem of resource dependent maintenance of complex services, deploy and update quickly, and simplify resource scheduling, operation and maintenance. In view of the numerous resource dependence of the network disk service, using fun can solve the construction and maintenance process of various resources very conveniently and quickly. Next we will demonstrate how to deploy the network disk service system using fun.

Specific deployment process

The network disk service currently implements three kinds of identity login authentication functions: nail, Alipay and special account password (for testing).

Because nailing and Alipay login methods need to go through some application processes to obtain the corresponding appID and appSecret, here, in order to simplify the process, we only use special account passwords.

The specific steps for deployment are as follows:

Preparatory work:

Cloud account preparation

Code preparation

Fun download and configuration

Configure template.yml for fun

Deploy template.yml definition resources and obtain secondary domain names and other information

Update apis function

Deploy Site and create an groupID instance

Deploy UI and configure storage instance

Prepare the work, prepare the cloud account.

First, prepare a cloud account that can log in to Aliyun's official website, and log in to the console to obtain the UID (accountID), accessKeyID and accessKeySecret of this account. The resources used to deploy the demo are all in East China 2 (cn-shanghai). Other region resources can also be used, but you must ensure that all services are in the same region.

Then activate the following services:

Code preparation

Function calculation supports multiple code upload methods. Here, it is packaged into a zip file and uploaded to a bucket of the cloud account (you can create it in East China 2 by yourself). The reference code is as follows:

Apis.zip

Note: apis is implemented using nodejs, and you need to execute the tnpm I command to install the project dependency before packaging.

The directory structure after decompression is as follows:

|-- controllers # Control layer, main logic |-- routes # api routing |-- services # Public method, including calling methods of various cloud products |-- utils |-- models |-- node_modules # Project dependency |-- index.js # Service entry |-- conf.js # config configuration |-- package.json

Site.zip

| |-- deploy.py # deployment function |-- site # site code directory |-- 404.html |-- index.html |-- docs |-- static |

After decompression, the file structure is as follows:

Ui.zip

|-- deploy.py # deployment function |-- ui # ui code directory |-- 404.html |-- index.html |-- static

After decompression, the file structure is as follows:

Fun download and configuration

If your system is already configured with node 8 or above, you can directly execute the following command to install fun

Npm install @ alicloud/fun-g

If you do not want to install the node runtime environment, you can download fun's binary execution directly and refer to the release binary of various environments fun.

Then set accountID, accessKeyID, accessKeySecret and region. Use the parameters of the applied cloud account in the preparation step. In this example, select cn-shanghai:

# if it is binary, execute. / ${binary_name} configfun configtemplate.yml configuration

Template.yml template configuration is the most important part of this deployment, because the network disk service depends on a large number of resource items (mainly API gateway resources and table storage resources). It is recommended to download the reference template and modify it directly. The content to be modified has been marked with "TODO".

The syntax specification of template.yml refers to the fun specification documentation.

This section details the relevant parts of the configuration:

ROSTemplateFormatVersion: '2015-09-01'Transform:' Aliyun::Serverless-2018-04-03'Resources: apsaradrivefc: Type: 'Aliyun::Serverless::Service' Properties: Description: apsaradrive Policies:-AliyunOTSFullAccess-AliyunSTSAssumeRoleAccess-AliyunOTSFullAccess-AliyunLogFullAccess-AliyunFCInvocationAccess LogConfig: Project: log-yunpan # TODO replace sls project name Logstore: fclog apis: Type: 'Aliyun::Serverless::Function' Properties: Description: apis Runtime: nodejs8 Timeout: 30 MemorySize: 128 Handler: index.handler CodeUri: oss://apsaradrive-ui/download/apis.zip # TODO replaced the code location EnvironmentVariables: # TODO replace all the values which has' TODO' tag ENV_CONFIG: '...' DeploySite: # function name which deploys site Type: 'Aliyun::Serverless::Function' Properties: Handler:' deploy.my_handler' Runtime: 'python2.7' Description:' function to deploy site' MemorySize: 128 Timeout: 30 CodeUri: 'oss://apsaradrive-ui/download/site.zip' # TODO replaced By the code location DeployUI: # function name witch deploys ui Type: 'Aliyun::Serverless::Function' Properties: Handler:' deploy.my_handler' Runtime: 'python2.7' Description:' function to deploy ui' MemorySize: 128Timeout: 30 CodeUri: 'oss://apsaradrive-ui/download/ui.zip '# TODO replaced by the code location apsaradriveapis: Type:' Aliyun::Serverless::Api'... Ts-yunpan: # TODO replace ots instance name Type: 'Aliyun::Serverless::TableStore' Properties: ClusterType: HYBRID Description: for apsaradrive. Log-yunpan: # TODO replace sls project name Type: 'Aliyun::Serverless::Log' Properties: Description: log fclog: Type:' Aliyun::Serverless::Log::Logstore' Properties: TTL: 10 ShardCount: 1

The yaml file defined above mainly does the following things:

Create a Service named 'apsaradrivefc', and create a Service Role with AliyunOTSFullAccess, AliyunSTSAssumeRoleAccess, AliyunOTSFullAccess, AliyunLogFullAccess, and AliyunFCInvocationAccess permissions for the Service. Then create three Function under this Service, named 'apis',' DeploySite' and 'DeployUI'.

Create a LOG Project named 'log-yunpan', and then create a Logstore named' fclog' under this LOG Project as the log collection store performed by Function under the Service apsaradrivefc.

Create an OTS Instance named 'ts-yunpan', and generate various Table on this side.

Create an API gateway packet, apsaradriveapis, and the various API definitions under it.

What needs to be modified:

OTS Instance name: there are three areas that need to be modified globally. Please unify it, because OTS Instance is globally unique in region scope. Try to keep naming uniqueness. Note: OTS Instance naming convention.

LOG Project name: there are two areas that need to be modified globally. Please unify it, because LOG Project is globally unique in region scope. Try to keep naming uniqueness. Note: LOG Project naming convention.

Three CodeUri, respectively, are the code locations of the three modules apis, Site and UI. Currently, the default configuration in template.yml is available and can not be modified.

The second-level domain name of the API gateway grouping needs to be configured internally in the EnvironmentVariables attribute of apis, because this parameter has not been generated in the current step and will not be modified here.

Deploy template.yml definition resources

Execute the following command directly:

# execute in the directory where template.yml resides # if it is binary, execute. / ${binary_name} configfun deploy

At this time, you can see the various resources created by fun for the network disk service on the console of OTS, SLS, RAM, API Gateway and FC, as shown below:

Update apis function

The secondary domain name of the corresponding API gateway group can be found in the deployment output log of fun in the previous step, as follows:

To modify the EnvironmentVariables attribute in the apis Function, you need to configure the second-level domain name of the API gateway grouping, as shown below:

Still execute the following command to update the environment variables for function

Fun deploy

At this point, the apis module is deployed.

Deploy Site and create an groupID instance

The implementation of the Site module is actually a static website. We can make use of the static website feature of OSS. The steps are as follows:

Modify endpoint (secondary domain name of apis) and ui_endpoint in static/global.js (if it is in East China 2, it is ${uid}-ui.oss-cn-shanghai.aliyuncs.com)

Create the bucket of ${uid}-site and set it to public-read ACL

Set the default home page and 404 page of the website for this bucket

Upload the entire code directory

Encapsulate all the steps according to the functional calculation programming model as follows and create it as Function: DeploySite. The encapsulated Function implementation is as follows:

Import oss2import loggingimport jsonimport shutilimport osimport sysreload (sys) sys.setdefaultencoding ('utf8') from oss2.models import BucketWebsitedef delete_file (filePath): if os.path.exists (filePath): for fileList in os.walk (filePath): for name in fileList [2]: os.remove (os.path.join (fileList [0], name)) shutil.rmtree (filePath) def my_handler (event, context): # event is json string Parse it evt = json.loads (event) logger = logging.getLogger () endpoint = 'oss-cn-shanghai-internal.aliyuncs.com' creds = context.credentials auth = oss2.StsAuth (creds.accessKeyId, creds.accessKeySecret, creds.securityToken) # create bucket for site named ${uid}-site, and set the ACL to public read bucket_name ='% s murite'% (evt ['uid']) bucket = oss2.Bucket (auth, endpoint) Bucket_name) try: bucket.create_bucket (permission=oss2.BUCKET_ACL_PUBLIC_READ) except oss2.exceptions.BucketAlreadyExists: logger.info ("Bucket% s is already exist"% bucket_name) # set static website bucket.put_bucket_website (BucketWebsite ('index.html') ) # modify the config for site code_dir ='/ tmp/site/' delete_file (code_dir) shutil.copytree ('/ code/site/', code_dir) data = upright 'with open (os.path.join (code_dir) 'static/global.js')) as f: for line in f.readlines (): if line.startswith (' var endpoint ='): line = u'var endpoint =\% s\' \ n'% (evt ['endpoint']) elif line.startswith (' var ui_endpoint ='): line = u'var ui_endpoint =\ 'http://%s.oss-cn-shanghai.aliyuncs.com\'; \ n'% (bucket_name) data + = line with open (os.path.join (code_dir,'/ tmp/site/static/global.js'), "w") as f: f.writelines (data) # put all the static code to the bucket for root,dirs,files in os.walk (code_dir): for filespath in files: full_file_name = os.path.join (root) Filespath) bucket.put_object_from_file (full_file_ name [len (code_dir):], full_file_name)

Therefore, only the Function of Invoke is needed to complete the deployment of the Site module. Next, we will demonstrate the process of Invoke.

Log in to the function calculation console, find the corresponding Function: DeploySite, modify the trigger event, and then click execute:

{"uid": "replace with you accountID", "endpoint": "replace with the SLD"}

Then open the Site page (if uid is apsaradrive, then the URL of Site is: http://apsaradrive-site.oss-cn-shanghai.aliyuncs.com), and after login (using the default username: admin, password: 1234 login), create a group instance.

Deploy UI and configure storage instance

The process of deploying UI is similar to Site. You can also find the DeployUI on the console and modify the trigger event. Note that the event format is defined as follows:

{"uid": "replace with you accountID", "group_id": "replace with the group_id you just create in the Site web"endpoint": "replace with the SLD"}

After clicking execute, you can enter the Site page through the Group entry of the UI page:

Configure the relevant Storage on the UI page:

Next, you can perform various file operations on your network disk.

The above content is how to quickly build serverless network disk service, have you learned the knowledge or skills? If you want to learn more skills or enrich your knowledge reserve, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.