Shulou(Shulou.com)05/31 Report--

This article mainly introduces the relevant knowledge of how to realize the video frame capture architecture method based on Serverless technology, the content is detailed and easy to understand, the operation is simple and fast, and it has a certain reference value. I believe you will gain something after reading this Serverless-based video frame capture architecture method. Let's take a look.

Overview of video frame capture requirements

The live broadcast industry is restricted by more and more laws, regulations and policies. under the constraints of general industry standards and operating procedures, every live broadcast platform has the obligation to deal with illegal live broadcast content, and improper interaction between anchors and viewers to take measures to contribute to the more standardized development of the live broadcast industry. How to monitor the illegal content in the live stream at the first time is a common challenge that the live broadcast platform needs to face. Video frame capture is a routine operation to meet the needs of content audit. Video screenshots can be captured at different frequencies according to the different risk levels of live video streaming, and the saved images can be uploaded to self-built or third-party content review platforms to identify porn-related, political, advertising and other scenes. In addition, some specific business requirements also need to be realized by video clipping, such as intelligent analysis of students' listening status by online classroom applications.

Analysis on the Architecture of Video frame capture Technology

The frame capture operation of the video stream can be realized by the FFmpeg command. FFmpeg's frame capture command is very easy to use. After capturing one picture at a time, you can upload the picture to the object storage OSS and send the corresponding frame capture information to the message queue Kafka. In this way, the audit service (which can be a third-party service or a self-built service) can obtain the frame capture information from Kafka and pull the corresponding image from OSS for processing. In this architecture, Kafka is introduced to ease the load of audit services during peak business periods through asynchronous processing mechanisms.

FFmpeg is easy to use, but it is a very computationally demanding operation for CPU. If the video stream is intercepted at a fixed frequency of 1 second, a 16-core ECS can probably undertake the task of intercepting 100 video streams at the same time. In order to ensure the stability of the service during the peak period, it is necessary to prepare a large number of ECS to deploy the video frame capture service. And most Internet applications have obvious peaks and troughs, such as the prime time every night is the peak of business, and the business volume after 24:00 will show a significant downward trend. Such business fluctuations bring great challenges to the overall resource planning. If the frame capture service is deployed according to a fixed ECS cluster size, there will be two obvious disadvantages:

In order to support the business peak, the cluster size must be evaluated according to the number of users in the peak period, which will cause huge waste in the business trough period.

In some scenarios, such as the star effect, there will be a sudden increase in business volume, and temporary expansion of the cluster may be needed. In this case, the expansion speed often lags behind the growth rate of business flow, resulting in the downgrading of some businesses.

In order to better improve the resource utilization, applications can also be deployed through elastic ECS instances in conjunction with containerization, so as to dynamically adapt the cluster scale to the change of real business volume. However, in the actual situation, the implementation of this kind of elastic scaling strategy is more complex, and the elastic scaling ability lags behind relatively, so the effect may not be too good. The fundamental reason is that in the traditional service architecture, an application is kept running for a long time after startup, and multiple business requirements will be processed concurrently during the run, no matter how the business volume changes. the computing power occupied by this application will not change essentially.

Is there a straightforward way to pull up the corresponding computing power to undertake the frame interception task after the live video stream is opened, and automatically release the computing power after the video stream is closed? This method does not require permanent application instances, can achieve the real allocation of computing resources on demand, and does not need to use additional means to dynamically adjust the cluster size of frame-intercepting services, which is the most ideal scheme.

As the representative of cloud native Serverless technology, Ali Cloud function Computing FC just realizes this idea.

Serverless Architecture based on function Computing FC

Function Computing FC is an event-driven fully managed computing service. Using function calculation, users do not need to purchase and manage infrastructure such as servers, but only need to write and upload code. Function computing automatically prepares computing resources, runs tasks flexibly and reliably, and provides functions such as log query, performance monitoring and alarm. With the help of function computing FC, you can quickly build any type of application and service, and only pay for the resources actually consumed by the task.

Function Computing FC provides an event-driven computing model, and the execution of functions is event-driven. The execution of a function can be triggered by the function consumer itself or by some other event source. You can create a trigger in a specified function that describes a set of rules that the event source triggers when an event satisfies these rules. For example, for HTTP triggers, a user's HTTP request can trigger a function; for OSS triggers, adding or modifying a file on OSS can trigger a function. In video clip scenarios, the function only needs to trigger a frame capture function actively through the business program before each LVB stream starts pushing. Therefore, the original frame capture architecture only needs to make a small adjustment, it can be migrated to the function computing platform to enjoy the value of Serverless.

Implementation of Video frame capture Technology based on Serverless Architecture

Now, we use a few simple steps to build a Serverless architecture based on function computing FC to meet the requirements of video frame capture. Function Computing FC provides a native running environment for Node.js, Python, PHP, Java and other languages, especially scripting languages such as Python, which can directly modify the scheduling code on the function computing platform, which is very easy to use, so the sample code in this paper is implemented through Python.

Of course, functional computing FC has no requirements for development languages, and any mainstream development language can support it very well. By calculating the Custom Runtime provided by FC, you can establish a custom running environment for the task language. A Custom Runtime is essentially a HTTP Server, and this HTTP Server takes over all requests from the function computing system, including from event calls or HTTP function calls.

Output video stream

We can develop through third-party video streaming services, but in order to make it easier to debug locally, we can realize the output of video streams through self-built RTMP services. The relatively simple way is to purchase an ECS and deploy Nginx to implement the RTMP service, which requires loading the nginx-rtmp-module module. We can find many related tutorials on the Internet. I will not repeat them in this article.

With the RTMP service, we can go to http://ffmpeg.org/ to download the compiled ffmpeg package and use the FFmpeg command to push the local video files to the RTMP service. For example, in the following ways:

Ffmpeg-re-I test.flv-vcodec copy-acodec aac-ar 44100-f flv rtmp://xxx.xxx.xxx.xxx:1935/stream/test

Next, we open the browser, enter the corresponding RTMP LVB address, and then pull up the corresponding player to watch the LVB. Rtmp://xxx.xxx.xxx.xxx:1935/stream/test

Install Funcraft

Funcraft is a tool that supports the deployment of Serverless applications, which can help users easily manage function computing, API gateways, log services and other resources. Funcraft through a resource configuration file template.yml, can achieve development, construction, deployment and other operations, we can use the function to calculate FC to achieve Serverless architecture in the process, greatly reduce the configuration and deployment workload.

There are three ways to install Funcraft, including npm package management installation, download binary installation, and Homebrew package manager installation. For an environment that does not have npm installed, the easiest way is to download the binary installation. We can download the Funcraft installation package for the corresponding platform through https://github.com/alibaba/funcraft/releases, and after decompression, we can use it. You can verify that the Funcraft package is installed successfully with the following command:

Fun-version

If you execute the command and return the version number corresponding to Funcraft, such as 3.6.20, then the installation is successful.

Before using fun for the first time, you need to execute the fun config command to initialize the configuration. For this operation, you need to provide general information such as Aliyun Account ID, Access Key Id, Secret Access Key, Default Region Name and other general information, which can be obtained from the upper right of the home page of the function calculation console. Other information, such as timeout, can be directly used by default values.

Configure OSS

Since the files saved after frame capture need to be uploaded to the object storage OSS for backup, we need to activate the Aliyun OSS service and create a corresponding Bucket. For specific operations, please refer to https://www.aliyun.com/product/oss.

Configure Log Service SLS

Log service SLS (Log Service) is an one-stop service for log data provided by Aliyun. You need to configure log items and log repositories in the corresponding service of the function through the log service, and grant the service permission to access the log service. The function log will be printed to the configured log repository, and all function logs under the same service will be printed to the same log repository. You can store the log executed by the function to the Ali Cloud log service, and then perform code debugging, fault analysis, data analysis and other operations according to the function log stored in the log service.

You can configure the log service SLS by referring to the creation of log project and log repository. To ensure that both log project and log repository have been created successfully, you need to use the information of log project and log repository when deploying the function.

Write a function

Now we use the simplest piece of Python code to experience how to intercept frames by calculating FC with functions. To make it easier for readers to understand, we simplify the business logic for the time being and only do the following two actions:

Capture a picture through the FFmpeg command

Save to OSS

Import json, oss2, subprocessHELLO_WORLD = b'Snapshot OK!\ n'OSS_BUCKET_NAME = b'snapshot'def handler (environ, start_response): logger = logging.getLogger () context = environ ['fc.context'] request_uri = environ [' fc.request_uri'] for k V in environ.items (): if k.startswith ('HTTP_'): pass try: request_body_size = int (environ.get (' CONTENT_LENGTH') 0) except (ValueError): request_body_size = 0 # get the address of the live stream rtmp_url = request_body.decode ("UTF-8") # capture a picture through the FFmpeg command cmd = [/ code/ffmpeg','- iTunes, rtmp_url,'- frames:v', '1records,' / tmp/snapshot.png'] try: subprocess.run (cmd, stdout=subprocess.PIPE) Stderr=subprocess.PIPE, check=True) except subprocess.CalledProcessError as exc: err_ret = {'returncode': exc.returncode,' cmd': exc.cmd, 'output': exc.output.decode (),' stderr': exc.stderr.decode ()} print (json.dumps (err_ret)) raise Exception (context.request_id + 'transcode failure') # upload to OSS creds = context.credentials auth = oss2.StsAuth (creds.access_key_id Creds.access_key_secret, creds.security_token) bucket = oss2.Bucket (auth, 'http://oss-{}-internal.aliyuncs.com'.format(context.region), OSS_BUCKET_NAME) logger.info (' upload pictures to OSS...') For filename in os.listdir ("/ tmp"): bucket.put_object_from_file ("example/" + filename, "/ tmp/" + filename) status = '200 OK' response_headers = [(' Content-type', 'text/plain')] start_response (status, response_headers) return [HELLO_WORLD]

Let's analyze this code. First of all, in addition to the standard module of Python, the Python runtime environment of function Computing FC also includes some commonly used modules, including oss2, which is used to operate Ali Cloud object storage OSS in the function. Therefore, we can introduce the oss2 module directly into the code.

Function Computing FC integrates many types of triggers. This example function uses HTTP triggers, and each HTTP request triggers the execution of a function. For Python code that uses HTTP triggers, the entry function is handler, where the environ parameter carries the client-side information and context information of the calling function. We can parse the address of the STMP live stream from the HTTP request Body, and capture an image through the FFmpeg command.

In this code, the ffmpeg executable is located in the / code directory and can be executed through the / code/ffmpeg path. This is because when we deploy the function, we have packaged the ffmpeg executable and this code in this directory, and when we next introduce the function deployment, we will further explain how to package the function code with the executable.

In the process of uploading the image file saved in the / tmp directory to OSS, we can obtain the credential to access OSS directly from the function context, so that we no longer need to get information such as accessKey,accessSecret through the configuration file, thus reducing the workload.

Deployment function

First, we create a working directory locally, and create a subdirectory called code under this directory, and copy the ffmpeg executable of the Linux environment to the code directory, so that we can call the ffmpeg command through the path / code/ffmpeg in the code.

Next, start the most important work by creating a template.yml file in the current working directory that describes all the deployment information.

ROSTemplateFormatVersion: '2015-09-01'Transform:' Aliyun::Serverless-2018-04-03'Resources: # Service snapshotService: Type: 'Aliyun::Serverless::Service' Properties: Description:' Snapshot Semo' Policies:-AliyunOSSFullAccess # function LogConfig: Project: fc-bj-pro Logstore: fc-log # function created previously Snapshot: Type: 'Aliyun::Serverless::Function' Properties: Handler: index.handler Runtime: python3 MemorySize: 128 Timeout: 600 CodeUri:'. / code' # HTTP trigger Events: http-test: Type: HTTP Properties: AuthType: ANONYMOUS Methods: ['POST']

The configuration information is relatively simple, we need to define a service first. Service is the unit of functional computing resource management. From a business scenario, an application can be split into multiple services. From the dimension of resource usage, a service can be composed of multiple functions. For example, a data processing service is divided into two parts: data preparation and data processing. The resource requirement of the data preparation function is small, so you can choose a small specification instance. The resource demand of the data processing function is large, so you can choose a large size instance. You must create a service before creating a function. All functions under the same service share some of the same settings, such as service authorization and log configuration. In this code, we create a service called snapshotService, which has full access to OSS and references the log project and log repository we created earlier.

In the configuration of function instance specifications, since each computing instance only needs to process one video stream, we can choose the lowest specification, that is, an instance with 128MB memory.

Next, we define a function, configure its corresponding runtime environment, entry method, code directory, timeout, and so on, and define a HTTP trigger for this function. In this code, the function is named snapshot, the corresponding runtime environment is Python3, and a HTTP trigger named http-test is defined.

Execute fun deploy in this working directory, and if you see the prompt server SnapshotService deploy success, it means that the code and the ffmpeg program have been packaged and deployed on the cloud.

In the services and functions menu of the console, we can see the uploaded services and function information, and we can even view and modify the function code online.

Execution function

Since this is a function of type HTTP, we can issue a HTTP request to the function calculation FC through the curl command or other HTTP tools such as Postman to verify the execution result of the frame capture operation. Of course, the function calculation FC console also provides a visual operation interface to verify the function, in which you can quickly initiate a HTTP request.

If the function executes successfully, we can go to the OSS console to check whether the captured image has been uploaded successfully. So far, we have built the most basic Serverless video frame capture architecture. You can use the HTTP request trigger function to intercept an image from the video stream and upload it to OSS.

Continuous frame interception

The frame capture operation of a single image is very simple. After the execution of the FFmpeg command, you can upload the images in the temporary folder directly to OSS, and then complete the life cycle of the function. Frame capture of a single image can meet many business scenarios, but if you need to capture frames continuously at a fixed frequency and upload the saved image to OSS in real time, you need to make some changes to the code.

Configure message queuing Kafka

To reduce the workload of the content audit service during peak business periods, we can introduce message queuing Kafka between the frame capture service and the content audit service, so that the content audit service can process saved images asynchronously by consuming messages received from Kafka. In the video frame capture architecture, Kafka plays a very important role in information transfer. The greater the concurrency of live broadcasting and the higher the frame capture frequency, the greater the pressure on Kafka. Especially during the peak period of business, we need to keep Kafka stable under heavy workload. Using the message queue Kafka provided by Aliyun directly can help us greatly reduce the maintenance workload of Kafka cluster and obtain highly available Kafka services that can be dynamically expanded in the simplest way.

We can open the Kafka activation page and purchase Kafka instances with corresponding specifications according to the needs of the actual scenario. In the basic information of the Kafka console, we can see the default access point corresponding to the Kafka instance.

Next, we go to the Topic management interface and create a Topic for the frame capture service.

The default access point and Topic name of the Kafka instance are the information we need to use in the next steps.

Install Kafka client SDK

Before that, we need to do some additional operations to get the function's ability to write to Kafka.

Because we need to use Kafka SDK, we can install the kafka SDK module through the Funcraft tool in conjunction with the Python package management tool pip:

Fun install-runtime python3-package-type pip kafka-python

After executing the command, there is the following prompt:

At this point, we will find that a .fun folder is generated under the directory, where the dependent packages we installed are located:

Get access to resources in VPC.

By default, the function calculation cannot access the resources in the VPC, because we need to allow the function to access the Kafka service deployed in the VPC, we need to manually configure the VPC feature and related permissions for the service. We can refer to the configuration function to access the resources in the VPC and connect the function to the Kafka service. The principle is that by granting the

ENI access to VPC, and insert the ENI ENI into the instance that executes the function, so that the function can access the resources in your VPC.

Code implementation

You can use the following FFmpeg command to intercept frames frequently as specified:

Ffmpeg-I rtmp://xxx.xxx.xxx.xxx:1935/stream/test-r 1-strftime 1 / tmp/snapshot/%Y%m%d%H%M%S.jpg

While the command is running, the current process of the Python program will wait for the video stream push to finish, so we need to modify the function code to start a new scanning process. The scanning process constantly checks the picture directory, and once it finds that a new picture is generated, it uploads the picture to OSS, sends the frame capture information to Kafka, and finally deletes the picture from the picture directory.

Import logging, json, oss2, subprocessfrom multiprocessing import Processfrom kafka import KafkaProducerHELLO_WORLD = b'Snapshot OK!\ n'OSS_BUCKET_NAME = b'snapshot'logger = logging.getLogger () output_dir ='/ tmp/shapshot'# scan image directory def scan (bucket, producer): flag = 1 while flag: for filename in os.listdir (output_dir): if filename = = 'over': # ffmpeg command completed Ready to stop scanning flag = 0 continue logger.info ("found image:% s", snapshotFile) try: full_path = os.path.join (output_dir, filename) # upload to OSS bucket.put_object_from_file ("snapshot/" + filename) Full_path) # send to Kafka producer.send ('snapshot', filename.encode (' utf-8')) # delete picture os.remove (full_path) except Exception as e: logger.error ("got exception:% s for% s", e.message, filename) time.sleep (1) def handler (environ) Start_response): logger = logging.getLogger () context = environ ['fc.context'] # create picture output folder if not os.path.exists (output_dir): os.mkdir (output_dir) # parse HTTP request Get the address of the live stream request_uri = environ ['fc.request_uri'] for k, v in environ.items (): if k.startswith (' HTTP_'): pass try: request_body_size = int (environ.get ('CONTENT_LENGTH') 0) except (ValueError): request_body_size = 0 rtmp_url = request_body.decode ("UTF-8") # launch KafkaProducer producer = KafkaProducer (bootstrap_servers='XX.XX.XX.XX:9092,XX.XX.XX.XX:9092') # launch OSS Bucket creds = context.credentials auth = oss2.StsAuth (creds.access_key_id, creds.access_key_secret, creds.security_token) bucket = oss2.Bucket (auth 'http://oss-{}-internal.aliyuncs.com'.format(context.region), OSS_BUCKET_NAME) # start the scanning process scan_process = Process (target=scan, args= (bucket, producer)) # cmd = ["/ code/ffmpeg", "- y", "- I", rtmp_url, "- f", "image2", "- r", "1" by FFmpeg command "- strftime", "1", os.path.join (output_dir, "% Y%m%d%H%M%S.jpg") try: subprocess.run (cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True) except subprocess.CalledProcessError as exc: err_ret = {'returncode': exc.returncode,' cmd': exc.cmd, 'output': exc.output.decode () 'stderr': exc.stderr.decode ()} logger.error (json.dumps (err_ret)) raise Exception (context.request_id +' transcode failure') # write flag file Child process end work os.system ("touch% s"% os.path.join (output_dir, 'over')) scan_process.join () producer.close () status =' 200 OK' response_headers = [('Content-type',' text/plain')] start_response (status, response_headers) return [HELLO_WORLD] is further optimized

Long video clip

The default elastic instance of FC calculated by function is 600 seconds, that is, the maximum execution time of a function is 10 minutes, that is, a function will exit automatically if it has not finished the calculation task after running for 10 minutes after it is triggered. This limit will affect the truncation of video streams with a playback time of more than 10 minutes. Long videos are very common. How to bypass this limit for long videos? We can solve the problem through the following three solutions:

Each function only truncates one frame: when the frame capture frequency is relatively low, or when you only need to intercept the video stream at a specific point in time, we do not need to keep the life cycle of the function consistent with the playback cycle of the video stream. You can allow each function to capture only a single frame after it is started. With custom triggers, functions can be started at the necessary point in time, or functions can be more complex orchestrated through Serverless workflows.

Through multiple function relays: function calculation FC has a built-in fc2 module, which can be used to call each other between functions. In this way, we can control the running time of each frame-cutting function to less than 10 minutes, for example, 8 minutes is a fixed running cycle. Before the end of one function, start another function relay to complete the interception task until the end of the video stream. This scheme is very suitable for scenarios where the accuracy of the frame capture frequency is not very high, because when the tasks are transferred by the two functions, the accuracy of the frame capture frequency can not be strictly guaranteed for about a second.

Use performance examples: in addition to the default elastic instances, function Computing FC also provides performance instances with large performance strength, higher resource limit, more adaptation scenarios, and the ability to break the 10-minute limit of execution time. The expansion speed of performance instances is slow, and the elastic scaling ability is not as good as that of elastic instances. However, we can improve the resilience of performance instances by combining multiple concurrency of single instance and reservation mode. For a specific introduction, you can refer to single instance multiple concurrency and reservation mode.

Cost optimization

Function computing provides rich measurement models, competitive pricing, and detailed resource usage indicators, combined with the application-centric architecture of Serverless, so that resource management is unprecedentedly convenient and competitive costs can be obtained in different scenarios.

According to the differences in resource specifications and flexibility requirements, function calculation provides two measurement modes: prepaid (annual or monthly plan) and post-payment (pay-by-quantity). Under normal circumstances, you only need to use the pay-by-quantity model, and you only need to pay for the actual function computing resources, and you do not need to purchase resources in advance. However, according to the actual use of resources every day, users can flexibly choose the prepaid mode to save costs. The prepaid mode means that the user buys a certain amount of computing power in advance, and during the life cycle of the prepaid computing power, the resources consumed by the function can be deducted second by second, while the unit price of the prepaid mode is always less than that of the post-payment model.

On the resource center page of the function computing console, you can clearly see the actual use of resources under the current account, including the stable and flexible parts of the use of resources, through this information, prepaid and post-paid resources can be allocated reasonably. In the resource usage details chart, the green curve is the actual resource usage per day, and the yellow line represents the usage that can be deducted by prepaid resources. We can appropriately increase the proportion of prepaid resources according to the actual situation, so that more resource usage is covered by prepaid resources, thus reducing the cost of sorting resources.

This is the end of the article on "how to implement the video frame capture architecture based on Serverless technology". Thank you for reading! I believe that everyone has a certain understanding of the knowledge of "how to realize the video frame capture framework based on Serverless technology". If you want to learn more, you are welcome to follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.