Shulou(Shulou.com)06/02 Report--

This article shares with you how to understand the implementation of iOS Short Video SDK technology. Xiaobian thinks it is quite practical, so share it with you to learn. I hope you can gain something after reading this article. Let's not say much. Let's have a look together with Xiaobian.

The Short Video SDK mainly contains two core functions: "Video Recording" and "Video Editing."

Video recording includes: video capture, beauty, filters, camera switching, audio and video capture parameters settings and other functions;

Video editing includes: video import, synthesis, special effects, filters, subtitles, stickers, music, playback, export and other functions.

I. Short Video SDK framework

As shown in the above figure, the technical framework of Short Video SDK is divided into three levels, namely, core processing layer, service logic layer and service interface layer:

Core processing layer: responsible for the technical implementation of the bottom layer of Short Video, including media combination, GPU data input, video transition, video special effects, video pixels, adding subtitles/stickers, image to video, GPU data, audio Media Processing Service, audio reverberation, audio adjustment, audio delay, video preview and media synthesis;

Business logic layer: responsible for implementing business logic of Short Video, including recording logic processing and editing logic processing, specifically including material timeline management, audio timeline management, transition effect management, caption layer management, sticker layer management and video special effect management;

Service interface layer: responsible for the connection of upward services, including recording interface and editing interface.

II. Video recording

The implementation process of video recording is as shown in the above figure. First, the acquisition camera is turned on to collect images and sounds. During the acquisition process, Media Processing Service and audio processing are supported for the collected data respectively, including filtering, noise reduction, adding stickers, mixing, etc. After processing, the video editing process is entered, and finally the MP4 file is generated by ffmpeg encoding. Video recording uses GPU and hardware encoding throughout, making CPU usage very low, even on poor performance phones can also run well.

Click [Read] to learn more about "Short Video SDK details"

III. Video editing

Video editing refers to editing the recorded or selected video content. Its business process is to import the video content to be edited first, and then perform video and audio processing respectively, including adding filters, subtitles, stickers, video clipping and copying, video shifting and rotation, adding background music, mixing, etc. The processed audio and video can be previewed in real time, and finally the edited media files can be synthesized and exported.

Media Processing Service

The implementation flow of Media Processing Service is as shown in the figure above. The input video content is subjected to video transition and special effect processing such as cropping, segmentation, copying, rotation and filter. In addition, subtitles and stickers are added. The processed video content is mixed by mixer and finally encoded and output.

Media Processing Service mainly uses ffmpeg to decode the original video, then maps the decoded YUV data to GPU texture to adjust the parameters, and then edits the adjusted parameters.

2. Audio processing

The implementation flow of audio processing is as shown in the above figure. First, audio tracks need to be separated from video sources to form native audio tracks. Then, audio tracks of added background music are inserted into audio tracks of original sound. By adjusting the volume of original sound and background music, a mixture of original audio and background music is formed. Finally, the mixed audio tracks and processed video tracks are synthesized by media merging.

3. Media preview

The implementation process of media preview is as shown in the above figure. Variable content such as video content, audio content and background music is processed through custom synthesis thread, including information transfer, transition, etc. Finally, the processed video and audio are previewed instantly through player.

4. Media synthesis

The implementation flow of media synthesis is as shown in the above figure. First, multiple video contents and audio contents after mixing are read according to time chain, and then the video contents and audio contents are merged respectively. The merging implementation method is to encode the video contents and audio contents based on ffmpeg, merge the encoded video and audio into an overall media file, and finally output the merged mp4 file to the outside.

IV. Application Scenarios of Short Video

1. E-commerce product display

Merchants can quickly merge, crop and edit videos or pictures shot through the Short Video SDK, and make short videos for commodity display with special effects such as subtitles, stickers and music, so as to display commodities more vividly and improve users 'shopping experience. Combined with Jingdong Video on Demand products and player SDK, a complete end-to-end solution for Short Video can be formed.

2. Social Creative Sharing

Short Video help users express their views more vividly and deliver more realistic scenes. In addition, new videos can be created through special effects such as beauty, filters, subtitles, stickers, etc., merging and editing functions, combining personal creativity or focus content into a new Short Video, improving video hotspots and click rates.

The above is how to understand the implementation of iOS Short Video SDK technology. I believe that some knowledge points may be seen or used in our daily work. I hope you can learn more from this article. For more details, please follow the industry information channel.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.