Shulou(Shulou.com)11/24 Report--

Beauty is a technology often used in daily life, such as short video, video social networking, live broadcast and so on. Good beauty can add points to your appearance, skin color and makeup in the video, and then bring a better visual experience. However, beauty is not everything. When we watch live broadcasts, some platforms appear whitening and red screen, dazzling screen stripes and blurred details when we watch live broadcasts. As a result, beauty sense disappears, or some platform beauty parameters are not configured properly, and the VJ looks like plastic surgery failure. In this case, if you ignore the details of the beauty and RTC integration process, the video will not be able to present the best picture quality, and the visual viewing experience will be greatly reduced. In the long run, it will lead to a decline in the popularity of anchors, a decline in the flow of the platform, and so on.

The "Real-time HD Super Picture quality" solution recently released by Sound Network is a set of solutions for real-time video scenes focused on improving video picture quality and user experience, including the ultimate picture quality, beauty, silky and smooth, low-code HD, PC broadcasting, game upgrade, data monitoring, use of worry-free eight gift packages, of which "Beauty" has RTC + beauty best practices,

A series of capabilities, such as high-precision facial features fitting, highly stable point tracking, and no dead corner coverage of beauty scenes, can not only bring more beautiful and natural beauty effects to VJs, but also help enterprises lower the development threshold and quickly integrate online beauty.

RTC + Beauty Beauty Best practices to put an end to black screen and stutter after Beauty access.

When an enterprise's LVB / video platform is connected to Beauty, it is very important to choose the best integration method. No matter how good the beauty effect is, if there is a problem with the access method, it is also a waste of work. As a pioneer of real-time interactive cloud industry, Sound Network has accumulated a wealth of experience in real-time audio and video integration of beauty SDK, and summed up the best practices of real-time audio and video access to each beauty process. Because the integration mode of each manufacturer is different and complex, it needs to switch and call back and forth in different video formats, which is prone to errors in access processing, resulting in a series of problems such as black screen, stutter, hot and so on.

In the continuous practice, the sound network has summed up several optimal technology paths, such as:

Farewell to black screen: sound net uses RTC SDK direct collection, dynamically uses the best beauty processing, texture input processing or I420 input processing according to the collection data type, and the strong compatibility greatly reduces the probability of black screen.

Ensure fluency: the sound network uses a combination of multiple processing methods, single texture processing, I420 processing and dual input processing, as well as asynchronous processing optimization, which minimizes the time-consuming beauty processing and achieves a stable and smooth frame rate. Overseas low-end computers can also easily enjoy high-definition image quality.

Avoid being hot: the sound network chooses to put more beauty treatments on GPU to reduce the use of CPU and avoid hot phones, so that VJs do not have to worry about broadcasting for a long time, helping the platform to further improve VJs' activity and stickiness.

In addition, with the blessing of the best practice of beauty, the details such as hair and eyelashes are restored with high fidelity; in terms of color performance, while restoring the true color to the maximum extent, the whiter effect of the anchor portrait is realized, which is more in line with the aesthetic appreciation of Asians and meets the aesthetic needs of both domestic and overseas. It not only helps the VJ present the VJ's human beauty in a higher definition, but also takes into account the true restoration of the details of the background items, which is suitable not only for show shows and video dating scenes, but also for e-commerce scenes.

Scene Beauty API significantly lowers the development threshold

In order to help developers and enterprises effectively reduce development costs and quickly launch beauty features, Soundnet provides beauty scene API solutions to reduce integration workload and code volume. In terms of integration, the customer's integration workload is reduced by 90%. It originally takes 7-14 days to develop, but it only takes 1-3 hours to complete after optimization.

In terms of the amount of code, the original requirement of 300 + lines of code has been reduced to less than 10 lines, in which Soundnet has solved many core technical difficulties: brainless switching between different beauty filter vendors, automatic compatibility between multiple video frame formats, best practices between Beauty filter SDK and RTC, best configuration of preset beauty filter parameters and general configuration of images.

Scene beauty API can adapt to a variety of scenarios, including show single anchor, show competition, show joint broadcasting, show multi-screen, e-commerce live broadcast, etc., achieving low consumption and high stability in all scenarios.

High precision facial features fitting and high stable point tracking

Beauty is a comprehensive technology to beautify the face in the image. the primary task of beauty is to accurately locate the face and facial features in the image, and then carry out personalized beautification according to the features of the face. The basic technologies of beauty dependence include face detection and face key point detection. Face detection is to find the location of the face from the image and use a rectangular box to mark the face. Because the information provided by face detection is rough, it is necessary to detect face key points in order to obtain more detailed and detailed face information, such as the face shape, mouth, eyes, nose position and geometry.

Face key point detection, also known as face key point location or face alignment, is to further locate the facial features on the basis of the specific location of the face in the image obtained by face detection. These facial features position information is some discrete points with a clear definition of speech, so they are called face key points. Usually, facial key points are defined in the areas of cheek, mouth, eyes, nose and eyebrows, which connect facial key points and can describe the geometric features of human face. Using the key points of the face, we can locate the position of the mouth, eyes, nose and eyebrows, and infer the geometry of the face, mouth, eyes and head posture.

On the other hand, the beauty ability of the "super picture quality" of the sound network can achieve high-precision facial features fitting, keep the facial features and contours of each part of the face always accurate, and keep the points stable no matter in static or moving scenes, reject jitter and delay, and finally locate the key points of the face accurately, quickly and stably under different expression, posture, lighting and occlusion conditions, so as to achieve the best beauty effect.

The best beauty effect of multi-scene and multi-race

After ensuring the accurate recognition of the key points of the face, the sound network can also achieve the best beauty effect of multi-scene and multi-race. On the one hand, the sound network can fine tune complex scenes, such as bright and dark light, large expression, large angle, large occlusion, small face, motion change and other scenes, which can ensure the most beautiful appearance effect, and achieve omni-directional coverage without dead angles in the beauty scene at the same time.

On the other hand, for multi-ethnic facial features, voice network can also accurately identify, and ensure the best beauty effect of different races, for enterprises, beauty features can be more flexibly adapted to the sea App.

If you want to further the beauty solution and "Real-time HD Super Image quality" solution, you can find this article on the official account of Wechat, and click on the bottom to read the original text for further consultation.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.