Shulou(Shulou.com)11/24 Report--

On August 22, the 2023 nail ecological conference of "riding on the back of AI" was held in Shanghai. At the special forum for digital employees, Zhao Jiayu, vice president of nailing and head of the audio and video division, delivered a keynote speech, sharing the progress and application of nailing audio and video in intelligence, as well as the open prospect of video conferencing field based on nail AI PaaS.

Zhao Jiayu said that Nail announced from April that it would fully integrate the large model and complete intelligent reconstruction. with the deepening of the combination of AI and audio and video, many innovative applications have been realized in the nailing conference. In addition to the previous real-time translation, subtitle transliteration and intelligent summary, this time it has also released three new functions: digital separation instead of attending the conference, natural language control conference operation, and Vincent virtual background. Among them, the digital split can replace users to attend the meeting when there is a conflict during the meeting time, and can synchronize the relevant meeting information and conclusions in real time.

He said that in the future, nail audio and video hopes to open more fields to ecological partners, work together to build rich and valuable applications, and improve user experience and efficiency. At the same time, Nail also plans to embed audio and video SDK into the application of ecological partners to create high-quality and easy-to-use audio and video solutions to meet the needs of different industries and scenarios.

The following is the full text of Zhao Jiayu's speech:

Experience and intelligence are the two core keywords of the nail audio and video division in the past year.

In terms of experience, we set up a special project last year and made a lot of optimization from the inside to the outside, and now the entire interface interaction of the nailing meeting will be more beautiful, more convenient and more immersive. At the same time, in the underlying technical architecture, from the acquisition, playback, codec, network to the server, we also do the whole link optimization. Audio and video, especially video conferencing, the most basic demand when we have a meeting is to be able to have a good meeting and hear each other's voice, so we have improved our audio algorithm and anti-weak network. We also look forward to making more use of our nail meetings.

Speaking of intelligence, in fact, the field of video conferencing has done a lot of combination with AI before this big model, integrating AI technology into audio algorithm and video algorithm. For example, AI applications related to video algorithms, such as beauty filter and virtual background, are all very familiar with scene applications. The audio algorithm is also combined with AI to achieve better noise reduction. At present, nail meetings have supported the elimination of more than 300 common noises, such as car horns and keyboard taps, so that users can pay more attention to the meeting.

Another common requirement we have in meetings is subtitles, which are essentially AI capabilities. Now the nailing meeting has also fully integrated Ali Tongyi listening, supporting the translation between Chinese, English and Japanese, and realizing real-time translation and translation. In addition, there is an intelligent director. When people have a meeting in the conference room, if their colleagues on the cable join in the discussion, they will sit too far away to hear them, or speak something that the other party cannot hear. With the combination of AI and original differential array technology, our hummingbird audio lab can pick up sound for a single conference equipment with a distance of 10 meters. When the speaker speaks, support accurate sound source location, real-time track and enlarge his avatar, so that each speaker becomes C-bit, creating a better sense of dialogue.

With the advent of the big model era, we have also carried out related research, hoping to use the big model to do the nailing meeting all over again. Today, Nail launched a new AI PaaS intelligent base, which is open to ecological partners. In the nailing meeting, we also implemented a lot of interesting functions using AI PaaS.

The first is the digital separation of meetings instead of meetings. Professionals will face Wenshan Huihai every day, Wenshan now we use intelligent documents to help you solve, Huihai we have also made some attempts to provide users with the ability of digital split. If your meeting is in conflict or the meeting is not so relevant to you, you can ask your digital personality to open it instead of you. During the meeting, you can actively set some tasks for it through the dialog box of nailing the magic wand, such as focusing on a specific topic. When someone discusses this topic, AI will automatically notify you and tell you about it. You can also set the digital body to send you a summary of the meeting every few minutes, and you can also take the initiative to talk to the digital person to learn about all kinds of information, such as who are in the meeting and whether there are any disputes. At the end of the meeting, the summary of the meeting will be automatically pushed to you.

The second is to use natural language to do intelligent control of meetings. In the past, many users complained that there were too many nailing functions and the entrance was difficult to find. Nailing meetings are actually faced with this problem, although we have made a lot of simplifications, but the threshold for nailing meetings is still a little high for some users. Today, we simplify the function operation through the intelligent assistant in the nail magic wand. You only need to put forward your requirements through natural language, and AI will automatically identify the intention and help you complete the operation. For example, when there are too many people in the meeting, someone accidentally turns on the microphone. It will be more troublesome for you to find this person and mute it. You can tell AI to mute the person who is talking. When you join the meeting, you can also type directly without paying attention to where the entrance is. Through these capabilities, we make the operation and management of meetings easier.

The third is the virtual background of Wensheng. The virtual background has always been loved by users, especially in the past few years, our "little flowers of the motherland" have played this function out of flowers, one in "class in space" and the other in "prairie", and we also use it in our daily work. can effectively avoid the embarrassment of cluttered background. In the past, we used to use the virtual background in addition to the existing template pictures, we need to export to search and download pictures to upload, but now we do not need it. We automatically generate the virtual background through the Wensheng picture, completely liberating the imagination.

Next, the nailing meeting will gradually open the application through AI PaaS to help partners upgrade the scene. In the future, more ecological products can be embedded in nailing meetings, such as interviews, whiteboards and other applications. For example, for the interview assistant that we are co-creating with our recruitment partners, the conversation AI during the interview process can be assisted in real time through resume content and communication information. We hope to be experienced and open, in the intelligent era, together with ecological partners to bring users more interesting and valuable scene applications.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.