Shulou(Shulou.com)11/24 Report--

Thanks to CTOnews.com netizen Coje_He for the clue delivery! CTOnews.com, November 21 (Xinhua)-- Deepmind has released an audio model called Lyria, which can be used to generate music with musical instruments and vocal voices. In addition, Deepmind has developed a music authoring tool, Dream Track, by working with YouTube to integrate the Lyria model, which claims to enable video creators to "turn ideas into works more efficiently".

The researchers describe the current challenge of generating music from the AI model because the music itself contains a very high information density, in which there may be multiple beats, notes and harmonies every second. This also makes "generating music" more complex than "generating language (text to speech)". For the AI model, it is also more difficult to maintain continuity in long music sequences, because the model needs to maintain the fluency and consistency of music in different phrases, stanzas and long paragraphs.

In addition, because music clips often contain multiple parts and instruments at the same time, which further increases the difficulty of music generation, the relevant audio model must be able to coordinate a variety of sounds and melodies, so as to make the generated music more natural.

The Lyria AI model developed by Deepmind is an attempt to deal with the above pain points. the most important feature of this model is that it can generate high-quality music including musical instruments and human voice.

▲ source Deepmind in addition, the Lyria model is also good at music transformation and continuation tasks, so the model can also generate novel or unified subsequent clips based on existing music clips.

The researchers also stressed that the Lyria model has fine-tuning options that allow users to accurately generate music styles and expressions, so the model can "meet the needs of professional music creation while making it easy for amateur users to use."

▲ source DeepmindCTOnews.com noted that at present, YouTube has applied the Lyria model in the short video function "Shorts", and the relevant results have been integrated into YouTube's experimental music creation tool Dream Track, which allows users to generate a variety of soundtracks, and can choose the music styles of artists such as Charlie Puth, Charli XCX and Sia to create a "new interpretation".

▲ source Deepmind it is reported that users can simply enter the theme in Dream Track, and then choose an artist to generate 30 seconds of soundtrack, lyrics, accompaniment and other content for the short video.

▲ diagram source Deepmind

▲ Picture Source Deepmind in addition, Deepmind also said that researchers are extensively exploring the application of AI in the field of music creation. In the future, users only need to hum, and AI will match the melody into a complete song with lyrics, convert the ancient MIDI music into a Remix version, or add a variety of musical instruments to the track.

Deepmind also mentioned that all content generated by the Lyria model will be marked with a SynthID watermark. This is a watermarking mechanism to identify whether a song is generated by AI, which claims to embed "watermark imperceptible" to music generated by AI without affecting the auditory experience.

▲ image source Deepmind researchers mentioned that audio with "sound watermark" can maintain detectability even if noise is added, or MP3 compression is performed, or even for changing tone speed, and the Lyria model can also identify the part of the song generated by the Lyria model by detecting the SynthID in the song, making it easier to identify music theme content and generate subsequent music clips.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.