Shulou(Shulou.com)06/02 Report--

Mr. Qigong said that Tang poems were all shouted out.

But the flirtatious talents in the heyday of the Tang Dynasty sat together and shouted, and this painting style was obviously very inconsistent. So more accurately, Tang poetry is sung.

Wang Zhiyi drank with Gao Shi and Wang Changling, and finally listened to the story of female singers singing "Liangzhou ci". Li Bai's "fighting the South of the City" and "will enter the Wine" are all directly used in Han Yuefu. Liu Yuxi's "Zhuzhi ci" is directly a folk song in the mountains of the south.

These poets all use logic to tell us that Tang poetry has songs and can be sung. So the question is, more than 1300 years have passed, where are these Tang songs now?

Recently, Huawei's Mate 20 Pro R & D team worked with Professor Zhao Weiping of the Shanghai Conservatory of Music and other researchers to complete an attempt: through AI technology and the terminal AI computing power provided by Huawei Mate 20 Pro, they joined hands to "reproduce" the missing notes behind Tang poetry and make Tang qu reverberate in their ears.

Here I would like to explain to you the practice of this "Tang flavor dish". Perhaps many friends have had this question: AI sounds great, but what can it actually do?

The answer may be in some rhyme of "Spring River Flowers and Moon Night".

Where is Tang Yin?

Summer is coming, and friends who like ancient rhythms are ready to go out in search of ingredients. Generally speaking, they will find that the rhythm of the Tang Dynasty is even harder than that of Tricholoma matsutake.

China has always been a country that pays attention to rhythm and is good at recording it. The chime unearthed in the Neolithic Age is actually the "stone memory" of the rhythm perceived by the ancients at that time. As early as in Guanzi, there has been a record and accurate description of the concept of "palace merchants".

There are a lot of records about rhythm in all dynasties. But only in the Tang Dynasty, this prosperous dynasty may be because it is too inclusive and does not like to record rhythms in a systematic way. Finally, the phonology of the Tang Dynasty became a "literature black hole". There are Liu James Law in Sui Dynasty and Wang Pu Law in five dynasties, but there are few records on the system of applying specific rhythm in Tang Dynasty, and we can only search for a little shadow of prosperous Tang Dynasty from folk music and Central Asian music in the Western region.

Another possibility is to look for Tang sounds in medieval literature. Due to the lack of direct literature records, academia often can only rely on ancient music to restore the sound of the Tang Dynasty. "in modern times, the research on the law of each generation is quite fruitful. But at present, there is no Qin score handed down in the Tang Dynasty, only the Jieshi Diaoyou Orchid handed down by Liang Qiuming in the Southern Dynasty is copied by the Tang Dynasty. This is the only evidence of the actual use of the law in the Tang Dynasty."

This "Jieshi Diaoyou Orchid" is very powerful. It is not only the only surviving Tang Dynasty music, but also the only surviving ancient song composition.

The reason why "Jieshi Diaoyou Orchid" is the only written score, not because other ancient scores have been destroyed, but because since the middle and late Tang Dynasty, the composer began to use the reduced word score to record the piano music.

The so-called subtraction spectrum, in form, looks like a compound character formed by reducing strokes of Chinese characters. From the point of view of people who do not have the foundation of Guqin, they are basically books of heaven. The basic logic of this spectrum is to divide a word block into upper and lower parts. The upper part represents the left finger and emblem, and the lower half represents the string and right finger. To put it bluntly, the subtractive score is to express the technique of playing the piano in Chinese characters. This kind of score can only remember the fingering movements and string sequence and emblem, but not the beat and rhythm like the European music score. Therefore, it is difficult for people with a music foundation to grasp the trend of their music.

According to legend, the reduced character spectrum was invented by Cao Rou in the Tang Dynasty, but there are many other legends, and some people even attribute the patent of the reduced character spectrum to the famous Cao Zhi. In the Ming Dynasty, Yuan Junzhe described the development process of the ancient piano score according to the "word spectrum" in the Taiyin complete works of Tian Zhiweng in the Southern Song Dynasty: "the score-making began in Yongmen Zhou and Zhang Fu, so the other score is not as good as that of future generations. Zhao Yeli is famous for both ancient and modern times, and it is easy to find out. The sages made it, intending to take Zhou Bei, but his text was extremely complicated, but the movement and the more two lines did not form a sentence. Later, Cao Rou wrote the method of reducing words, which was particularly easy to understand."

However, due to the early age of the literature, the real inventor of this "music code" has been difficult to test. It began in the Tang Dynasty and flourished in the Song Dynasty, and a large number of reduced word scores began to appear in the Ming Dynasty, which objectively became the hope of preserving Tang Yin and Tang qu. If you want to "recreate" Tang music, cut-word scores and guqin music have become the most suitable raw materials.

The ingredient has been found. The next step is to simmer it in AI for a while.

Put the "subtracted word spectrum" into the AI soup and cook it into a sour and sweet delicious "rule".

Having found good ingredients, it is not easy to start cooking. If you want to use the subtractive spectrum as the base material to reconstruct Tang qu, the first step is to allow AI to automatically recognize the subtractive spectrum.

At present, it is said that the number of subtracted words belonging to the Tang Dynasty is not large. However, the piano score of this period generally did not form a relatively unified law of word reduction after the Song Dynasty. Different pianists may use different rules of word subtraction, and eventually form a variety of different subtraction systems. Although the fingering in this period is relatively simple, the naming method is not unified and the narrative format is complex, which leaves a lot of problems for AI.

With the help of researchers such as Professor Zhao Weiping, the research and development team of Huawei Mate 20 Pro must complete the task of identifying ancient spectra with AI. In general, AI recognizes the subtractive spectrum in three steps:

1. Image preprocessing. Most of the subtractive spectrum belongs to photocopy, the image is blurred, and the notes and music are often mixed and interlaced, so it is difficult for AI to identify clearly. In this step, the technical team is required to invoke the AI capabilities of HiAI Engine on Huawei mobile phones, such as document detection, document correction, image and text super-resolution enhancement, to ensure that the phone can clearly identify each ancient spectrum and quickly complete the collection and digitization of ancient spectrum documents. For the R & D team, this is a task that can be "lazy", because Huawei phones, including Mate 20 Pro, have built-in HiAI Engine, which can be used to shoot subtracted word spectrum with twice the result with half the effort by directly calling the open corresponding AI capability interface.

2. Train the spectral character detection and recognition model of the subtracted word spectrum, and construct the coding and decoding rules and character library. The ultimate goal of the team is to directly transform the subtractive spectrum into a simplified spectrum by extracting features from the subtracted spectrum. However, at the level of transformation rules, the team will face a new problem: AI character recognition in the past, mainly through OCR recognition + natural language understanding. However, the subtraction spectrum is not a text, so the general OCR recognition can not be used, so the team specially trained the AI model for the detection and recognition of the subtraction spectrum. Finally, by building the coding and decoding rules based on the subtracted word spectrum, the R & D team successfully completed the biggest challenge of all processes.

3. Based on AI technology, the translation program of subtractive spectrum is established. Finally, using the trained AI model, the R & D team is able to quickly convert a large number of subtracted word spectrum into modern simplified spectrum. At this point, the AI recognition and translation of the subtractive spectrum is over.

Turn the "model" into Datang-style "music".

These days, a mature AI should be able to compose music by himself.

Although this sounds like a complete difficulty for AI, it is indeed the only way to "reconstruct Tang qu". After getting a large number of digital samples of Tang qu music, we start a lot of training in the cloud, and finally get a deep learning model that can understand the compilation rules of Tang qu.

After that, the R & D team migrates the trained model to Huawei Mate 20 Pro, using only the computing power of end-to-side AI to complete local reasoning, that is, AI arrangement.

In fact, AI arrangement and composition has become a hot spot in the industry in the past two years. Since American pop singer Taryn Southern published an AI arrangement called "I am AI" in 2017, related algorithmic models have emerged one after another.

For example, OpenAI released MuseNet. The model AI learned from hundreds of thousands of MIDI archives, understood the elements of harmony, rhythm and style, and eventually composed its own music using different instruments and different styles.

In Chinese science and technology circles, one of the representative products of composing music in AI is Huawei Mate 20 Pro.

The so-called AI arrangement and composition is essentially running a complex AI model to learn and reverse output different rules of music, musical instruments, rhythm, music theory and so on. This kind of task can just prove the end-to-side AI calculation power of Huawei Mate 20 Pro.

Through the analysis and understanding of the music in the ancient music, a piece of music belonging to Datang was "sung" by AI.

In the video, we can see that the "Spring River Flowers and Moon Night", which was praised by Mr. Wen Yiduo as "overwhelming the whole Tang Dynasty", finally appeared in the ears with the singer's voice and the Tang song "reproduced" in the ancient music.

A cell phone, a whole AI kitchen.

Here is a digression. By running down the process of "reconstructing Tang qu" with AI, we may be able to find such a problem:

Different from ordinary AI development, the cross-border AI experiment of Huawei mobile phone is basically carried out on the local environment of Huawei Mate 20 Pro.

Of course, on the one hand, this is the test of the HUAWEI HiAI open capability platform, in which HiAI Foundation's Kirin 980provides AI model reasoning and computing power, HiAI Engine provides a variety of directly callable AI capabilities API,HIAI Services provides voice assistant and speech recognition services, and may also see such a trend: more and more complex AI tasks will be deployed and applied in terminal products. In the case of "Reconstruction of Tang qu", we can see the following trends:

1. HiAI Engine on Huawei mobile phones has opened up a large number of AI capabilities, such as speech recognition, text recognition, semantic understanding and so on. These capabilities can make AI applications easier and faster to develop and help developers, and there will be more AI capabilities available in the future.

2. Due to the need of privacy protection, the user data involved in the application should be kept locally. In the case of "reconstructing Tang qu", the R & D staff is finally required to complete a recitation. Although sound is not private data in a general sense, it is best to stay local to avoid unnecessary trouble caused by uploading to the cloud, so more and more AI models will be deployed on the end side.

3. The acceleration of end-to-side AI model requires real-time experience. Just imagine, if we sing a Tang poem, but the soundtrack version can not be presented for a long time, it will lead to a poor experience. In order to ensure the real-time experience, the acceleration of the mobile AI model is also essential. The NPU computing power provided by HIAI Foundation can support the efficient operation of the AI model on the mobile side.

These three trends are not only reflected in the case of "reconstructing Tangqu". As more and more mobile devices become more intelligent, the HUAWEI HiAI open capability platform will enable more AI scenarios to help developers quickly develop and deploy AI applications.

The smell is still out of the box.

Finally, it may be necessary for us to talk about the practical use of "reconstructing Tang qu" besides causing endless "prosperous Tang Dynasty reverie".

The whole "Reconstruction of Tang qu" project has left the academic circles with a rapid subtraction spectrum recognition and conversion system, as well as intuitive research and application of ancient music. Next, these AI applications will be used by researchers in related fields for a long time.

By extension, there are a large number of academic applications waiting for the landing of similar technologies. The "non-text symbol recognition" brought about by subtractive spectrum recognition can also be applied to the recognition and translation of minority documents, gold and stone documents, and special documents. For example, to convert Suzhou code into financial and financial materials, and so on. On the other hand, document recognition, the technical logic of speech generation, can be applied to the pronunciation, phonology and music of all previous dynasties in addition to "reconstructing Tang qu".

At this stage, most end-users think that the combination of AI and voice can only do speech recognition and semantic understanding. However, from the case of "Reconstruction of Tang qu", it can be seen that identifying ancient music, reasoning melodies, and "reproducing" lost music-there is more potential for AI and sound to be tapped.

In the final analysis, Huawei's Mate 20 Pro R & D team is doing one thing: associate AI with those fantastic ideas. Whether in the rainforest, in the pile of paper, or in every bit of life, AI can accomplish some unexpected tasks.

The so-called throwing brick to attract jade, can also throw Tang qu and attract public voices. AI needs to be developed by thousands of smart minds and enjoyed by billions of interesting souls.

Behind the dish of "Reconstruction of Tang qu", the true flavor of Mobile AI is that everyone, everything, can be related to AI.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.