Shulou(Shulou.com)11/24 Report--

Dr. Xue Hongyang is an expert in the field of multimodal human-computer interaction and joined Yizhi Intelligence as a partner as the head of the virtual human business research and development team in December last year. Dr. Xue, a graduate of Zhejiang University, creatively put forward the first important multimodal task of video Q & A with Professor Zhao Zhou of the Institute of artificial Intelligence of Zhejiang University in 2017, and made outstanding scientific research achievements in this field. However, in Dr. Xue's view, the research results will eventually need carriers to achieve landing in order to exert the due value of science and technology, so he has been committed to applying AI to various industries after graduation. Turn technology into truly valuable productivity.

Fruitful research results to create a precedent in the field of video Q & A

Dr. Xue Hongyang has been working all the way in the School of computer Science of Zhejiang University since his undergraduate degree, and then went to the State key Laboratory of computer aided Design and Graphics (CAD&CG) to study for his doctorate directly, under the personal guidance of Professor he Xiaofei and Professor Cai Deng. During his doctoral period, he has achieved fruitful research results, and his diligence and talent have also won various honors, including the National doctoral Scholarship, ARM Scholarship and Outstanding graduates from Zhejiang Province.

Dr. Xue focuses on computer vision and multimodal research, in-depth exploration of 3D computer vision, image completion, video Q & An and other different research directions. Among them, the most eye-catching is the important multimodal task of video Q & A, which was put forward jointly with Professor Zhao Zhou at the beginning, and then creatively put forward a series of innovative attention mechanisms, such as unified attention mechanism and the model based on attention mechanism on syntax tree, etc., achieved the research results of SOTA in the field of video Q & An at that time, and published many papers in CCF Class A journals and conferences. This series of research results have laid the foundation for Xue Hongyang's postdoctoral career, and have been widely used in many scene fields in recent years, realizing the commercial transformation of scientific research achievements.

Dr. Xue is doing research.

Using AI to empower games to create a new height of intelligent decision-making

After graduating from his doctorate, Dr. Xue Hongyang put his enthusiasm and wisdom into the forefront of the field of artificial intelligence, that is, the research and application of reinforcement learning, and used games as the carrier to explore the theory and practice of AI in intelligent decision-making. And many of his research results have been widely used in the game products of NetEase and other well-known companies, injecting excellent companion robots and competitive AI into the game, giving the game more interesting. This not only has a significant improvement in user stickiness and game revenue, but also saves a lot of manpower and resources needed for the development of traditional games playing robots. This approach greatly reduces the cost of developing game AI, further improves the overall efficiency, and brings inestimable value to the entire game industry.

It was during this period that Dr. Xue Hongyang began to gradually apply the latest academic achievements to the actual game AI problems, groping and building a set of landing methodology and process. At the same time, in the Global AI Football match (GRF) held by Google in 2020, Dr. Xue Hongyang successfully guided the team to achieve the fourth place among more than 1000 teams around the world with his reinforcement learning algorithm, which once again proved his excellent strength and forward-looking vision in the field of AI, and represented a further step in the link between technology and applications that Dr. Xue has been pursuing.

Dr. Xue is studying the literature.

The research results of digital human algorithm lead the progress of the industry

In addition to exploring the application of artificial intelligence in the game field, after joining Yizhi Intelligence, Dr. Xue began to actively devote himself to the research of algorithm technology related to digital human products. Its research content covers many aspects such as voice-driven mouth shape, face generation, face fusion, AIGC and so on, and a large number of research results have been successfully transformed into the company's digital human-related products. What is more worth mentioning is that in just a few months, Dr. Xue has overcome the current problem of low accuracy of digital human voice and lip synchronization.

In digital human products, the fusion and correlation ability of speech and image modal features is the key to determine the accuracy of voice and lip synchronization. The digital human algorithm model, led by Dr. Xue Hongyang, can accurately associate different modal features, and the feature correlation module has a strong generalization ability, and only a small amount of data is needed for model training. digital people can achieve the industry-leading lip synchronization effect, and its synchronization accuracy is nearly 99.9%, which is higher than that of other similar products in the industry, which belongs to the top level in China.

In addition, the algorithm has high generalization ability and robustness, and can support more than 100 languages around the world. even in the case of foreign language speech-driven, the effect of digital human is no different from that of Chinese. And the algorithm can also be compatible with speech including background noise and background music, in these cases, it can also synthesize very realistic digital human video in most of the time to meet the needs of applications in singing and other diversified situations. Not only that, the digital human algorithm can also support real-time interaction, can communicate with people smoothly and naturally within the delay imperceptible to users, and achieve the same effect as real people. At present, the innovative research results have been applied to intelligent digital human products, including short video production, digital live broadcast and so on, and its effect has won unanimous praise and praise from industry peers and users.

Since its launch in March this year, Intelligent Bud Digital Man has rapidly occupied the high ground of the market by virtue of its unique advantages, and the channel network covers all provinces throughout the country, empowering major brands. Through in-depth cooperation with various enterprises, Yapeng Digital Man has achieved diversified business expansion and application in just a few months, thanks to the innovative R & D and continuous efforts of Dr. Xue and Yizhi Intelligent team.

Dr. Xue and his little friend Yichi

Promote the development of digital human technology and promote social progress

In the future, Dr. Xue Hongyang said that he will continue to devote himself to the research and optimization of algorithms related to digital human, so as to improve the intelligence and reality of digital human. He hopes that by reducing the operating cost of digital people, this technology will become an infrastructure closely related to everyone in the new era, so that everyone can experience the convenience brought by digital people.

Dr. Xue Hongyang firmly believes that digital human technology can effectively improve the overall production efficiency of society, reduce the cost of daily communication, and thus promote social progress. As the director of the Joint Research Center of Zhejiang University, he also expects more people to join him in this far-reaching scientific research work. The digital labor in the intelligent era will be truly transformed into a new form of labor that promotes shared development and realizes human liberation.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.