Shulou(Shulou.com)11/24 Report--

Data show that there are about 2780 people with hearing impairment in China, and a large number of hearing impaired newborns are increasing every year. With the development and progress of society, the sense of rights, participation and equality of people with hearing impairment are increasing, and their desire to know information and communicate with people with good hearing are also enhanced. Although with the development of science and technology, more and more people with hearing impairment can communicate through voice to text, sign language is still the primary language of communication between deaf people.

Sign language is a special expression and main communication tool for people with hearing impairment. It is a symbol system for communication by means of hand movements, finger gestures, body posture and facial expressions. Different from auditory language, sign language is a visual language. Because the expression and grammatical logic of sign language are different from those learned by hearing people, most people with good hearing can not understand the expression of sign language. Therefore, for people with hearing impairment, access to information is on the one hand, what they desire most is their inner expression, to be "understood" and "understood".

On the afternoon of September 10, the 200p artificial intelligence computing power launch ceremony of Tianjin artificial Intelligence Computing Center was held in Hebei District, Tianjin. At the meeting, Tianjin disabled Persons' Federation, Tianjin University of Technology, Tianjin artificial Intelligence Computing Center and Huawei Technology Co., Ltd. jointly signed a contract, based on the calculation power of Teng AI and the AI framework of Teng Teng complete scene MindSpore, all parties will jointly develop a large model of sign language understanding for deaf people in Haihe er language, create more accurate sign language-Chinese translation products, and help hearing impaired people express and transmit information more conveniently and accurately. Broaden the channels of communication with the outside world.

Photo: Sifang will jointly develop a large model of sign language understanding for the deaf in Haihe er language.

The Engineering College of the Deaf of Tianjin University of Technology is the first college of higher engineering special education in China jointly sponsored by the Tianjin Municipal Government and the China disabled Persons' Federation. It is also the first university in China to offer "inclusive" education for hearing impaired students (hearing impaired students enter regular classes and cross-college training). While actively exploring and constructing an integrated education system for hearing impaired students, the college is also continuing to explore how to build and use massive sign language data resources to improve the quality of life and learning of people with hearing impairment and promote the integration of the deaf and the deaf. Assist the national barrier-free construction through the technical power of artificial intelligence.

Photo: yuan Tiantian, vice dean of the Deaf artificial School of Tianjin University of Technology, introduces the big sign language model.

However, there are the following difficulties in the construction of sign language large model: (1) it is difficult to collect sign language corpus, and it is necessary to be familiar with standard sign language model demonstration, professional photography and view processing; (2) the data quality of sign language corpus is generally not high. Need to collect many times, strict comparison and optimization 3. Lack of linguistic and grammatical analysis system of sign language, and lack of automatic tagging intelligent model and diversity of sign language expression, which requires a large number of manual tagging and continuous correction iteration; 4. The accuracy of single-channel sign language recognition is low, and multi-channel sign language data fusion is difficult. Sign language view needs to be combined with multi-modal channels such as facial expressions, action amplitude, surrounding environment and so on.

In order to solve the above problems, Tianjin disabled Persons' Federation and Tianjin University of Technology together with Huawei TECH4ALL Digital inclusion team, relying on the surging AI computing power provided by Tianjin artificial Intelligence Computing Center, and based on Huawei Penteng AI basic software and hardware platform, especially the full scene AI framework Jisi MindSpore, jointly developed the "Haihe er language" sign language understanding model for the deaf, and worked together to create more accurate sign language-Chinese translation products.

Photo: yuan Tiantian introduces the advantages and application scenarios of the sign language model

The big model of Haihe er language is the first large model of autonomous and controllable deaf sign language understanding in China, which is based on the deaf sign language view, combined with action amplitude, speed, character expression and surrounding environment to deeply and accurately understand the meaning of deaf sign language expression. and converted into sound and text that can be understood by able-bodied people, which is unique in China. More than one million sign language data sets will be "fed" to the "Haihe er language" large model. After model and algorithm processing, we can efficiently generate a sign language multimodal model, which can support more than 1 billion of the data tagging sign language to Chinese translation and training.

In the future, the research results of the large model of sign language understanding for the deaf in Haihe er language are expected to move towards a broader social category, which will be applied to many fields such as smart city, wisdom education, wisdom, intelligent pension, intelligent travel and public welfare projects.

Break down communication barriers and make life warmer. Technology has never stopped exploring. People with hearing impairment are an important part of our society that can not be ignored. Accurate and timely access to the expression of people with hearing impairment and transmitting information to more people with hearing impairment and healthy people is an important demand for the construction of a harmonious society and the development of ecological civilization in our country. Huawei TECH4ALL Digital inclusion team will continue to work with more partners to help people with hearing impairment integrate into society more easily and make the digital world more equal and sustainable.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.