Shulou(Shulou.com)11/24 Report--

CTOnews.com, May 25 (Xinhua)-- researchers at the Barcelona Supercomputing Center (BSC) and the Catalan University of Technology (UPC) have developed a new tool for translating sign language into text to give people with hearing impairment more inclusiveness and accessibility. The tool uses artificial intelligence technologies, including self-attention mechanisms and faster throughput, to more accurately understand the gestures of sign speakers and produce accurate text transcription.

▲ Image Source Arxiv this tool is based on the How2Sign dataset, a publicly available large-scale, multimodal and multi-view dataset, including 80 hours of ASL instructional videos and corresponding English text records. This dataset is published by BSC and UPC and is used to train the model to convert sign language into text. Researchers use expansive three-dimensional network (I3D) to preprocess the data, which is a video extraction method, which can filter the video in three dimensions and obtain spatio-temporal information directly from the video. A converter machine learning model is used, which is similar to the model behind other artificial intelligence tools such as ChatGPT.

Laia Tarr é s, lead author of the ▲ source paper "Sign Language Translation from Instructional Videos", said: "the new tool we developed is an extension of a previously released publication called How2Sign, where the data needed to train the model is published. using this already available data, we have developed a new open source software that can learn the mapping between video and text."

▲ source Pexels researchers have found that their model can produce meaningful translation, but there is still room for improvement. The tool is still in the experimental stage, and researchers will continue to work to create a tool comparable to the technology used by people without hearing loss.

Laia Tarr é s also said: "this open tool for automatic sign language translation is a valuable contribution to the scientific community concerned about the accessible environment, and its publication represents an important step towards creating more inclusive and accessible technologies for all."

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.