Shulou(Shulou.com)11/24 Report--

CTOnews.com, June 13 (Xinhua)-- it is well known that the United Arab Emirates has been spending a lot of money on science, technology and education. Recently, the United Arab Emirates Institute of Science, Technology and Innovation (TII) created a 40 billion-parameter language model, Falcon-40B, which ranks first in Hugging Face's OpenLLM ranking and continues to dominate the list, beating previous competitors such as LLaMA, StableLM and RedPajama.

According to ▲, the United Arab Emirates Institute of Science, Technology and Innovation (TII) said Falcon-40B used 384 GPU for two months of training and nearly 500 million Token training. In order to improve the quality of the training process, the team also set up a set of workflow to select the "highest quality data" for AI model training. Through several benchmark tests, it outperformed LLaMA, Vicuna and Alpaca.

In addition, the team also uses the "multiple query attention" (Multiqery attention) mechanism to improve the efficiency of the model, that is, the model can perform multiple queries on each Token to better represent the relationship between different token in the same sequence, so as to accelerate the computational efficiency of the model, reduce the complexity of the model, and improve the maintainability of the model as a whole.

Dr Ray O. Johnson, CEO of the UAE Institute of Science, Technology and Innovation, said: "2023 will be the year of artificial intelligence. The Falcon model is a milestone for us, but this is just the beginning."

CTOnews.com noted that the Falcon model is already open source on huggingface, and the current open source versions are Falcon-7B-Instruct and Falcon-40B-Instruct, which can be accessed by CTOnews.com friends here.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.