Shulou(Shulou.com)11/24 Report--

IT House June 8 news, the domestic multimodal large language model TigerBot has been officially released recently, including 7 billion parameters and 180 billion parameters of two versions, has been open source in GitHub.

On the GitHub page of ▲ source TigerBot, it is reported that the main innovations brought about by TigerBot are:

An innovative algorithm of instruction completion supervised fine tuning is proposed to improve the learnability of the model.

Using ensemble and probabilistic modeling methods to achieve controllable factuality and creativity

Break through the memory and communication problems in mainstream frameworks such as deep-speed in parallel training

In addition, the model also makes a more suitable optimization from tokenizer to training algorithm for the more irregular distribution of Chinese language.

Researcher Chen Ye said on the official website of Hubo Science and Technology: "this model can quickly understand what kind of problems human beings are asking with only a small number of parameters. According to the automatic evaluation of OpenAI InstructGPT papers on public NLP data sets, TigerBot-7B has reached 96% of the overall performance of OpenAI models of the same size."

The GitHub page of the ▲ map source TigerBot is said to be "better than the OpenAI equivalent comparable model". The open source code includes basic training and inference code, quantification and inference code of the double-card reasoning 180B model. The data includes 100G pre-training data and supervised fine-tuning 1G or 1 million pieces of data.

CTOnews.com friends can find GitHub open source projects here.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.