In addition to Weibo, there is also WeChat
Please pay attention
WeChat public account
Shulou
2025-03-29 Update From: SLTechnology News&Howtos shulou NAV: SLTechnology News&Howtos > IT Information >
Share
Shulou(Shulou.com)11/24 Report--
CTOnews.com June 25 news, AI startup MosaicML recently released its language model MPT-30B, this model has 30 billion parameters, training cost "only a fraction of other similar competitor models," is expected to promote the industry to gradually reduce the training cost of such models, expand the application of AI models in a wider range of fields.
Naveen Rao, CEO and co-founder of MosaicML, said the MPT-30B costs $700,000 to train, far less than the tens of millions of dollars needed for similar products such as GPT-3. In addition, due to its lower cost and smaller size, the MPT-30B can also be trained more quickly and is more suitable for deployment on local hardware.
MosaicML uses Alibi and FlashAttention technologies to optimize models for longer text lengths and higher utilization of GPU computing. MosaicML is also one of the few labs to be able to use Nvidia H100 GPUs, with throughput per GPU increased by more than 2.4 times compared to previous efforts, resulting in faster completion times.
In addition to making AI technology more accessible, MosaicML focuses on improving data quality and improving model performance. They are currently developing a tool that will help users layer in domain-specific data during pre-training to ensure diverse and high-quality data combinations during training.
MosaicML says expanding the model to 30 billion parameters is only the first step, and they will then launch larger, higher-quality models with lower costs.
CTOnews.com notes that developers can now download and use the open-source MPT-30B base model from Hugging Face and fine-tune the model using their own data on local hardware.
Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.
Views: 0
*The comments in the above article only represent the author's personal views and do not represent the views and positions of this website. If you have more insights, please feel free to contribute and share.
Continue with the installation of the previous hadoop.First, install zookooper1. Decompress zookoope
"Every 5-10 years, there's a rare product, a really special, very unusual product that's the most un
© 2024 shulou.com SLNews company. All rights reserved.