Shulou(Shulou.com)11/24 Report--

Senior developer boss, finally can not help but come out to start a business.

Georgi Gerganov, which opened up the llama.cpp project in March this year, has broken 30, 000 stars on GitHub, and Stable Diffusion is only 8.8k.

This project allows developers to run Meta's LLaMA model without GPU, even on raspberry pie and MacBook.

△ running 7B LLaMA at 40 tok / s on M2 Max even managed to attract Xiaoza's attention: Meta is also running llama.cpp.

Perhaps because the response was too good, the big brother decided to start his own business with the core pure C language framework ggml: it was originally a sideline project a few months ago.

Even before the announcement, the company had already received pre-seed investment from Daniel Gross, a former CEONat Friedman and Y Combinator partner of GitHub.

As soon as the news came out, many developers came to congratulate.

There are some staunch advocates: ggml is popularizing large models to edge devices.

It wasn't long before someone suggested that Apple should buy it. (dog head)

The author of llama.cpp started ggml, a tensor library written in pure C language that helps developers run large models on consumer hardware with a GitHub star count of 4.4k.

Due to the amazing effect of acceleration, it has suddenly gained the support of many developers.

By the way,ggml 's gg happens to be the abbreviation of his name.

Brother's own two tens of thousands of star projects llama.cpp and whisper.cpp both use it.

The latter is an accelerated solution developed for OpenAI's Whisper automatic speech recognition model and can be run on Mac, Windows, Linux, iOS, Android, raspberry pie and web.

△ uses whisper.cpp to detect short voice commands on raspberry pie. Many startups, such as rewind, the main life search engine, use this solution.

There are also two projects running on the terminal at the same time.

△ runs four 13B LLaMA+Whisper Small instances simultaneously on a single M1Pro. According to my introduction, the ggml tensor library has the following characteristics:

Support for 16bit floating-point numbers; support for integer quantization (including 4-bit, 5-bit, 8-bit); automatic differentiation; built-in optimization algorithms (such as ADAM, L-BFGS); set specific optimizations for Apple chips; use AVX / AVX2 Intrinsic; on x86 architecture to provide Web support through WebAssembly and WASM SIMD; no third-party dependencies; run-time zero memory allocation; support guided language output.

At present, this library and related projects are free and open source, and the development process is fully made public; of course, it does not rule out the possibility that development is licensed to some commercial projects.

The neural network code is rewritten in C / C++ and the developer behind it, Georgi Gerganov, is also worth talking about.

His personal website is very simple and straightforward, throwing out all kinds of open source projects and nothing else. It can be seen that he is a big fan of C / C++ and believes in Vim.

Previously, he has improved efficiency by rewriting neural network inference code in C / C++ language, which is almost independent of other libraries. As for llama.cpp, he also came out Hacking in one night.

Besides, he has some interesting projects.

For example, check whether the keyboard can eavesdrop through the microphone, guess the title of Hacker News, Wordle clone, and so on.

It is worth mentioning that the two investors behind One More Thing are also interesting.

They also specialize in providing computing clusters for entrepreneurs by applying on the website. This wave is on Next Level.

Reference link:

[1] https://ggerganov.com/

[2] http://ggml.ai/

[3] https://twitter.com/ggerganov

This article comes from the official account of Wechat: quantum bit (ID:QbitAI), author: Yang Jing

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.