Shulou(Shulou.com)06/02 Report--

This article mainly explains "Why to use Go to write the infrastructure of machine learning instead of Python". The content in the article is simple and clear, and it is easy to learn and understand. Please follow the editor's train of thought to study and learn "Why write the infrastructure of machine learning with Go instead of Python".

There is no doubt that Python is the most popular language in machine learning programs.

Although programming languages such as R, C++, and Julia have their own unique user groups and use cases that make them enduring, Python is still the most widely used language and is used in every major machine learning framework. However, can you imagine? In the Cortex code base, which deploys the machine learning model as one of the open source platforms of API, 87.5% of the code is written in GO.

Image source: Cortex GitHub

The machine learning algorithm that Python boasts is only one part of the machine learning system in the production environment. To actually run machine learning API in a production environment on a large scale, your infrastructure needs to implement the following functions:

Automatic scaling to ensure that traffic fluctuations do not break the API

Implement API management to handle API synchronous deployment

Rolling updates to ensure that updating the model does not disrupt user service

Cortex aims to automate all of the above infrastructure, as well as other issues such as logging and cost optimization.

Go is ideal for building software that meets the above requirements for the following reasons:

1. Go ecosystem is very suitable for infrastructure projects.

Source: zcool

One of the benefits of open source is that you can keep learning from projects you appreciate. For example, Cortex exists in the Kubernetes (itself written in Go) ecosystem. Fortunately, we were able to refer to many excellent open source projects in this ecosystem, including:

Kubectl: Kubernetes CLI

Minikube: a tool for running Kubernetes locally

Helm: Kubernetes package management tool

Kops: a tool for managing production Kubernetes

Eksctl: the official CLI of Amazon EKS

All of the above tools are not only Kubernetes projects, but also written in the GE language. Whether you look at CockroachDB or Hashicorp infrastructure projects (including Vault, Nomad, Terraform, Consul, and Packer), you will find that they are all written in the Goe language.

Another impact of Go's popularity in infrastructure is that most engineers interested in infrastructure are familiar with the Go language. So it's easy to attract these engineers through GO. In fact, one of Cortex Labs's best engineers found us by searching for Go positions on AngelList. We feel very lucky that he found us.

two。 Concurrency is critical to the infrastructure of machine learning

Users can deploy many different models to different API and put all models in the same Cortex cluster for management. Cortex Operator also needs some API to manage these different deployments. For example:

Cortex calls Kubernetes API to deploy the model on the cluster.

Cortex invokes various AWS API (including EC2 Auto Scaling,S3,CloudWatch, etc.) to manage deployment on AWS.

Users do not interact directly with these API. Instead, Cortex programmatically calls these API to provide clustering, start deployment, and monitor API.

Cross-calling API in an efficient and reliable way is a major challenge. Using API in a concurrent manner is the most efficient approach, but it also causes problems, because now we have to worry about things like race conditions.

Go provides a simple solution to this problem: Goroutines.

Goroutines is a common function for Go to execute code concurrently. We can write another long article to delve into how Goroutines works, but in a nutshell, Goroutines is a lightweight thread automatically managed by the Go runtime. Multiple Goroutine can be placed on one OS thread, and if the Goroutine blocks the OS thread, the Go runtime automatically moves the rest of the Goroutine to the new OS thread.

Goroutines also provides a feature called "channel", which allows users to pass messages between Goroutines to help users schedule requests and prevent race conditions.

Of course, users can also use the latest tools such as asyncio to implement these functions in Python, but Go is designed with these situations in mind, and the use of GO can reduce the burden of our work and life.

3. Programming in the go language is a pleasure.

The last reason we use Go to build Cortex is that the Go language is really good.

It's a little painful compared to the introduction to Python,Go. However, the inclusiveness of Go brings comfort to large projects. We are still strictly testing the software, but static typing and compilation (two big problems for Go beginners) act as our guardrail, helping us to write (relatively speaking) error-free code.

You might say that other languages also offer specific advantages, but overall, Go maximizes our technical and aesthetic needs.

4. It is easier to build a cross-platform CLI using Go

Cortex CLI is a cross-platform tool that allows users to deploy the model and manage API directly from the command line. The following dynamic diagram shows the running CLI:

Initially, we wrote CLI in Python, but it turned out to be too difficult to distribute CLI across platforms. On the other hand, Go can compile binaries (without relying on management), which provides us with a convenient and easy way to distribute CLI across platforms without additional engineering work.

Binaries compiled by Go have obvious performance advantages over interpreted languages. According to the results of computer benchmark tests, GO is much faster than Python.

It may be no accident that many other infrastructure CLI tools are also written in GO.

Python is suitable for machine learning, and Go is suitable for infrastructure. We also like Python, which has a place in Cortex, especially in dealing with reasoning.

Cortex serves TensorFlow,PyTorch,scikit-learn and other Python models, which means that the interface with the model and the processing before and after reasoning are done in Python. However, even if the Python code is packaged and deployed into Docker containers, these containers are carefully orchestrated by code written in Go.

Thank you for your reading, the above is the content of "Why to write the infrastructure of machine learning with Go instead of Python". After the study of this article, I believe you have a deeper understanding of why to write the infrastructure of machine learning with Go instead of Python, and the specific use needs to be verified in practice. Here is, the editor will push for you more related knowledge points of the article, welcome to follow!

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.