Shulou(Shulou.com)06/01 Report--

This article mainly introduces the relevant knowledge of "how to deploy Ray clusters under Linux". The editor shows you the operation process through an actual case. The operation method is simple, fast and practical. I hope this article "how to deploy Ray clusters under Linux" can help you solve the problem.

Ray is a new high-performance distributed execution framework introduced by UC Berkeley RISE Lab. It uses an architecture different from traditional distributed computing systems and abstracts distributed computing. It has better computing performance than Spark. Ray is a distributed execution engine based on Python. The same code can be run on a single machine for efficient multiprocessing and can be used for a large number of calculations on the cluster.

Anaconda

For convenience, we'd better use Anaconda to build a separate python runtime environment. (of course, you can also directly use the python environment in the system, so you can skip this step.)

# download the installation script $wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh# add execute permissions $chmod Ubunx Miniconda3-latest-Linux-x86_64.sh# run the installation script $. / Miniconda3-latest-Linux-x86_64.sh install python

Prepare a python environment for ray, using python3.8.8 as an example:

# create a python environment named ray, version 3.8.8$ conda create-- name ray python=3.8.8# activates a python environment named ray $conda activate ray

After installation, it is best to log in again, or execute source ~ / .bashrc to make the environment variables take effect

Install ray

Install ray (version 1.7.0). Here, Ali's image source is specified to speed up:

$pip install-I https://mirrors.aliyun.com/pypi/simple-- trusted-host mirrors.aliyun.com-U 'ray [default]' = 1.7.0 start the head node

Start the Head node on 192.168.100.1:

$ray start-head-dashboard-host='0.0.0.0'-dashboard-port=8265

Normally, you will see the following output:

Local node IP: 192.168.100.12021-11-02 1833 INFO services.py:1250 INFO services.py:1250-- View the Ray dashboard at http://192.168.100.1:8265--------------------Ray runtime started.-Next steps To connect to this Ray runtime from another node, run ray start-- address='192.168.100.1:6379'-- redis-password='5241590000000000'...

The output contains two key pieces of information, which need to be noted:

Address of View the Ray dashboard at http://192.168.100.1:8265 web service ray start-- address='192.168.100.1:6379'-- address and password of redis-password='524159000':head to start the worker node

On 192.168.100.2, follow the steps above to install python and ray, noting that their versions must be consistent. (in addition, Worker is not required because the Head node itself has the worker role)

# Connect the specified Head address $ray start-- address='192.168.100.1:6379'-- redis-password='5241590000000000'Dashboad

Visit dashboard: http://192.168.100.1:8265

This is the end of the introduction on "how to deploy Ray clusters under Linux". Thank you for reading. If you want to know more about the industry, you can follow the industry information channel. The editor will update different knowledge points for you every day.

Welcome to subscribe "Shulou Technology Information " to get latest news, interesting things and hot topics in the IT industry, and controls the hottest and latest Internet news, technology news and IT industry trends.