How to Deploy a Ray Cluster on Bare-Metal Servers

This guide will show you how to connect and run a Ray cluster on your own dedicated physical servers using the Ray Cluster Launcher.

Understanding Ray and Distributed Computing

Linux

What is a Ray Cluster?

A Ray cluster is a group of computers working together as one giant machine. When you have a massive task, like training a complex Artificial Intelligence model or processing huge amounts of data, a single computer might not be fast enough. Ray solves this by splitting the heavy workload into smaller pieces and sharing them across multiple computers in your network.

In this system, there is always one "head node" and several "worker nodes." The head node acts as the manager, receiving your code and deciding which worker should handle which part of the task. The worker nodes do the actual heavy lifting and report their results back to the head node. This setup makes sure all your computer power is used efficiently without you having to manually manage every single machine.

By putting a Ray cluster on "bare-metal" servers, you are installing it directly onto physical hardware instead of using cloud machines. This gives you maximum speed, complete control over your hardware, and avoids unexpected cloud computing bills. The Ray Cluster Launcher makes this process easy by using a single configuration file to connect all your machines automatically.

Prerequisites

Supported Operating System: All servers must run Linux (x86_64 or ARM/aarch64). While Ray supports macOS and Windows (in beta) for local development, multi-node Ray clusters are officially only supported on Linux.
Networked Servers: Multiple physical servers connected to the same secure, private local network.
SSH Access: Passwordless SSH access to all servers using a dedicated, cluster-specific private key (do not use your personal key).
Matching Python: The exact same Python version must be installed on your local computer, the head node, and all worker nodes.
Security: A strong firewall protecting your network from the outside, while allowing internal communication between the servers on specific ports (like TCP 6379, 8265, and 10001). This is critical because Ray allows remote code execution and does not have built-in passwords.

Step-by-Step Installation

Install Ray on Your Local Computer

Open the terminal on your local computer (the machine you are using right now) and install Ray. This gives you the tools needed to start the cluster.

BASH

pip install -U "ray[default]"

Generate the Setup File

You need a configuration file to tell Ray where your servers are. Copy the built-in template to your current folder. This is safer than downloading it from the internet because it matches your exact Ray version.

BASH

python -c "import ray, os, shutil; shutil.copy(os.path.join(os.path.dirname(ray.__file__), 'autoscaler/local/example-full.yaml'), 'my-cluster.yaml')"

Edit the Configuration File

Open the my-cluster.yaml file in a text editor. You need to put in your server IP addresses, your dedicated SSH key, and the number of worker machines. Here is what your file should look like:

YAML

# Name of your cluster
cluster_name: my-bare-metal-cluster

# Tell Ray exactly how many worker nodes to start
min_workers: 2
max_workers: 2

provider:
    type: local
    head_ip: 10.0.0.100          # The IP address of your main manager server
    worker_ips: [10.0.0.101, 10.0.0.102] # The IP addresses of your worker servers

auth:
    ssh_user: ubuntu             # The username used to log into the servers
    # Use a dedicated SSH key for safety
    ssh_private_key: ~/.ssh/ray_cluster_key

# Tell Ray to install itself on all servers when they turn on
setup_commands:
    - pip install -U "ray[default]"

Launch the Cluster

Run this command from your local computer. The Ray launcher will automatically copy your SSH key to the head node and start the cluster processes on all machines.

Bash

ray up my-cluster.yaml

Connect and Test

Log into the head node to run your code and check if all machines are working together. Connect to the head node using this command:

Bash

ray attach my-cluster.yaml

Once connected, run this Python script to see all the combined power of your servers:

Python

python -c '
import ray
# Connect to the existing cluster
ray.init(address="auto")

# Print the total CPUs and resources from all connected servers
print("Cluster Resources:", ray.cluster_resources())
'

Type exit when you want to leave the head node.

Stop the Cluster

When you are completely finished with your work, you can safely turn off Ray on all your bare-metal servers with one command.

Bash

ray down my-cluster.yaml

You have now successfully built a powerful Ray cluster on your own physical servers. Through this guide, you have learned how to:

Install the Ray software across all of your bare-metal machines.
Start the main manager process on your head node.
Connect your worker nodes to the head node to share the computing workload.
Test the connection to ensure all servers are working together as one unified system.

With your bare-metal cluster fully operational, you are ready to tackle massive computing tasks and advanced AI workloads on your own hardware.

Explore GPU Dedicated Servers

CTCservers Recommended Tutorials

Discover CTCservers Dedicated Server Locations

CTCservers servers are available around the world, providing diverse options for hosting websites. Each region offers unique advantages, making it easier to choose a location that best suits your specific hosting needs.

North America

South America

Europe

Asia

Australia

Africa