Ray on EKS Deep Dive
Review baseline: Ray 2.58.0, KubeRay v1.7.1 Latest release checked: Ray 2.59.0 (October 2, 2026), checked October 16, 2026; examples use 2.58.0 Documentation reviewed: October 16, 2026
Overview
Ray distributes Python work using tasks, actors, ObjectRefs, and per-node object stores. Train, Tune, and Serve use that foundation while adding training, search, and serving policies. One object-store path does not automatically handle every communication or recovery concern.
KubeRay is the Kubernetes operator reconciling RayCluster, RayJob, and RayService. It does not select the application's ML library as a dispatcher. Ray work scheduling, Kubernetes Pod placement, and EC2 node provisioning are separate layers.
Component Map
| Concept | Problem It Solves | Deep Dive |
|---|---|---|
| Architecture | Tasks, actors, and the object store that everything else builds on | Part 1 |
| KubeRay Operator | Run Ray clusters as native Kubernetes resources (RayCluster/RayJob/RayService) | Part 2 |
| Ray Train & Tune | Distributed model training and hyperparameter search | Part 3 |
| Ray Serve | Model serving, including dedicated LLM-serving building blocks | Part 4 |
Why Run This on EKS
The trade-off is the same one covered elsewhere in this documentation site's data/ML sections: a team already running EKS can reuse the same node-pool autoscaling (via Karpenter), IAM, and observability patterns for Ray workloads as for everything else on the cluster, in exchange for operating the KubeRay operator and its RayCluster/RayJob/RayService resources directly rather than using a managed alternative.
The foundation check is a small single-node Ray run. It is not evidence of GPU training, multi-node recovery, a live EKS installation, or autoscaling.
Ray 2.59.0 enables token authentication by default for local clusters. Calling ray.init() without an address generates a token when ~/.ray/auth_token does not exist; RAY_AUTH_MODE=disabled turns it off. The release notes say the default is planned to extend to remote and multi-node clusters starting with 2.61. This series' examples were checked on 2.58.0 and were not rerun on 2.59.0.
Currently Covered
- Part 1: Ray Architecture: tasks, actors, the object store, and the head/worker cluster model
- Part 2: The KubeRay Operator: RayCluster, RayJob, RayService, and the two-tier autoscaling pattern with Karpenter
- Part 3: Ray Train and Ray Tune: distributed training and hyperparameter tuning
- Part 4: Ray Serve: model serving, Ray Serve LLM, and RayService-based production deployment