Add SGLang Ray inference example#40

Merged

robertnishihara merged 33 commits intomainfrom

Mar 18, 2026

Commits on Feb 11, 2026

Commits on Feb 22, 2026

Update sglang_ray_inference to use released packages and improve robustness

xyuzh
and
claude
committed
Revert Dockerfile to use sglang from feature branch fork

xyuzh
and
claude
committed
Refactor SGLang example: batch inference works, serve in progress

robertnishihara
and
claude
committed

Commits on Mar 6, 2026

Fix async_generate hang by setting stream=False

robertnishihara
and
claude
committed
Rename NUM_NODES to NUM_NODES_PER_REPLICA for clarity

robertnishihara
and
claude
committed
Remove unnecessary WORKDIR from Dockerfile

robertnishihara
and
claude
committed
Use main SGLang repo instead of fork

robertnishihara
and
claude
committed
Fix min_nodes to match autoscaling min_replicas

robertnishihara
and
claude
committed
Remove main() wrapper for easier copy/paste

robertnishihara
and
claude
committed
Use declarative compute configs instead of hardcoded instance types

robertnishihara
and
claude
committed
Fix job compute config to match actual GPU usage

robertnishihara
and
claude
committed
Fix declarative compute configs with correct syntax

robertnishihara
and
claude
committed
Use declarative compute config for head node

robertnishihara
and
claude
committed
Remove module docstrings

robertnishihara
and
claude
committed
Fix memory requirement for declarative compute config

robertnishihara
and
claude
committed
Update SGLang multi-node inference to use RayEngine API

robertnishihara
and
claude
committed

Commits on Mar 7, 2026

Commits on Mar 18, 2026