feat: Add Google Cloud Vertex AI provider support

[itdove]

Summary

Adds complete support for Google Cloud Vertex AI as an inference provider, enabling OpenShell sandboxes to use Claude models via GCP Vertex AI with OAuth authentication.

This implementation includes full end-to-end testing and supports both direct Claude CLI usage and inference routing via inference.local.

Features

Vertex AI Provider

• Provider discovery: Auto-discovers Vertex AI credentials from environment
• OAuth token generation: Generates tokens from GCP Application Default Credentials
• Credential injection: Injects actual values (not placeholders) for CLI tool compatibility
• Region support: Configurable region via ANTHROPIC_VERTEX_REGION (defaults to us-central1)
• Auto-configuration: Sets CLAUDE_CODE_USE_VERTEX=1 automatically

Inference Routing

• URL construction: Builds Vertex-specific URLs with project ID, region, and :streamRawPredict endpoint
• Model field handling: Removes model from request body (Vertex expects it in URL path)
• Bearer auth: Uses OAuth tokens as Bearer tokens
• API version: Uses vertex-2023-10-16 Anthropic API version
• Model ID format: Supports @ separator (e.g., claude-sonnet-4-5@20250929)

Direct Credential Injection

• Selective injection: Credentials needed by CLI tools are injected as actual environment variables
• Vertex credentials: ANTHROPIC_VERTEX_PROJECT_ID, VERTEX_OAUTH_TOKEN, CLAUDE_CODE_USE_VERTEX, ANTHROPIC_VERTEX_REGION
• Security: Only credentials essential for CLI tool compatibility are directly injected
• HTTP proxy resolution: Other credentials continue using openshell:resolve:env:* placeholders

Network Policy Support

• Custom policies: Sandboxes require network policy allowing Google Cloud endpoints
• OAuth endpoints: oauth2.googleapis.com, accounts.google.com
• Vertex AI endpoints: Regional Vertex AI endpoints (*-aiplatform.googleapis.com)
• Inference routing: inference.local endpoint for privacy-aware routing

Changes

Core Implementation

• crates/openshell-providers/src/providers/vertex.rs - Vertex AI provider plugin with OAuth generation
• crates/openshell-core/src/inference.rs - VERTEX_PROFILE with Bearer auth and vertex API version
• crates/openshell-server/src/inference.rs - Vertex URL construction with project ID and region
• crates/openshell-router/src/backend.rs - Critical fix: Removes model field from request body for Vertex AI
• crates/openshell-sandbox/src/secrets.rs - Direct credential injection for CLI compatibility
• crates/openshell-providers/Cargo.toml - Add gcp_auth dependency
• crates/openshell-providers/src/lib.rs - Register vertex provider
• crates/openshell-cli/src/main.rs - Add Vertex to provider type enum

Examples

• examples/vertex-ai/sandbox-policy.yaml - New: Network policy for Vertex AI endpoints
• examples/vertex-ai/README.md - New: Quick start guide with documentation references

Development Improvements

• tasks/scripts/cluster-deploy-fast.sh - Bash 3 compatibility fix (replaces mapfile)
• scripts/rebuild-cluster.sh - New: Quick rebuild script for development workflow
• scripts/setup-podman-macos.sh - Increase default memory from 8 GB to 12 GB for better build performance
• cleanup-openshell-podman-macos.sh - Improved cleanup with sandbox deletion

Documentation

• docs/sandboxes/manage-providers.md - Updated Vertex provider documentation, removed OAuth limitation note
• docs/inference/configure.md - Updated Vertex AI setup guide with OAuth token generation
• docs/get-started/install-podman-macos.md - Added rebuild/cleanup workflow documentation
• CONTRIBUTING.md - Added development rebuild workflow

Usage

Prerequisites

# Install Google Cloud SDK
brew install google-cloud-sdk

# Configure Application Default Credentials
gcloud auth application-default login

# Set project ID and region
export ANTHROPIC_VERTEX_PROJECT_ID=your-gcp-project-id
export ANTHROPIC_VERTEX_REGION=us-east5  # Optional, defaults to us-central1

Quick Start

# Create provider
openshell provider create --name vertex --type vertex --from-existing

# Create sandbox with Vertex AI
openshell sandbox create --name vertex-test --provider vertex \
  --upload ~/.config/gcloud/:.config/gcloud/ \
  --policy examples/vertex-ai/sandbox-policy.yaml

# Use Claude CLI (automatically uses Vertex AI)
claude

Inference Routing (Optional)

# Configure inference routing
openshell inference set --provider vertex --model claude-sonnet-4-5@20250929 --no-verify

# Test inside sandbox
curl -X POST https://inference.local/v1/messages \
  -H "content-type: application/json" \
  -d '{
    "anthropic_version": "vertex-2023-10-16",
    "max_tokens": 32,
    "messages": [{"role": "user", "content": "Say hello"}]
  }'

Testing

✅ Fully tested end-to-end on macOS with:

• Podman Machine (12 GB RAM)
• GCP project with Vertex AI Claude models enabled
• Application Default Credentials configured
• Provider creation and OAuth token generation
• Direct Claude CLI usage in sandboxes
• Inference routing via inference.local
• Network policy enforcement
• All regional Vertex AI endpoints

Key Test Results:

• ✅ OAuth token generation from ADC
• ✅ Credential injection into sandboxes
• ✅ Claude CLI auto-detects Vertex AI
• ✅ Inference routing removes model field correctly
• ✅ Vertex API responds with successful completions

Technical Details

Router Fix (Critical)

The router was incorrectly inserting the model field into request bodies for all providers. Vertex AI's :streamRawPredict endpoint expects the model in the URL path, not the request body, causing "Extra inputs are not permitted" errors.

Fix: Router now detects Vertex AI endpoints (aiplatform.googleapis.com) and removes the model field from the request body while keeping it in the URL path.

Credential Flow

1. User configures GCP Application Default Credentials
2. Provider plugin generates OAuth token from ADC at creation time
3. Credentials are stored in gateway database
4. When creating sandboxes, credentials are injected as actual environment variables
5. CLI tools (claude) automatically detect Vertex AI via CLAUDE_CODE_USE_VERTEX=1
6. OAuth tokens are refreshed from the uploaded ~/.config/gcloud/ directory

URL Structure

Vertex AI requests go to:

https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/publishers/anthropic/models/{model}:streamRawPredict

The router constructs this URL and removes the model field from the JSON body.

Development Workflow

Rebuilding After Changes

# Quick rebuild for testing code changes
bash scripts/rebuild-cluster.sh

# Recreate provider and sandbox after rebuild
openshell provider create --name vertex --type vertex --from-existing
openshell sandbox create --name test --provider vertex \
  --upload ~/.config/gcloud/:.config/gcloud/ \
  --policy examples/vertex-ai/sandbox-policy.yaml

Breaking Changes

None. All changes are additive.

Related Issues

Addresses the need for Vertex AI provider support for users who:

• Need to use Claude via GCP Vertex AI for billing/compliance
• Want to use existing GCP credentials and infrastructure
• Require OAuth-based authentication instead of API keys
• Work in organizations with GCP-only AI policies

Checklist

• Code follows project style guidelines
• Tests pass (cargo check and cargo test succeed)
• End-to-end testing completed successfully
• Documentation updated
• Example policy and README added
• Commit messages follow Conventional Commits format
• No secrets or credentials committed
• Router fix verified with live Vertex AI endpoints
• Network policy tested and working
• OAuth token generation tested

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 267788ed-44a5-4208-9b9a-d84db885ddc7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}