Skip to content

Commit 86ac7a8

Browse files
committed
Tighten benchmark guardrail ratio to 0.05
1 parent 9770c7c commit 86ac7a8

5 files changed

Lines changed: 9 additions & 9 deletions

File tree

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ jobs:
335335

336336
env:
337337
BENCH_RUNS: "1"
338-
BENCH_RUST_TO_PURE_MAX_RATIO: "0.10"
338+
BENCH_RUST_TO_PURE_MAX_RATIO: "0.05"
339339

340340
steps:
341341
- uses: actions/checkout@v4

bin/check-rust-benchmark

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ def usage
1616
BENCH_TOPICS Number of topics (default: 8)
1717
BENCH_MAX_ITER Variational max iterations (default: 20)
1818
BENCH_EM_MAX_ITER EM max iterations (default: 40)
19-
BENCH_RUST_TO_PURE_MAX_RATIO Maximum allowed rust_mean/pure_mean ratio (default: 0.10)
19+
BENCH_RUST_TO_PURE_MAX_RATIO Maximum allowed rust_mean/pure_mean ratio (default: 0.05)
2020
USAGE
2121
end
2222

@@ -40,7 +40,7 @@ benchmark_env = {
4040
"BENCH_MAX_ITER" => ENV.fetch("BENCH_MAX_ITER", "20"),
4141
"BENCH_EM_MAX_ITER" => ENV.fetch("BENCH_EM_MAX_ITER", "40")
4242
}
43-
threshold = Float(ENV.fetch("BENCH_RUST_TO_PURE_MAX_RATIO", "0.10"))
43+
threshold = Float(ENV.fetch("BENCH_RUST_TO_PURE_MAX_RATIO", "0.05"))
4444

4545
stdout, stderr, status = Open3.capture3(benchmark_env, *command, chdir: repo_root)
4646
unless status.success?

docs/modernization-handoff.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -179,7 +179,7 @@ Performance tracking:
179179
- `./bin/benchmark-backends`
180180
- `./bin/check-rust-benchmark`
181181
- `docs/rust-orchestration-guardrails.md`
182-
- CI currently enforces `BENCH_RUST_TO_PURE_MAX_RATIO=0.10` in `benchmark-guardrail`
182+
- CI currently enforces `BENCH_RUST_TO_PURE_MAX_RATIO=0.05` in `benchmark-guardrail`
183183

184184
## CI Jobs Expected
185185

docs/porting-strategy.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ Completed in `codex/experiment-ruby3-modernization`:
6868
- Gem packaging now excludes local Rust cargo build artifacts (`target/**`) for clean release builds.
6969
- Backend benchmark driver added (`bin/benchmark-backends`) to track pure/native/rust runtime deltas.
7070
- Rust orchestration guardrail policy documented (`docs/rust-orchestration-guardrails.md`) with benchmark threshold checker (`bin/check-rust-benchmark`).
71-
- CI benchmark guardrail job added (`benchmark-guardrail`) to enforce Rust/pure runtime ratio on Ubuntu (currently `BENCH_RUST_TO_PURE_MAX_RATIO=0.10`).
71+
- CI benchmark guardrail job added (`benchmark-guardrail`) to enforce Rust/pure runtime ratio on Ubuntu (currently `BENCH_RUST_TO_PURE_MAX_RATIO=0.05`).
7272
- Source install path now has explicit Rust build policy via `LDA_RUBY_RUST_BUILD=auto|always|never`.
7373
- Docker install-policy matrix script added (`bin/docker-test-install-policies`) to verify source install behavior across environments.
7474
- CI now runs install-policy matrix checks on Ubuntu.

docs/rust-orchestration-guardrails.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,10 @@ Run:
2525

2626
Default benchmark policy:
2727

28-
- `BENCH_RUST_TO_PURE_MAX_RATIO=0.10`
29-
- i.e., Rust mean runtime must be no worse than 10% of pure mean runtime on the benchmark fixture/config.
28+
- `BENCH_RUST_TO_PURE_MAX_RATIO=0.05`
29+
- i.e., Rust mean runtime must be no worse than 5% of pure mean runtime on the benchmark fixture/config.
3030
- CI benchmark guardrail job enforces the same ratio with `BENCH_RUNS=1` for runtime stability.
31-
- latest tightening evidence (2026-03-02): 12 consecutive successful CI benchmark-guardrail runs on `codex/rust-orchestration-phase8` (`22555725309` .. `22556698440`) observed Rust/Pure ratios in `[0.0252, 0.0288]`, supporting a tighter `0.10` threshold with headroom.
31+
- latest tightening evidence (2026-03-02): 17 consecutive successful CI benchmark-guardrail runs on `codex/rust-orchestration-phase8` (`22555725309` .. `22557953998`) observed Rust/Pure ratios in `[0.0252, 0.0288]`, supporting a tighter `0.05` threshold with headroom.
3232

3333
Configurable environment knobs:
3434

@@ -37,7 +37,7 @@ Configurable environment knobs:
3737
- `BENCH_TOPICS` (default `8`)
3838
- `BENCH_MAX_ITER` (default `20`)
3939
- `BENCH_EM_MAX_ITER` (default `40`)
40-
- `BENCH_RUST_TO_PURE_MAX_RATIO` (default `0.10`)
40+
- `BENCH_RUST_TO_PURE_MAX_RATIO` (default `0.05`)
4141

4242
## When to tighten thresholds
4343

0 commit comments

Comments
 (0)