Skip to content

Roda: add framework#186

Open
p8 wants to merge 1 commit intoMDA2AV:mainfrom
p8:roda
Open

Roda: add framework#186
p8 wants to merge 1 commit intoMDA2AV:mainfrom
p8:roda

Conversation

@p8
Copy link
Copy Markdown
Contributor

@p8 p8 commented Mar 26, 2026

No description provided.

@p8 p8 requested a review from MDA2AV as a code owner March 26, 2026 20:30
Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @p8, nice to see Roda joining HttpArena! 🎉 Roda's routing tree approach is really elegant and should be fast. A few things I spotted:

Copy-paste from Sinatra:

  • config.ru middleware responses have 'server' => 'sinatra' instead of 'server' => 'roda' (both MethodGuard and UploadHandler)
  • Comments mention "Rack/Sinatra" in the UploadHandler

Duplicate route in app.rb:

r.is('/baseline11') { handle_baseline11 }
r.is('/baseline11') { handle_baseline11 }  # duplicate

Roda plugins needed:
content_type, headers, and halt aren't available in Roda by default — they need plugins:

plugin :halt
plugin :default_headers, 'Server' => 'roda'

For content_type, you'd use response['Content-Type'] = 'application/json' or the :content_type plugin. And halt 500 would need the :halt plugin.

Without these plugins loaded, the app will likely crash on startup or when hitting those routes.

Otherwise the structure looks solid — jemalloc, YJIT, iodine, thread-local SQLite connections. Same patterns that work well for Sinatra/Rails. Looking forward to seeing the benchmark numbers!

@p8 p8 force-pushed the roda branch 4 times, most recently from a3c6fe4 to 9f7ab3d Compare March 26, 2026 20:46
Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @p8, nice update! The config.ru is clean now and the duplicate route is gone 👍

Two things that'll likely crash at runtime:

  1. halt needs plugin :halt — Roda doesn't have halt by default. You're calling halt 500, 'No dataset' in /json and /compression but the plugin isn't loaded. Without it you'll get a NoMethodError.

  2. params needs a plugin or use r.params — In the /db route you use bare params['min']. Roda doesn't expose params in the route block without a plugin. Either add plugin :all_verbs or just use r.params['min'] / request.params['min'] which goes through Rack directly.

Quick fix would be adding these near the top:

plugin :halt

And changing params to r.params in the db route (or add plugin :typecast_params if you want Roda's param handling).

Everything else looks solid — the routing tree, iodine setup, thread-local SQLite, gzip level 1. Should benchmark well once these are sorted!

Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @p8, nice — plugin :halt is in now 👍 and the config.ru cleanup looks great.

One thing that'll still crash:

params in /db needs a receiver — Roda doesn't have a bare params method. You're calling params['min'] and params['max'] but that'll raise NoMethodError. Quick fix: use request.params['min'] instead (same as you already do with request.GET in /baseline2).

Also no /upload route — that's fine if you want to skip the upload profile for now, just noting it in case it was an oversight.

Everything else looks solid!

Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more thing I missed earlier — the route is `/baseline11` but it should be `/baseline1` (same for the method name). This will 404 on the baseline1 test.

Also `params['min']' in `/db` still needs `request.params` (or `r.params`) — bare `params` isn't available in Roda without a plugin.

Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed one more thing from the latest push — in the /db route, you have:

rows.map do |r|

But r is already used as the Roda route variable from route do |r|. Shadowing it here means if anything inside that block tries to use the route r, it'll get a database row instead. Probably won't bite you in this block since you're just building JSON, but it's a subtle bug waiting to happen. Renaming the block param to row would be cleaner:

items = rows.map do |row|
  {
    'id' => row['id'], 'name' => row['name'], ...
  }
end

Also still seeing /baseline11 — should be /baseline1 👀

@p8 p8 force-pushed the roda branch 2 times, most recently from aa665fd to 1e2c205 Compare March 26, 2026 21:10
@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Nice cleanup @p8! plugin :halt is in, request.params in /db ✅, and row instead of r shadowing ✅

Still seeing /baseline11 on line 44 — should be /baseline1. This will 404 on the baseline1 test since the benchmark hits /baseline1.

@p8 p8 force-pushed the roda branch 4 times, most recently from 1335f77 to f40ffce Compare March 27, 2026 17:57
@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Actually, scratch my last comment about /baseline11 — I just checked the Rails entry and it's /baseline11 there too. That's the correct endpoint name. My bad! 🤦

Looks like the new commit is a force-push rewrite. The original /baseline11 route was correct all along. LGTM if the rest looks good to you @MDA2AV!

Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @p8, this is looking much cleaner! Iodine as the server, jemalloc, YJIT — solid setup.

One bug I spotted:

Line 67 — r.is '/compression' has a leading slash

In Roda's routing tree, r.is matches against the remaining path with the leading slash already stripped. So r.is '/compression' won't match /compression — it needs to be r.is 'compression' (no leading slash), same as your other routes.

Everything else looks good to me 👍

@p8 p8 force-pushed the roda branch 2 times, most recently from 084af32 to 7c2111f Compare March 27, 2026 19:00
Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! All the issues are addressed — compression route without leading slash ✅, request.params for query access ✅, no more variable shadowing ✅. Clean Roda entry with Iodine + YJIT + jemalloc. Ready for benchmarking 🚀

@p8
Copy link
Copy Markdown
Contributor Author

p8 commented Mar 27, 2026

/benchmark

@github-actions
Copy link
Copy Markdown

🚀 Benchmark run triggered for roda (all profiles). Results will be posted here when done.

@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Benchmark run triggered 🚀 Results will show up once it completes — should take a bit. Curious to see how Roda stacks up!

@github-actions
Copy link
Copy Markdown

Benchmark Results

Framework: roda | Profile: all profiles

roda / baseline / 512c (p=1, r=0, cpu=64)
  Best: 79820 req/s (CPU: 4083.7%, Mem: 659.9MiB) ===

roda / baseline / 4096c (p=1, r=0, cpu=64)
  Best: 67466 req/s (CPU: 3876.8%, Mem: 752.5MiB) ===

roda / baseline / 16384c (p=1, r=0, cpu=64)
  Best: 67309 req/s (CPU: 4221.6%, Mem: 747.7MiB) ===

roda / pipelined / 512c (p=16, r=0, cpu=unlimited)
  Best: 312181 req/s (CPU: 1380.4%, Mem: 388.1MiB) ===

roda / pipelined / 4096c (p=16, r=0, cpu=unlimited)
  Best: 310194 req/s (CPU: 1396.9%, Mem: 488.6MiB) ===

roda / pipelined / 16384c (p=16, r=0, cpu=unlimited)
  Best: 307380 req/s (CPU: 1309.6%, Mem: 470.1MiB) ===

roda / limited-conn / 512c (p=1, r=10, cpu=unlimited)
  Best: 64999 req/s (CPU: 4683.0%, Mem: 646.0MiB) ===

roda / limited-conn / 4096c (p=1, r=10, cpu=unlimited)
  Best: 63655 req/s (CPU: 3689.5%, Mem: 639.4MiB) ===

roda / json / 4096c (p=1, r=0, cpu=unlimited)
  Best: 59981 req/s (CPU: 908.9%, Mem: 972.1MiB) ===

roda / json / 16384c (p=1, r=0, cpu=unlimited)
  Best: 59338 req/s (CPU: 906.0%, Mem: 1.1GiB) ===

roda / compression / 4096c (p=1, r=0, cpu=unlimited)
  Best: 5030 req/s (CPU: 4158.1%, Mem: 970.5MiB) ===

roda / compression / 16384c (p=1, r=0, cpu=unlimited)
  Best: 4924 req/s (CPU: 4043.2%, Mem: 1000.0MiB) ===

roda / noisy / 512c (p=1, r=0, cpu=unlimited)
  Best: 159410 req/s (CPU: 1303.9%, Mem: 381.2MiB) ===

roda / noisy / 4096c (p=1, r=0, cpu=unlimited)
  Best: 156603 req/s (CPU: 1286.4%, Mem: 491.6MiB) ===

roda / noisy / 16384c (p=1, r=0, cpu=unlimited)
  Best: 141840 req/s (CPU: 1243.5%, Mem: 602.6MiB) ===

roda / mixed / 4096c (p=1, r=5, cpu=unlimited)
  Best: 14793 req/s (CPU: 3864.8%, Mem: 2.3GiB) ===

roda / mixed / 16384c (p=1, r=5, cpu=unlimited)
  Best: 14210 req/s (CPU: 3681.4%, Mem: 3.3GiB) ===
Full log
==============================================
=== roda / mixed / 4096c (p=1, r=5, cpu=unlimited) ===
==============================================
51375dda25a220f24867319353aa41459ca696725f7c136dbf9370646116c595
[wait] Waiting for server...
[ready] Server is up

[run 1/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   238.36ms   12.10ms   964.50ms    1.70s    2.40s

  270889 requests in 15.00s, 248544 responses
  Throughput: 16.57K req/s
  Bandwidth:  648.17MB/s
  Status codes: 2xx=221905, 3xx=0, 4xx=26639, 5xx=0
  Latency samples: 248543 / 248544 responses (100.0%)
  Reconnects: 53513
  Per-template: 26319,28342,27701,26914,26881,21872,22573,26639,20392,20910
  Per-template-ok: 26319,28342,27701,26914,26881,21872,22573,0,20392,20910

  WARNING: 26639/248544 responses (10.7%) had unexpected status (expected 2xx)
  CPU: 3864.8% | Mem: 2.3GiB

[run 2/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   243.59ms   12.50ms   963.70ms    1.76s    2.41s

  265115 requests in 15.00s, 243586 responses
  Throughput: 16.24K req/s
  Bandwidth:  640.59MB/s
  Status codes: 2xx=217211, 3xx=0, 4xx=26375, 5xx=0
  Latency samples: 243585 / 243586 responses (100.0%)
  Reconnects: 52509
  Per-template: 26393,28088,26294,25812,26121,21471,22195,26375,20139,20697
  Per-template-ok: 26393,28088,26294,25812,26121,21471,22195,0,20139,20697

  WARNING: 26375/243586 responses (10.8%) had unexpected status (expected 2xx)
  CPU: 3916.7% | Mem: 3.2GiB

[run 3/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   245.15ms   12.50ms    1.01s    1.73s    2.38s

  264336 requests in 15.00s, 241749 responses
  Throughput: 16.11K req/s
  Bandwidth:  636.00MB/s
  Status codes: 2xx=215503, 3xx=0, 4xx=26246, 5xx=0
  Latency samples: 241749 / 241749 responses (100.0%)
  Reconnects: 52088
  Per-template: 26375,27494,25977,25593,25926,21430,22180,26246,20044,20484
  Per-template-ok: 26375,27494,25977,25593,25926,21430,22180,0,20044,20484

  WARNING: 26246/241749 responses (10.9%) had unexpected status (expected 2xx)
  CPU: 3867.5% | Mem: 4.1GiB

=== Best: 14793 req/s (CPU: 3864.8%, Mem: 2.3GiB) ===
  Input BW: 1.45GB/s (avg template: 104924 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / mixed / 16384c (p=1, r=5, cpu=unlimited) ===
==============================================
9ba468ab491424e66f3d9fe6a2a3a08f3c74ee100bf42eede7e3a14db3197fc0
[wait] Waiting for server...
[ready] Server is up

[run 1/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   409.07ms   13.00ms    1.56s    2.83s    3.40s

  262593 requests in 15.00s, 227722 responses
  Throughput: 15.18K req/s
  Bandwidth:  590.47MB/s
  Status codes: 2xx=200977, 3xx=0, 4xx=26745, 5xx=0
  Latency samples: 227722 / 227722 responses (100.0%)
  Latency overflow (>5s): 91
  Reconnects: 51032
  Errors: connect 0, read 280, timeout 0
  Per-template: 23487,23189,23197,23715,24896,21603,23435,26745,18768,18687
  Per-template-ok: 23487,23189,23197,23715,24896,21603,23435,0,18768,18687

  WARNING: 26745/227722 responses (11.7%) had unexpected status (expected 2xx)
  CPU: 3635.1% | Mem: 1005.0MiB

[run 2/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   406.02ms   14.00ms    1.55s    2.84s    3.63s

  271715 requests in 15.00s, 240858 responses
  Throughput: 16.05K req/s
  Bandwidth:  574.40MB/s
  Status codes: 2xx=213152, 3xx=0, 4xx=27706, 5xx=0
  Latency samples: 240858 / 240858 responses (100.0%)
  Latency overflow (>5s): 47
  Reconnects: 51401
  Per-template: 23113,23609,24642,26243,28391,24581,26436,27707,18018,18118
  Per-template-ok: 23113,23609,24642,26243,28391,24581,26436,1,18018,18118

  WARNING: 27706/240858 responses (11.5%) had unexpected status (expected 2xx)
  CPU: 3681.4% | Mem: 3.3GiB

[run 3/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   482.98ms   13.40ms    1.74s    3.30s    4.50s

  234862 requests in 15.00s, 207575 responses
  Throughput: 13.84K req/s
  Bandwidth:  445.15MB/s
  Status codes: 2xx=185397, 3xx=0, 4xx=22178, 5xx=0
  Latency samples: 207575 / 207575 responses (100.0%)
  Latency overflow (>5s): 161
  Reconnects: 44041
  Per-template: 18643,20059,22096,24338,26512,22592,23426,22178,13598,14133
  Per-template-ok: 18643,20059,22096,24338,26512,22592,23426,0,13598,14133

  WARNING: 22178/207575 responses (10.7%) had unexpected status (expected 2xx)
  CPU: 2711.3% | Mem: 4.1GiB

=== Best: 14210 req/s (CPU: 3681.4%, Mem: 3.3GiB) ===
  Input BW: 1.39GB/s (avg template: 104924 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-roda
httparena-bench-roda
[skip] roda does not subscribe to async-db
[skip] roda does not subscribe to baseline-h2
[skip] roda does not subscribe to static-h2
[skip] roda does not subscribe to baseline-h3
[skip] roda does not subscribe to static-h3
[skip] roda does not subscribe to unary-grpc
[skip] roda does not subscribe to unary-grpc-tls
[skip] roda does not subscribe to echo-ws
[restore] Restoring CPU governor to powersave...

@BennyFranciscus
Copy link
Copy Markdown
Collaborator

🎉 Benchmark results are in! Roda is looking really solid:

Baseline (512c): 79.8K req/s — nearly 2x Rails (~40K) on the same Puma runtime. The minimal routing overhead really shows here.

Pipelined (512c): 312K req/s — strong. ~1.45x ahead of Rails (215K). Roda's lightweight request handling shines with pipelining.

JSON: ~60K req/s — decent throughput for per-request JSON computation.

Compression: ~5K req/s — consistent with other Ruby/Puma frameworks. GZIPOutputStream is the bottleneck, not the framework.

Memory: 660MB at baseline vs Rails' 2.3GB — Roda's minimalist design pays off big time on memory. Nearly 3.5x less.

Mixed (4096c): ~14.8K req/s — solid.

The "do less, go faster" philosophy is on full display here. Same Ruby runtime (Puma + MN threads), but Roda's thinner abstraction layer gives it a huge edge. Nice entry @p8!

@p8 p8 force-pushed the roda branch 2 times, most recently from ff7c6fb to 6c28116 Compare March 27, 2026 22:55
@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Hey @p8, noticed the latest push still has 'server' => 'sinatra' in the MethodGuard middleware in config.ru — should be 'server' => 'roda'. Small thing but it'll show up in the 405 response headers.

Everything else in the rewrite looks clean 👍

Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Server header is fixed 👍 Roda entry looks ready to go!

Copy link
Copy Markdown
Collaborator

@BennyFranciscus BennyFranciscus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up — the /compression endpoint has a hardcoded placeholder string instead of the actual large dataset:

payload = "aaaaaaaaaaaaaa" #opts[:large_json_payload]

Looks like debug code that was left in. The large payload is correctly pre-built at startup but never used. Should be opts[:large_json_payload] there.

Everything else looks solid — Roda routing tree API used correctly, iodine server, standard Ruby libs, jemalloc + YJIT. No rule violations besides that bug.

Repository owner deleted a comment from BennyFranciscus Mar 28, 2026
@p8 p8 force-pushed the roda branch 2 times, most recently from 9bcb81c to 9a0a1a3 Compare March 29, 2026 14:51
@p8
Copy link
Copy Markdown
Contributor Author

p8 commented Mar 29, 2026

/benchmark

@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Fix looks good 👍 Server header is correct now. Ready for merge from my side.

@github-actions
Copy link
Copy Markdown

🚀 Benchmark run triggered for roda (all profiles). Results will be posted here when done.

@github-actions
Copy link
Copy Markdown

Benchmark Results

Framework: roda | Profile: all profiles

roda / baseline / 512c (p=1, r=0, cpu=64)
  

roda / baseline / 4096c (p=1, r=0, cpu=64)
  

roda / baseline / 16384c (p=1, r=0, cpu=64)
  

roda / pipelined / 512c (p=16, r=0, cpu=unlimited)
  

roda / pipelined / 4096c (p=16, r=0, cpu=unlimited)
  

roda / pipelined / 16384c (p=16, r=0, cpu=unlimited)
  

roda / limited-conn / 512c (p=1, r=10, cpu=unlimited)
  

roda / limited-conn / 4096c (p=1, r=10, cpu=unlimited)
  

roda / json / 4096c (p=1, r=0, cpu=unlimited)
  

roda / json / 16384c (p=1, r=0, cpu=unlimited)
  

roda / compression / 4096c (p=1, r=0, cpu=unlimited)
  

roda / compression / 16384c (p=1, r=0, cpu=unlimited)
  

roda / noisy / 512c (p=1, r=0, cpu=unlimited)
  

roda / noisy / 4096c (p=1, r=0, cpu=unlimited)
  

roda / noisy / 16384c (p=1, r=0, cpu=unlimited)
  

roda / mixed / 4096c (p=1, r=5, cpu=unlimited)
  

roda / mixed / 16384c (p=1, r=5, cpu=unlimited)
  
Full log
#3 [internal] load .dockerignore
#3 transferring context: 2B done
#3 DONE 0.0s

#4 [internal] load build context
#4 transferring context: 143B done
#4 DONE 0.1s

#5 [1/6] FROM docker.io/library/ruby:4.0-slim@sha256:70b582c9e959be5680a3abbaaad6d7fc4539e1c2ec61610e3125d6138560b3ab
#5 resolve docker.io/library/ruby:4.0-slim@sha256:70b582c9e959be5680a3abbaaad6d7fc4539e1c2ec61610e3125d6138560b3ab 0.1s done
#5 DONE 0.1s

#6 [2/6] RUN apt-get update &&     apt-get install -y --no-install-recommends build-essential libsqlite3-dev libjemalloc2 &&     rm -rf /var/lib/apt/lists/*
#6 CACHED

#7 [3/6] WORKDIR /app
#7 CACHED

#8 [4/6] COPY Gemfile .
#8 CACHED

#9 [5/6] RUN bundle install --jobs=$(nproc)
#9 CACHED

#10 [6/6] COPY . .
#10 CACHED

#11 exporting to image
#11 exporting layers done
#11 exporting manifest sha256:5c65e7c3424ed8a4ef0545f207f65c74d8324db0968e5fa16284905401e75019 done
#11 exporting config sha256:a2a1c561e7d5d5912cbea742ea72f6459ece43594c11963baa4f8d25b83c8729 done
#11 exporting attestation manifest sha256:0e78016d47ea4c8d7bb8a257b32793f1e29318ad331f55be8db9b450f59d9a41 0.1s done
#11 exporting manifest list sha256:85c75d9bbafb5d065590445fd836297f2ea211a11068d3315e9640ea9fb86cb5
#11 exporting manifest list sha256:85c75d9bbafb5d065590445fd836297f2ea211a11068d3315e9640ea9fb86cb5 0.0s done
#11 naming to docker.io/library/httparena-roda:latest done
#11 unpacking to docker.io/library/httparena-roda:latest done
#11 DONE 0.2s

==============================================
=== roda / baseline / 512c (p=1, r=0, cpu=64) ===
==============================================
268fc3b2ff3fcf571dce623e1cffd94b509a9ebab68daafff7babc6e42e4f25b
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / baseline / 4096c (p=1, r=0, cpu=64) ===
==============================================
0e31135900d72c40bb77abc72cbd723e0c82cf67e5001647c7fd7cc108d538bf
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / baseline / 16384c (p=1, r=0, cpu=64) ===
==============================================
c7d6332622d607ec92496b66b885e84064806a5d2e58b2341251b5cc90d3abb6
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / pipelined / 512c (p=16, r=0, cpu=unlimited) ===
==============================================
732db31200bed2ac78ebd20aa531bd4a38b89e801fca2fd25163cad294a7ec06
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / pipelined / 4096c (p=16, r=0, cpu=unlimited) ===
==============================================
4b1de89294890560546c7673999f03a0304f95995159de9689f2ac614250eab9
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / pipelined / 16384c (p=16, r=0, cpu=unlimited) ===
==============================================
f92934e65a6dff07d52ac28fc298571cb49bb6072db1ab0af054bf0e13a7ef5c
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / limited-conn / 512c (p=1, r=10, cpu=unlimited) ===
==============================================
6831bf2c15b336750bc0a63b2bc7dcebf9ddef33170854e9ceebb29058c39b6c
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / limited-conn / 4096c (p=1, r=10, cpu=unlimited) ===
==============================================
336a0376048cbcb999f78b0c1ef99280de20eeea7075e9b875ad73f7d2cec408
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / json / 4096c (p=1, r=0, cpu=unlimited) ===
==============================================
54e387959bca5c68cdd84ad5174ed1e3a01ce6a69cd4d6182b7041084b7e806c
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / json / 16384c (p=1, r=0, cpu=unlimited) ===
==============================================
148e123bf324b81526a122d702cbbfdfb0e4c6e139ee64410c8668519d31e869
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda
[skip] roda does not subscribe to upload

==============================================
=== roda / compression / 4096c (p=1, r=0, cpu=unlimited) ===
==============================================
8143ff1bbc8e1ddc708d608935a594c5eaaec7ffdf830ee270fd88eefbf00047
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / compression / 16384c (p=1, r=0, cpu=unlimited) ===
==============================================
19899c6f3758654dafabce9cbcbe0d1f8d1649977badf552d7a5ef3f0ac67b63
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / noisy / 512c (p=1, r=0, cpu=unlimited) ===
==============================================
a9489bb399b06c8cb02af4f81f3eeba3a9cc6d32a5be4bfdb626359e41cf3f8f
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / noisy / 4096c (p=1, r=0, cpu=unlimited) ===
==============================================
996387a028c3f27aa7edcfc0db56c0f430f2125a68b642e54edcd9c54f3e2b45
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / noisy / 16384c (p=1, r=0, cpu=unlimited) ===
==============================================
45d1da2102c6663e09a805d2cc14ed95b91ff68a2edbc8c72221906a22d1a5f8
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / mixed / 4096c (p=1, r=5, cpu=unlimited) ===
==============================================
fc6cd75620ab549ad4a7c531dd49546dd34b41389dc1471b07eeae6ec4feceb3
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / mixed / 16384c (p=1, r=5, cpu=unlimited) ===
==============================================
e860beeb895bb0e3633fb4de4cbc960a37a4bfc4446bcba12b74dee50518f45f
[wait] Waiting for server...
FAIL: Server did not start within 30s — skipping
httparena-bench-roda
httparena-bench-roda
[skip] roda does not subscribe to async-db
[skip] roda does not subscribe to baseline-h2
[skip] roda does not subscribe to static-h2
[skip] roda does not subscribe to baseline-h3
[skip] roda does not subscribe to static-h3
[skip] roda does not subscribe to unary-grpc
[skip] roda does not subscribe to unary-grpc-tls
[skip] roda does not subscribe to echo-ws
[restore] Restoring CPU governor to performance...

@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Heads up @p8 — the latest benchmark run failed across all profiles. Every test hit "Server did not start within 30s."

Looking at the files, the Dockerfile has CMD ["bundle", "exec", "puma", "-C", "puma.rb"] but there's no puma.rb in the PR. Looks like it got dropped in the last push. The previous run (79.8K baseline) worked fine so it was definitely there before.

Once you add puma.rb back we can re-run and should be good to merge 👍

@p8
Copy link
Copy Markdown
Contributor Author

p8 commented Mar 29, 2026

/benchmark

@github-actions
Copy link
Copy Markdown

🚀 Benchmark run triggered for roda (all tests). Results will be posted here when done.

@github-actions
Copy link
Copy Markdown

Benchmark Results

Framework: roda | Profile: all profiles

roda / baseline / 512c (p=1, r=0, cpu=64)
  Best: 88973 req/s (CPU: 6461.2%, Mem: 7.6GiB) ===

roda / baseline / 4096c (p=1, r=0, cpu=64)
  Best: 50597 req/s (CPU: 6409.3%, Mem: 6.6GiB) ===

roda / baseline / 16384c (p=1, r=0, cpu=64)
  Best: 37473 req/s (CPU: 6324.3%, Mem: 7.6GiB) ===

roda / pipelined / 512c (p=16, r=0, cpu=unlimited)
  Best: 1143971 req/s (CPU: 9189.0%, Mem: 4.3GiB) ===

roda / pipelined / 4096c (p=16, r=0, cpu=unlimited)
  Best: 1397809 req/s (CPU: 10158.6%, Mem: 4.6GiB) ===

roda / pipelined / 16384c (p=16, r=0, cpu=unlimited)
  Best: 1197791 req/s (CPU: 10050.6%, Mem: 5.2GiB) ===

roda / limited-conn / 512c (p=1, r=10, cpu=unlimited)
  Best: 40034 req/s (CPU: 6503.9%, Mem: 9.9GiB) ===

roda / limited-conn / 4096c (p=1, r=10, cpu=unlimited)
  Best: 37874 req/s (CPU: 6521.7%, Mem: 11.5GiB) ===

roda / json / 4096c (p=1, r=0, cpu=unlimited)
  Best: 286555 req/s (CPU: 10379.7%, Mem: 10.6GiB) ===

roda / json / 16384c (p=1, r=0, cpu=unlimited)
  Best: 268743 req/s (CPU: 9919.1%, Mem: 10.2GiB) ===

roda / compression / 4096c (p=1, r=0, cpu=unlimited)
  Best: 9677 req/s (CPU: 12210.3%, Mem: 18.0GiB) ===

roda / compression / 16384c (p=1, r=0, cpu=unlimited)
  Best: 9088 req/s (CPU: 11422.7%, Mem: 20.0GiB) ===

roda / noisy / 512c (p=1, r=0, cpu=unlimited)
  Best: 101692 req/s (CPU: 4389.9%, Mem: 11.1GiB) ===

roda / noisy / 4096c (p=1, r=0, cpu=unlimited)
  Best: 398745 req/s (CPU: 9544.6%, Mem: 15.1GiB) ===

roda / noisy / 16384c (p=1, r=0, cpu=unlimited)
  Best: 384140 req/s (CPU: 9866.8%, Mem: 18.0GiB) ===

roda / mixed / 4096c (p=1, r=5, cpu=unlimited)
  Best: 25734 req/s (CPU: 9977.6%, Mem: 22.5GiB) ===

roda / mixed / 16384c (p=1, r=5, cpu=unlimited)
  Best: 24520 req/s (CPU: 9531.4%, Mem: 25.7GiB) ===
Full log
7f1fd5093bc6b6d6234d7e39b765ca8665338a5fe2ea14f0e4f9c1e3d47aaadd
[wait] Waiting for server...
[ready] Server is up

[run 1/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   127.94ms   65.60ms   346.00ms   782.70ms    1.22s

  454972 requests in 15.00s, 416854 responses
  Throughput: 27.78K req/s
  Bandwidth:  1.10GB/s
  Status codes: 2xx=371579, 3xx=0, 4xx=45275, 5xx=0
  Latency samples: 416854 / 416854 responses (100.0%)
  Reconnects: 89027
  Errors: connect 0, read 46, timeout 0
  Per-template: 44558,44822,45227,45492,45667,36909,37141,45275,35998,35765
  Per-template-ok: 44558,44822,45227,45492,45667,36909,37141,0,35998,35765

  WARNING: 45275/416854 responses (10.9%) had unexpected status (expected 2xx)
  CPU: 10048.0% | Mem: 23.1GiB

[run 2/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   129.26ms   69.20ms   341.50ms   738.30ms    1.50s

  471639 requests in 15.00s, 433280 responses
  Throughput: 28.88K req/s
  Bandwidth:  1.14GB/s
  Status codes: 2xx=386010, 3xx=0, 4xx=47270, 5xx=0
  Latency samples: 433247 / 433280 responses (100.0%)
  Reconnects: 92744
  Per-template: 46209,46541,46897,47259,47569,38522,38656,47270,37277,37047
  Per-template-ok: 46209,46541,46897,47259,47569,38522,38656,0,37277,37047

  WARNING: 47270/433280 responses (10.9%) had unexpected status (expected 2xx)
  CPU: 9977.6% | Mem: 22.5GiB

[run 3/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   129.69ms   70.10ms   347.70ms   726.80ms    1.18s

  470453 requests in 15.00s, 429756 responses
  Throughput: 28.64K req/s
  Bandwidth:  1.13GB/s
  Status codes: 2xx=382931, 3xx=0, 4xx=46825, 5xx=0
  Latency samples: 429756 / 429756 responses (100.0%)
  Reconnects: 91873
  Per-template: 45917,46181,46515,46857,47204,38092,38240,46825,37026,36899
  Per-template-ok: 45917,46181,46515,46857,47204,38092,38240,0,37026,36899

  WARNING: 46825/429756 responses (10.9%) had unexpected status (expected 2xx)
  CPU: 9984.7% | Mem: 23.1GiB

=== Best: 25734 req/s (CPU: 9977.6%, Mem: 22.5GiB) ===
  Input BW: 2.51GB/s (avg template: 104924 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-roda
httparena-bench-roda

==============================================
=== roda / mixed / 16384c (p=1, r=5, cpu=unlimited) ===
==============================================
7a788b6f27a2ea949e2ffabf9bfb3776ca4e6cd6505054d9e39e1063b78f74c4
[wait] Waiting for server...
[ready] Server is up

[run 1/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   507.81ms   367.60ms    1.11s    2.44s    4.14s

  458384 requests in 15.00s, 408539 responses
  Throughput: 27.23K req/s
  Bandwidth:  1.08GB/s
  Status codes: 2xx=367807, 3xx=0, 4xx=40732, 5xx=0
  Latency samples: 408539 / 408539 responses (100.0%)
  Latency overflow (>5s): 241
  Reconnects: 82030
  Errors: connect 0, read 541, timeout 0
  Per-template: 43929,44105,44370,44897,45269,37139,37137,40731,35323,35639
  Per-template-ok: 43929,44105,44370,44897,45269,37139,37137,0,35322,35639

  WARNING: 40732/408539 responses (10.0%) had unexpected status (expected 2xx)
  CPU: 9531.4% | Mem: 25.7GiB

[run 2/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   500.65ms   336.90ms    1.15s    2.61s    4.68s

  459219 requests in 15.01s, 409883 responses
  Throughput: 27.31K req/s
  Bandwidth:  1.08GB/s
  Status codes: 2xx=367051, 3xx=0, 4xx=42832, 5xx=0
  Latency samples: 409883 / 409883 responses (100.0%)
  Latency overflow (>5s): 322
  Reconnects: 82073
  Errors: connect 0, read 141, timeout 0
  Per-template: 43267,43983,44380,44799,45360,37217,37357,42832,35479,35209
  Per-template-ok: 43267,43983,44380,44799,45360,37217,37357,0,35479,35209

  WARNING: 42832/409883 responses (10.4%) had unexpected status (expected 2xx)
  CPU: 9369.5% | Mem: 27.3GiB

[run 3/3]
gcannon — io_uring HTTP load generator
  Target:    localhost:8080/
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  1
  Req/conn:  5
  Templates: 10
  Expected:  200
  Duration:  15s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   511.58ms   347.50ms    1.18s    2.61s    4.42s

  457893 requests in 15.02s, 408760 responses
  Throughput: 27.21K req/s
  Bandwidth:  1.07GB/s
  Status codes: 2xx=365428, 3xx=0, 4xx=43332, 5xx=0
  Latency samples: 408760 / 408760 responses (100.0%)
  Latency overflow (>5s): 314
  Reconnects: 81602
  Errors: connect 0, read 71, timeout 0
  Per-template: 43169,43563,44085,44735,45316,37242,37376,43332,34900,35042
  Per-template-ok: 43169,43563,44085,44735,45316,37242,37376,0,34900,35042

  WARNING: 43332/408760 responses (10.6%) had unexpected status (expected 2xx)
  CPU: 9198.7% | Mem: 28.1GiB

=== Best: 24520 req/s (CPU: 9531.4%, Mem: 25.7GiB) ===
  Input BW: 2.40GB/s (avg template: 104924 bytes)
[dry-run] Results not saved (use --save to persist)
httparena-bench-roda
httparena-bench-roda
[skip] roda does not subscribe to async-db
[skip] roda does not subscribe to baseline-h2
[skip] roda does not subscribe to static-h2
[skip] roda does not subscribe to baseline-h3
[skip] roda does not subscribe to static-h3
[skip] roda does not subscribe to unary-grpc
[skip] roda does not subscribe to unary-grpc-tls
[skip] roda does not subscribe to echo-ws
[restore] Restoring CPU governor to performance...

@BennyFranciscus
Copy link
Copy Markdown
Collaborator

Nice, server starts clean now! These numbers are interesting though — pretty different from the first run:

Baseline (512c): 89K req/s (was 79.8K) — ~11% bump, but memory jumped from 660MB to 7.6GiB and CPU from 4083% → 6461%. Looks like WEB_CONCURRENCY=auto is spawning way more workers on this run. The machine might have had different CPU availability between runs.

Pipelined: 1.4M req/s at 4096c — this is the big one. Was 310K before. That's a 4.5x jump. With more Puma workers handling the pipelined load, the throughput scales almost linearly. 1.4M req/s from Ruby is genuinely impressive.

JSON: 287K req/s (was 60K) — nearly 5x. Same story, more workers.

Memory is the trade-off here. 7.6GiB at baseline vs 660MB before. WEB_CONCURRENCY=auto on a 64-core machine spawns a lot of workers, and each Puma worker is a forked Ruby process. That's expected.

The ~10.9% 4xx responses are from the 405 method guard endpoint — that's working as intended.

Overall looking great. The first run must've had fewer workers for some reason (maybe CPU governor or container limits were different). These numbers with full worker scaling are the more representative ones. Ready to merge from my side 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants