Repository navigation
Conversation
|
/benchmark-multiple -f tako,tako-grpc,tako-gateway --save |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFrameworks: 3 | Test: ✅
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 3,139,709 | — | — | — | — | 6652.2% | 118MiB | -4.3% | -4.8% |
| pipelined | 4096 | 20,509,219 | — | — | — | — | 6354.0% | 126MiB | -8.4% | +0.8% |
| limited-conn | 4096 | 2,407,752 | — | — | — | — | 6144.4% | 154MiB | -2.2% | ~0% |
| json-comp | 4096 | 208,402 | — | — | — | — | 6229.1% | 245MiB | -1.3% | ~0% |
| json-comp | 16384 | 201,312 | — | — | — | — | 6485.6% | 594MiB | -3.5% | +0.2% |
| json-tls | 4096 | 1,009,044 | — | — | — | — | 6068.4% | 222MiB | +9.3% | -24.2% |
| 8gbit | 512 | 48,980 | 0.9796 |
53.24us | 138.0us | 2937.0us | 253.5% | 93MiB | -0.2% | -7.9% |
| static-tls | 1024 | 140,842 | — | — | — | — | 3229.7% | 157MiB | +4.8% | -3.1% |
| async-db | 1024 | 242,075 | — | — | — | — | 3681.6% | 96MiB | -0.1% | -1.0% |
| baseline-h2 | 256 | 829,933 | — | — | — | — | 4755.5% | 204MiB | +12.1% | -42.4% |
| baseline-h2 | 1024 | 724,529 | — | — | — | — | 4806.1% | 566MiB | +6.0% | -44.7% |
| static-h2 | 256 | 131,413 | — | — | — | — | 3003.9% | 318MiB | +1.9% | -5.1% |
| static-h2 | 1024 | 130,233 | — | — | — | — | 2740.0% | 1.0GiB | +3.1% | ~0% |
| baseline-h2c | 256 | 848,574 | — | — | — | — | 4676.3% | 176MiB | +14.0% | -55.2% |
| baseline-h2c | 1024 | 737,895 | — | — | — | — | 4758.3% | 624MiB | +10.7% | -49.2% |
| baseline-h2c | 4096 | 0 | — | — | — | — | 3869.5% | 977MiB | -100.0% | -56.6% |
| json-h2c | 1024 | 602,946 | — | — | — | — | 4047.6% | 551MiB | -16.6% | +36.7% |
| json-h2c | 4096 | 531,594 | — | — | — | — | 3839.8% | 1.4GiB | -14.4% | +16.7% |
| baseline-h3 | 64 | 536,729 | — | — | — | — | 2928.1% | 107MiB | -0.7% | +9.2% |
| static-h3 | 64 | 107,569 | — | — | — | — | 2817.0% | 175MiB | -7.7% | +18.2% |
| echo-ws | 512 | 4,191,358 | — | — | — | — | 6465.1% | 39MiB | +2.4% | +2.6% |
| echo-ws | 4096 | 4,066,143 | — | — | — | — | 6444.6% | 121MiB | ~0% | -0.8% |
| echo-ws | 16384 | 3,698,229 | — | — | — | — | 5989.3% | 388MiB | +5.4% | -0.5% |
| echo-ws-pipeline | 512 | 57,721,923 | — | — | — | — | 6603.9% | 40MiB | +903.4% | ~0% |
| echo-ws-pipeline | 4096 | 56,360,143 | — | — | — | — | 6592.1% | 122MiB | +838.1% | +1.7% |
| echo-ws-pipeline | 16384 | 50,973,465 | — | — | — | — | 6328.4% | 354MiB | +773.6% | -8.3% |
| echo-ws-limited | 512 | 2,156,338 | — | — | — | — | 5894.3% | 51MiB | +0.3% | ~0% |
| echo-ws-limited | 4096 | 2,570,415 | — | — | — | — | 6357.1% | 136MiB | +1.0% | -2.2% |
| latency-1m | 1024 | 997,908 | 0.9979 | 24.32us | 184.0us | 2637.0us | 2672.1% | 52MiB | ~0% | +2.0% |
| latency-10k | 1024 | 9,983 | 0.9983 | 26.99us | 108.0us | 146.0us | 25.8% | 49MiB | ~0% | -2.0% |
| latency-500k-8cpu | 1024 | 455,866 | 0.9117 |
17.50us | 1794560.0us | 1925632.0us | 800.2% | 33MiB | -1.0% | -10.8% |
| async | 32000 | 2,235,424 | — | — | — | — | 5199.2% | 966MiB | -0.4% | -0.3% |
Composite score, main against this run, in the board's default view:
| Family | main | this run | Δ |
|---|---|---|---|
| HTTP/1.1 | 6635.6 (6 of 107) | 6619.7 (6 of 107) | -15.9 |
| HTTP/2 | 568.3 (28 of 40) | 516.1 (30 of 40) | -52.2 |
| HTTP/3 | 509.7 (11 of 19) | 477.9 (11 of 19) | -31.8 |
| Gateway | 896.2 (6 of 9) | 896.2 (6 of 9) | +0.0 |
| gRPC | 179.5 (13 of 13) | 170.7 (13 of 13) | -8.8 |
| WebSocket | 2075.9 (10 of 38) | 2988.7 (1 of 38) | +912.8 |
The results on main were measured in another session, so a few points up or down can be the machine and not the change.
✅ tako-grpc
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| unary-grpc | 256 | 554,095 | 5116.2% | 196MiB | -4.1% | -45.4% |
| unary-grpc | 1024 | 510,326 | 5211.5% | 720MiB | -0.6% | -36.1% |
| unary-grpc-tls | 256 | 564,324 | 5267.4% | 220MiB | -9.4% | -30.8% |
| unary-grpc-tls | 1024 | 485,284 | 5040.5% | 582MiB | -4.6% | -43.2% |
❌ tako-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 75,631 | 3811.3% | 2.7GiB | -2.3% | -3.6% |
| gateway-64 | 1024 | 64,716 | 3501.4% | 5.7GiB | -6.0% | +7.5% |
| gateway-h3 | 64 | 81,376 | 3427.4% | 416MiB | -2.0% | -2.6% |
| gateway-h3 | 256 | 57,583 | 2573.7% | 1.5GiB | -24.7% | +174.3% |
Log tail
#37 exporting manifest list sha256:ba368aae0b68fb70e66e2de42d06a35f82bffdf211899c62a9f3898067171b68 0.0s done
#37 naming to docker.io/library/httparena-tako-gateway-production-stack-authsvc:latest done
#37 unpacking to docker.io/library/httparena-tako-gateway-production-stack-authsvc:latest 0.0s done
#37 DONE 0.3s
#38 [edge] resolving provenance for metadata file
#38 DONE 0.0s
#39 [server] resolving provenance for metadata file
#39 DONE 0.1s
#40 [authsvc] resolving provenance for metadata file
#40 DONE 0.0s
Image httparena-tako-gateway-production-stack-server Built
Image httparena-tako-gateway-production-stack-authsvc Built
Image httparena-tako-gateway-production-stack-edge Built
Container httparena-tako-gateway-production-stack-cache-1 Creating
Container httparena-tako-gateway-production-stack-authsvc-1 Creating
Container httparena-tako-gateway-production-stack-authsvc-1 Created
Container httparena-tako-gateway-production-stack-cache-1 Created
Container httparena-tako-gateway-production-stack-server-1 Creating
Container httparena-tako-gateway-production-stack-server-1 Created
Container httparena-tako-gateway-production-stack-edge-1 Creating
Container httparena-tako-gateway-production-stack-edge-1 Created
Container httparena-tako-gateway-production-stack-cache-1 Starting
Container httparena-tako-gateway-production-stack-authsvc-1 Starting
Container httparena-tako-gateway-production-stack-authsvc-1 Started
Container httparena-tako-gateway-production-stack-cache-1 Started
Container httparena-tako-gateway-production-stack-cache-1 Waiting
Container httparena-tako-gateway-production-stack-cache-1 Healthy
Container httparena-tako-gateway-production-stack-server-1 Starting
Container httparena-tako-gateway-production-stack-server-1 Started
Container httparena-tako-gateway-production-stack-authsvc-1 Waiting
Container httparena-tako-gateway-production-stack-authsvc-1 Error dependency authsvc failed to start
dependency failed to start: container httparena-tako-gateway-production-stack-authsvc-1 exited (0)
─── httparena-tako-gateway-production-stack-edge-1 — status=created exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-edge-1 — the container produced no output at all
─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all
─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
| 7:C 13 Nov 2026 11:33:47.352 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
| 7:C 13 Nov 2026 11:33:47.352 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
| 7:C 13 Nov 2026 11:33:47.352 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
| 7:C 13 Nov 2026 11:33:47.352 * Configuration loaded
| 7:M 13 Nov 2026 11:33:47.352 * monotonic clock: POSIX clock_gettime
| 7:M 13 Nov 2026 11:33:47.356 * Running mode=standalone, port=6379.
| 7:M 13 Nov 2026 11:33:47.356 * Server initialized
| 7:M 13 Nov 2026 11:33:47.357 * Ready to accept connections tcp
| [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───
─── httparena-tako-gateway-production-stack-authsvc-1 — status=exited exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — the container produced no output at all
[FAIL] gateway compose up failed
httparena-postgres
[info] restoring loopback MTU to 65536
Preserve load-generator failures, validate completed requests, and suppress stale composite scores after failed runs. Update h2load for duration-mode shutdown, fix auth sidecar build caching, and cover both benchmark runners with regression tests.
|
/benchmark-multiple -f tako,tako-grpc,tako-gateway --save Rerun with Tako 2.7.1 and the benchmark fixes in this PR. Before approving, please ensure the runner's native h2load is >=1.69.0 (or set H2LOAD to an installed compatible h2load-h3); older clients can hang after the duration expires and are now rejected. |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
…uthsvc on Rust 1.100, unbreak the aspnet gateway entries (#1767) * validate.sh: stop the standalone container before the compose profiles The standalone container runs on --network host whenever an entry subscribes to a compose profile, and it was left running while the gateway and production-stack stacks came up. An entry that also serves h2/h3 keeps 8443 (and 8080), so the stack's edge failed to bind, shared the port, or exited on the QUIC port, and the checks reached the standalone server instead of the stack. * authsvc: ship the real binary under cargo 1.100's build-dir layout production-stack has failed on the bench host since the morning of 2026-11-13 on every entry (roadrunner-gateway #1757, morojs #1761, tako-gateway #1762): the shared authsvc container exits 0 with no output, the edge never sees it healthy, and the run ends with no results. rust:1-alpine moved to Rust 1.100 on 2026-11-12. Its cargo keeps each package's artifacts under target/release/build/<package>/<hash>/ rather than target/release/deps/, so the cache-priming step's `rm -rf target/release/deps/authsvc*` stopped matching anything. The placeholder built from `fn main() {}` stayed up to date, the second `cargo build` finished in 0.04s without compiling src/, and the image shipped the placeholder: a 418 KB binary that exits 0 at once. `cargo clean --release -p authsvc` removes the placeholder's artifacts wherever cargo keeps them, and a grep for the server's startup line fails the build if the placeholder is ever shipped again. Built against the current rust:1-alpine (1.100.0): a 1.2 MB binary that logs "authsvc listening on ...", answers /_health with 200 and an /_auth without a token with 401. * aspnet-minimal_nginx, aspnet-minimal_caddy: build against aspnet-minimal's Services and Types Both Dockerfiles still COPY aspnet-minimal's AppData.cs and Models.cs, which #1002 (2026-07-18) split into Services/ and Types/, so neither image has built since. They were the only failures in #1615's validation sweep, and their published results have not been refreshed since July. Copy Handlers.cs with Services/ and Types/, register DatabaseService, DatasetService, ItemService and the AppJsonContext resolver the way aspnet-minimal's Program.cs does, and give the nginx entry's /api routes DatabaseService's Postgres pool in place of AppData.PgDataSource. The caddy entry gains the StackExchange.Redis reference DatabaseService needs; it connects to nothing unless REDIS_URL is set, which gateway-h3 never does. --------- Co-authored-by: Jacob Rothstein <hi@jbr.me>
|
The authsvc fix is now on main in #1767. It's the same Because both changes edit the same lines, this branch now conflicts with main in |
# Conflicts: # frameworks/_shared/authsvc/Dockerfile
|
Update: I merged main into the branch (6bb3c3b) and took main's authsvc Dockerfile, so you don't need to resolve that conflict. Pull before your next push. |
Benchmark ResultsFrameworks: 3 | Test: ❌
|
| Test | Conn | RPS | Rate | CPU/req | p99 | p99.9 | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|---|---|---|---|
| baseline | 4096 | 3,201,484 | — | — | — | — | 6335.0% | 124MiB | -2.5% | ~0% |
| pipelined | 4096 | 20,799,968 | — | — | — | — | 6767.5% | 121MiB | -7.1% | -3.2% |
| limited-conn | 4096 | 2,421,703 | — | — | — | — | 6447.1% | 154MiB | -1.6% | ~0% |
| json-comp | 4096 | 204,795 | — | — | — | — | 6478.6% | 243MiB | -3.0% | -0.8% |
| json-comp | 16384 | 204,554 | — | — | — | — | 6501.0% | 592MiB | -1.9% | -0.2% |
| json-tls | 4096 | 1,001,696 | — | — | — | — | 5949.9% | 224MiB | +8.5% | -23.5% |
| 8gbit | 512 | 48,904 | 0.9781 |
52.03us | 135.0us | 7498.0us | 252.0% | 93MiB | -0.4% | -7.9% |
| static-tls | 1024 | 136,403 | — | — | — | — | 3608.4% | 160MiB | +1.5% | -1.2% |
| async-db | 1024 | 235,674 | — | — | — | — | 3547.8% | 97MiB | -2.8% | ~0% |
| baseline-h3 | 64 | 534,875 | — | — | — | — | 3061.3% | 102MiB | -1.0% | +4.1% |
| static-h3 | 64 | 113,798 | — | — | — | — | 3298.6% | 172MiB | -2.4% | +16.2% |
| echo-ws | 512 | 4,120,095 | — | — | — | — | 6328.1% | 40MiB | +0.7% | +5.3% |
| echo-ws | 4096 | 4,094,900 | — | — | — | — | 6056.1% | 113MiB | +0.8% | -7.4% |
| echo-ws | 16384 | 3,711,259 | — | — | — | — | 6101.8% | 394MiB | +5.8% | +1.0% |
| echo-ws-pipeline | 512 | 57,394,140 | — | — | — | — | 6602.8% | 41MiB | +897.7% | +2.5% |
| echo-ws-pipeline | 4096 | 55,968,937 | — | — | — | — | 6260.2% | 114MiB | +831.6% | -5.0% |
| echo-ws-pipeline | 16384 | 51,593,811 | — | — | — | — | 6078.2% | 364MiB | +784.2% | -5.7% |
| echo-ws-limited | 512 | 2,143,254 | — | — | — | — | 5859.9% | 50MiB | -0.3% | -2.0% |
| echo-ws-limited | 4096 | 2,573,728 | — | — | — | — | 6101.9% | 139MiB | +1.2% | ~0% |
| latency-1m | 1024 | 997,802 | 0.9978 | 24.09us | 180.0us | 3041.0us | 2636.7% | 49MiB | ~0% | -3.9% |
| latency-10k | 1024 | 9,982 | 0.9982 | 27.90us | 105.0us | 141.0us | 26.6% | 50MiB | ~0% | ~0% |
| latency-500k-8cpu | 1024 | 464,802 | 0.9296 |
17.15us | 1362432.0us | 1405440.0us | 795.7% | 33MiB | +0.9% | -10.8% |
| async | 32000 | 2,236,354 | — | — | — | — | 5165.3% | 921MiB | -0.4% | -5.0% |
Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.
Log tail
Throughput: 2.13M req/s
Bandwidth: 239.29MB/s
Status codes: 2xx=21267553, 3xx=0, 4xx=0, 5xx=0
Latency samples: 21267553 / 21267553 responses (100.0%)
[info] CPU 4812.2% | Mem 757MiB
[run 2/3]
gcannon v0.5.3
Target: localhost:8080/
Threads: 64
Conns: 32000 (500/thread)
Pipeline: 1
Req/conn: unlimited (keep-alive)
Expected: 200
Duration: 10s
Thread Stats Avg p50 p90 p99 p99.9
Latency 14.19ms 14.00ms 15.80ms 17.70ms 19.60ms
22395542 requests in 10.00s, 22363542 responses
Throughput: 2.24M req/s
Bandwidth: 251.62MB/s
Status codes: 2xx=22363542, 3xx=0, 4xx=0, 5xx=0
Latency samples: 22363542 / 22363542 responses (100.0%)
[info] CPU 5165.3% | Mem 921MiB
[run 3/3]
gcannon v0.5.3
Target: localhost:8080/
Threads: 64
Conns: 32000 (500/thread)
Pipeline: 1
Req/conn: unlimited (keep-alive)
Expected: 200
Duration: 10s
Thread Stats Avg p50 p90 p99 p99.9
Latency 14.20ms 14.00ms 15.80ms 17.70ms 19.80ms
22376767 requests in 10.00s, 22344767 responses
Throughput: 2.23M req/s
Bandwidth: 251.41MB/s
Status codes: 2xx=22344767, 3xx=0, 4xx=0, 5xx=0
Latency samples: 22344767 / 22344767 responses (100.0%)
[info] CPU 5180.0% | Mem 972MiB
=== Best: 2236354 req/s (CPU: 5165.3%, Mem: 921MiB) ===
[info] input BW: 102.37MB/s (avg template: 48 bytes)
[info] saved results/async/32000/tako.json
httparena-bench-tako
httparena-bench-tako
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 23 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 9 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536
❌ tako-grpc
No results captured.
Log tail
#13 unpacking to docker.io/library/httparena-tako-grpc:latest 0.0s done
#13 DONE 0.5s
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] skip: tako-grpc does not subscribe to baseline
[info] skip: tako-grpc does not subscribe to pipelined
[info] skip: tako-grpc does not subscribe to limited-conn
[info] skip: tako-grpc does not subscribe to json-comp
[info] skip: tako-grpc does not subscribe to json-tls
[info] skip: tako-grpc does not subscribe to 8gbit
[info] skip: tako-grpc does not subscribe to static-tls
[info] skip: tako-grpc does not subscribe to async-db
[info] skip: tako-grpc does not subscribe to fortunes
[info] skip: tako-grpc does not subscribe to baseline-h2
[info] skip: tako-grpc does not subscribe to static-h2
[info] skip: tako-grpc does not subscribe to baseline-h2c
[info] skip: tako-grpc does not subscribe to json-h2c
[info] skip: tako-grpc does not subscribe to baseline-h3
[info] skip: tako-grpc does not subscribe to static-h3
[info] skip: tako-grpc does not subscribe to gateway-64
[info] skip: tako-grpc does not subscribe to gateway-h3
[info] skip: tako-grpc does not subscribe to production-stack
==============================================
=== tako-grpc / unary-grpc / 256c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
==============================================
=== tako-grpc / unary-grpc / 1024c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
==============================================
=== tako-grpc / unary-grpc-tls / 256c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
==============================================
=== tako-grpc / unary-grpc-tls / 1024c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
[info] skip: tako-grpc does not subscribe to echo-ws
[info] skip: tako-grpc does not subscribe to echo-ws-pipeline
[info] skip: tako-grpc does not subscribe to echo-ws-limited
[info] skip: tako-grpc does not subscribe to latency-1m
[info] skip: tako-grpc does not subscribe to latency-10k
[info] skip: tako-grpc does not subscribe to latency-500k-8cpu
[info] skip: tako-grpc does not subscribe to async
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 23 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 4 profile/connection measurements failed
[info] restoring loopback MTU to 65536
❌ tako-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-h3 | 64 | 81,857 | 3477.9% | 488MiB | -1.4% | +14.3% |
| gateway-h3 | 256 | 58,187 | 2636.9% | 1.5GiB | -23.9% | +174.3% |
Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.
Log tail
59
.
41.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h3
58. Stopping all clients.
finished in 5.04s, 58652.60 req/s, 626.06MB/s
requests: 293263 total, 301455 started, 293263 done, 293263 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 298418 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 3.06GB (3282372703) total, 21.17MB (22193977) headers (space savings 50.52%), 3.03GB (3258297580) data
UDP datagram: 94869 sent, 2356090 received
min max median p95 p99 mean sd +/- sd
request : 982us 1.55s 68.18ms 539.61ms 799.05ms 134.06ms 172.77ms 87.08%
connect : 4.20ms 129.26ms 14.28ms 72.38ms 103.43ms 27.78ms 23.83ms 83.59%
TTFB : 6.70ms 194.78ms 49.98ms 146.65ms 178.92ms 59.31ms 40.35ms 70.70%
req/s : 126.38 418.11 222.36 308.96 365.55 229.08 44.68 69.53%
min RTT : 11us 1.60ms 50us 218us 919us 93us 151us 96.48%
smoothed RTT: 12.48ms 63.10ms 28.42ms 46.93ms 59.87ms 29.64ms 8.77ms 75.78%
packets sent: 295 464 373 414 442 372.58 27.53 68.75%
packets recv: 5008 16937 8923 12485 14782 9205.48 1822.22 69.53%
packets lost: 0 0 0 0 0 0.00 0.00 100.00%
GRO packets : 1 17 4 14 14 6.38 5.39 71.02%
[info] CPU 2636.9% | Mem 1.5GiB
[info] proxy: 2062% 1.3GiB | server: 575% 130MiB
=== Best: 58187 req/s (CPU: 2636.9%, Mem: 1.5GiB) ===
[info] saved results/gateway-h3/256/tako-gateway.json
==============================================
=== tako-gateway / production-stack / 256c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
==============================================
=== tako-gateway / production-stack / 1024c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
[info] skip: tako-gateway does not subscribe to unary-grpc
[info] skip: tako-gateway does not subscribe to unary-grpc-tls
[info] skip: tako-gateway does not subscribe to echo-ws
[info] skip: tako-gateway does not subscribe to echo-ws-pipeline
[info] skip: tako-gateway does not subscribe to echo-ws-limited
[info] skip: tako-gateway does not subscribe to latency-1m
[info] skip: tako-gateway does not subscribe to latency-10k
[info] skip: tako-gateway does not subscribe to latency-500k-8cpu
[info] skip: tako-gateway does not subscribe to async
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 25 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 4 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536
|
/benchmark-multiple -f tako,tako-grpc,tako-gateway -t baseline-h2,static-h2,baseline-h2c,json-h2c,unary-grpc,unary-grpc-tls,gateway-64,production-stack --save Updated against main and fixed client setup in 598c95a. The previous run rejected the old native h2load before these profiles could start. Default setup now selects a supported native client (including h2load-h3 in HTTP/2 mode) or prepares the pinned HTTP/2 Docker image before host tuning. Explicit H2LOAD/H2LOAD_CMD overrides still require a supported version. This reruns the blocked profiles and preserves the successful HTTP/1, HTTP/3 and WebSocket results from the previous run. |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
…son-h2c,unary-grpc,unary-grpc-tls,gateway-64,production-stack) [skip ci]
Benchmark ResultsFrameworks: 3 | Test: ✅
|
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| baseline-h2 | 256 | 872,980 | 4754.1% | 208MiB | +17.9% | -41.2% |
| baseline-h2 | 1024 | 719,930 | 4770.7% | 544MiB | +5.3% | -46.9% |
| static-h2 | 256 | 129,744 | 2846.6% | 337MiB | +0.6% | +0.6% |
| static-h2 | 1024 | 129,095 | 3172.9% | 977MiB | +2.2% | -4.6% |
| baseline-h2c | 256 | 865,081 | 4817.3% | 209MiB | +16.3% | -46.8% |
| baseline-h2c | 1024 | 734,010 | 4776.7% | 551MiB | +10.1% | -55.2% |
| baseline-h2c | 4096 | 690,356 | 4818.1% | 1.3GiB | +5.2% | -40.9% |
| json-h2c | 1024 | 606,563 | 4065.6% | 483MiB | -16.1% | +19.9% |
| json-h2c | 4096 | 522,847 | 3813.1% | 1.3GiB | -15.8% | +8.3% |
Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.
✅ tako-grpc
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| unary-grpc | 256 | 578,661 | 5128.5% | 201MiB | +0.1% | -44.0% |
| unary-grpc | 1024 | 493,020 | 5171.3% | 532MiB | -4.0% | -52.8% |
| unary-grpc-tls | 256 | 575,891 | 5188.0% | 226MiB | -7.5% | -28.9% |
| unary-grpc-tls | 1024 | 498,325 | 5139.5% | 584MiB | -2.1% | -43.0% |
Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.
❌ tako-gateway
| Test | Conn | RPS | CPU | Mem | Δ RPS | Δ Mem |
|---|---|---|---|---|---|---|
| gateway-64 | 512 | 76,150 | 3900.4% | 2.6GiB | -1.6% | -7.1% |
| gateway-64 | 1024 | 70,030 | 3831.0% | 5.4GiB | +1.7% | +1.9% |
| production-stack | 1024 | 47,003 | 3375.5% | 3.2GiB | -0.4% | +10.3% |
Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.
Log tail
21.
1.30Warm-up phase is over for thread #Main benchmark duration is started for thread #34.43.
.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2
0. Stopping all clients.
43
finished in 5.05s, 44929.80 req/s, 478.07MB/s
requests: 224649 total, 241033 started, 224649 done, 224649 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 225225 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 2.33GB (2506453027) total, 2.20MB (2305013) headers (space savings 93.02%), 2.33GB (2498621320) data
min max median p95 p99 mean sd +/- sd
request : 167us 1.60s 342.57ms 705.71ms 837.61ms 345.31ms 223.65ms 60.89%
connect : 4.33ms 899.65ms 49.59ms 280.85ms 556.28ms 99.70ms 112.52ms 88.96%
TTFB : 10.78ms 992.33ms 199.53ms 580.38ms 806.34ms 245.08ms 169.53ms 70.41%
req/s : 36.99 53.58 43.79 47.78 50.39 43.87 2.35 73.83%
=== h2load-writes ===
starting benchmark...
7.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2
finished in 5.00s, 2518.60 req/s, 35.16KB/s
requests: 12593 total, 13617 started, 12593 done, 12593 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 12593 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 175.80KB (180018) total, 55.14KB (56467) headers (space savings 91.85%), 0B (0) data
min max median p95 p99 mean sd +/- sd
request : 1.30ms 1.36s 433.08ms 711.80ms 839.12ms 392.66ms 229.42ms 64.22%
connect : 3.65ms 38.11ms 10.69ms 30.74ms 35.82ms 13.02ms 7.86ms 78.13%
TTFB : 5.86ms 272.07ms 21.87ms 49.15ms 271.90ms 27.25ms 33.19ms 97.66%
req/s : 15.60 23.60 19.79 22.20 23.20 19.67 1.49 71.09%
[info] CPU 3375.5% | Mem 3.2GiB
[info] server: 239% 381MiB | cache: 0% 4MiB | authsvc: 303% 289MiB | edge: 2834% 2.6GiB
=== Best: 47003 req/s (CPU: 3375.5%, Mem: 3.2GiB) ===
[info] saved results/production-stack/1024/tako-gateway.json
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 16 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 1 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536
|
/benchmark -f tako-gateway -t production-stack --save The previous run completed HTTP/2, gRPC and gateway-64 successfully, but production-stack reads returned intermittent non-2xx responses. Logs now come from the real Compose services and survive failed measurements. If every attempt fails, an additional replay reports exact HTTP statuses and does not publish a score. Local isolated runs completed 842,812 requests without reproducing the failure. |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
|
|
|
/benchmark -f tako-gateway -t production-stack --save Merged current main (35e43f2) into this branch. The previous precheck failed with |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Benchmark ResultsFramework: No results captured Full log |
|
/benchmark -f tako-gateway -t production-stack --save Retrying after #1810 was merged into main. The previous run captured HTTP 499 responses during the diagnostic replay; this rerun checks production-stack with the updated workflow. |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
|
|
Benchmark ResultsFramework: No results captured Full log |
|
/benchmark -f tako-gateway -t production-stack --save Merged current main and pushed it directly. The previous save failed because the bot lacked Added an unscored fixed-request control after failed production attempts: 100000 reads and 10000 writes, with in-flight requests allowed to finish. The original failure stays failed and the control never publishes a score. Locally, Caddy 2.11.9 / Go 1.27.3 with Go-only backends reproduced HTTP 499 on requests whose own contexts were still live when other requests were canceled. This matches golang/go#82547. The no-cancellation control and the real h2load fixed-request control completed without errors. This rerun compares timed shutdown with request draining on the benchmark host. |
|
👋 Benchmark request received. A collaborator will review and approve the run. |
Updates the HTTP, gRPC and gateway entries to Tako 2.7.1, including HTTP/3 cancellation fixes and patched QUIC dependencies.
Rejects failed or incomplete H2/H3 measurements, reports actual p99 separately from maximum latency, selects a supported HTTP/2 client, and prevents stale composite scores after failures.
Preserves logs from the actual Compose services. When every production-stack attempt fails, separate timed and fixed-request diagnostic replays report exact status codes without publishing scores. The fixed control drains 100000 reads and 10000 writes while preserving client wrappers and process limits.
Includes current main so the results bot does not need to push newly merged workflow files without workflow permissions.
Validation: parser, process-failure and Git merge regressions pass locally, as do the h2load Docker smoke check and a real 110000-request control. An isolated Caddy/Go setup reproduced HTTP 499 on requests with live contexts without Tako, consistent with golang/go#82547. The new remote control will test this against the benchmark host's failures.