Skip to content

fix: update Tako to 2.7.1 and validate benchmark results - #1762

Open
dancixx wants to merge 15 commits into
MDA2AV:mainfrom
dancixx:chore/tako-2.7.0
Open

dancixx wants to merge 15 commits into
MDA2AV:mainfrom
dancixx:chore/tako-2.7.0

Conversation

@dancixx

@dancixx dancixx commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Updates the HTTP, gRPC and gateway entries to Tako 2.7.1, including HTTP/3 cancellation fixes and patched QUIC dependencies.

Rejects failed or incomplete H2/H3 measurements, reports actual p99 separately from maximum latency, selects a supported HTTP/2 client, and prevents stale composite scores after failures.

Preserves logs from the actual Compose services. When every production-stack attempt fails, separate timed and fixed-request diagnostic replays report exact status codes without publishing scores. The fixed control drains 100000 reads and 10000 writes while preserving client wrappers and process limits.

Includes current main so the results bot does not need to push newly merged workflow files without workflow permissions.

Validation: parser, process-failure and Git merge regressions pass locally, as do the h2load Docker smoke check and a real 110000-request control. An isolated Caddy/Go setup reproduced HTTP 499 on requests with live contexts without Tako, consistent with golang/go#82547. The new remote control will test this against the benchmark host's failures.

@dancixx

dancixx commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark-multiple -f tako,tako-grpc,tako-gateway --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 3 | Test: all tests | Failed: 1

✅ tako

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
baseline 4096 3,139,709 — — — — 6652.2% 118MiB -4.3% -4.8%
pipelined 4096 20,509,219 — — — — 6354.0% 126MiB -8.4% +0.8%
limited-conn 4096 2,407,752 — — — — 6144.4% 154MiB -2.2% ~0%
json-comp 4096 208,402 — — — — 6229.1% 245MiB -1.3% ~0%
json-comp 16384 201,312 — — — — 6485.6% 594MiB -3.5% +0.2%
json-tls 4096 1,009,044 — — — — 6068.4% 222MiB +9.3% -24.2%
8gbit 512 48,980 0.9796 ⚠️ 53.24us 138.0us 2937.0us 253.5% 93MiB -0.2% -7.9%
static-tls 1024 140,842 — — — — 3229.7% 157MiB +4.8% -3.1%
async-db 1024 242,075 — — — — 3681.6% 96MiB -0.1% -1.0%
baseline-h2 256 829,933 — — — — 4755.5% 204MiB +12.1% -42.4%
baseline-h2 1024 724,529 — — — — 4806.1% 566MiB +6.0% -44.7%
static-h2 256 131,413 — — — — 3003.9% 318MiB +1.9% -5.1%
static-h2 1024 130,233 — — — — 2740.0% 1.0GiB +3.1% ~0%
baseline-h2c 256 848,574 — — — — 4676.3% 176MiB +14.0% -55.2%
baseline-h2c 1024 737,895 — — — — 4758.3% 624MiB +10.7% -49.2%
baseline-h2c 4096 0 — — — — 3869.5% 977MiB -100.0% -56.6%
json-h2c 1024 602,946 — — — — 4047.6% 551MiB -16.6% +36.7%
json-h2c 4096 531,594 — — — — 3839.8% 1.4GiB -14.4% +16.7%
baseline-h3 64 536,729 — — — — 2928.1% 107MiB -0.7% +9.2%
static-h3 64 107,569 — — — — 2817.0% 175MiB -7.7% +18.2%
echo-ws 512 4,191,358 — — — — 6465.1% 39MiB +2.4% +2.6%
echo-ws 4096 4,066,143 — — — — 6444.6% 121MiB ~0% -0.8%
echo-ws 16384 3,698,229 — — — — 5989.3% 388MiB +5.4% -0.5%
echo-ws-pipeline 512 57,721,923 — — — — 6603.9% 40MiB +903.4% ~0%
echo-ws-pipeline 4096 56,360,143 — — — — 6592.1% 122MiB +838.1% +1.7%
echo-ws-pipeline 16384 50,973,465 — — — — 6328.4% 354MiB +773.6% -8.3%
echo-ws-limited 512 2,156,338 — — — — 5894.3% 51MiB +0.3% ~0%
echo-ws-limited 4096 2,570,415 — — — — 6357.1% 136MiB +1.0% -2.2%
latency-1m 1024 997,908 0.9979 24.32us 184.0us 2637.0us 2672.1% 52MiB ~0% +2.0%
latency-10k 1024 9,983 0.9983 26.99us 108.0us 146.0us 25.8% 49MiB ~0% -2.0%
latency-500k-8cpu 1024 455,866 0.9117 ⚠️ 17.50us 1794560.0us 1925632.0us 800.2% 33MiB -1.0% -10.8%
async 32000 2,235,424 — — — — 5199.2% 966MiB -0.4% -0.3%

Composite score, main against this run, in the board's default view:

Family main this run Δ
HTTP/1.1 6635.6 (6 of 107) 6619.7 (6 of 107) -15.9
HTTP/2 568.3 (28 of 40) 516.1 (30 of 40) -52.2
HTTP/3 509.7 (11 of 19) 477.9 (11 of 19) -31.8
Gateway 896.2 (6 of 9) 896.2 (6 of 9) +0.0
gRPC 179.5 (13 of 13) 170.7 (13 of 13) -8.8
WebSocket 2075.9 (10 of 38) 2988.7 (1 of 38) +912.8

The results on main were measured in another session, so a few points up or down can be the machine and not the change.

✅ tako-grpc

Test Conn RPS CPU Mem Δ RPS Δ Mem
unary-grpc 256 554,095 5116.2% 196MiB -4.1% -45.4%
unary-grpc 1024 510,326 5211.5% 720MiB -0.6% -36.1%
unary-grpc-tls 256 564,324 5267.4% 220MiB -9.4% -30.8%
unary-grpc-tls 1024 485,284 5040.5% 582MiB -4.6% -43.2%

❌ tako-gateway

Test Conn RPS CPU Mem Δ RPS Δ Mem
gateway-64 512 75,631 3811.3% 2.7GiB -2.3% -3.6%
gateway-64 1024 64,716 3501.4% 5.7GiB -6.0% +7.5%
gateway-h3 64 81,376 3427.4% 416MiB -2.0% -2.6%
gateway-h3 256 57,583 2573.7% 1.5GiB -24.7% +174.3%
Log tail
#37 exporting manifest list sha256:ba368aae0b68fb70e66e2de42d06a35f82bffdf211899c62a9f3898067171b68 0.0s done
#37 naming to docker.io/library/httparena-tako-gateway-production-stack-authsvc:latest done
#37 unpacking to docker.io/library/httparena-tako-gateway-production-stack-authsvc:latest 0.0s done
#37 DONE 0.3s

#38 [edge] resolving provenance for metadata file
#38 DONE 0.0s

#39 [server] resolving provenance for metadata file
#39 DONE 0.1s

#40 [authsvc] resolving provenance for metadata file
#40 DONE 0.0s
 Image httparena-tako-gateway-production-stack-server Built 
 Image httparena-tako-gateway-production-stack-authsvc Built 
 Image httparena-tako-gateway-production-stack-edge Built 
 Container httparena-tako-gateway-production-stack-cache-1 Creating 
 Container httparena-tako-gateway-production-stack-authsvc-1 Creating 
 Container httparena-tako-gateway-production-stack-authsvc-1 Created 
 Container httparena-tako-gateway-production-stack-cache-1 Created 
 Container httparena-tako-gateway-production-stack-server-1 Creating 
 Container httparena-tako-gateway-production-stack-server-1 Created 
 Container httparena-tako-gateway-production-stack-edge-1 Creating 
 Container httparena-tako-gateway-production-stack-edge-1 Created 
 Container httparena-tako-gateway-production-stack-cache-1 Starting 
 Container httparena-tako-gateway-production-stack-authsvc-1 Starting 
 Container httparena-tako-gateway-production-stack-authsvc-1 Started 
 Container httparena-tako-gateway-production-stack-cache-1 Started 
 Container httparena-tako-gateway-production-stack-cache-1 Waiting 
 Container httparena-tako-gateway-production-stack-cache-1 Healthy 
 Container httparena-tako-gateway-production-stack-server-1 Starting 
 Container httparena-tako-gateway-production-stack-server-1 Started 
 Container httparena-tako-gateway-production-stack-authsvc-1 Waiting 
 Container httparena-tako-gateway-production-stack-authsvc-1 Error dependency authsvc failed to start
dependency failed to start: container httparena-tako-gateway-production-stack-authsvc-1 exited (0)

─── httparena-tako-gateway-production-stack-edge-1 — status=created exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-edge-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
  | 7:C 13 Nov 2026 11:33:47.352 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  | 7:C 13 Nov 2026 11:33:47.352 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  | 7:C 13 Nov 2026 11:33:47.352 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 13 Nov 2026 11:33:47.352 * Configuration loaded
  | 7:M 13 Nov 2026 11:33:47.352 * monotonic clock: POSIX clock_gettime
  | 7:M 13 Nov 2026 11:33:47.356 * Running mode=standalone, port=6379.
  | 7:M 13 Nov 2026 11:33:47.356 * Server initialized
  | 7:M 13 Nov 2026 11:33:47.357 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=exited exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — the container produced no output at all
[FAIL] gateway compose up failed
httparena-postgres
[info] restoring loopback MTU to 65536

Preserve load-generator failures, validate completed requests, and suppress stale composite scores after failed runs. Update h2load for duration-mode shutdown, fix auth sidecar build caching, and cover both benchmark runners with regression tests.
@dancixx dancixx changed the title chore: update tako to 2.7.0 fix: update Tako to 2.7.1 and validate benchmark results Nov 13, 2026
@dancixx

dancixx commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark-multiple -f tako,tako-grpc,tako-gateway --save

Rerun with Tako 2.7.1 and the benchmark fixes in this PR. Before approving, please ensure the runner's native h2load is >=1.69.0 (or set H2LOAD to an installed compatible h2load-h3); older clients can hang after the duration expires and are now rejected.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

MDA2AV added a commit that referenced this pull request Nov 13, 2026
…uthsvc on Rust 1.100, unbreak the aspnet gateway entries (#1767)

* validate.sh: stop the standalone container before the compose profiles

The standalone container runs on --network host whenever an entry
subscribes to a compose profile, and it was left running while the
gateway and production-stack stacks came up. An entry that also serves
h2/h3 keeps 8443 (and 8080), so the stack's edge failed to bind, shared
the port, or exited on the QUIC port, and the checks reached the
standalone server instead of the stack.

* authsvc: ship the real binary under cargo 1.100's build-dir layout

production-stack has failed on the bench host since the morning of
2026-11-13 on every entry (roadrunner-gateway #1757, morojs #1761,
tako-gateway #1762): the shared authsvc container exits 0 with no output,
the edge never sees it healthy, and the run ends with no results.

rust:1-alpine moved to Rust 1.100 on 2026-11-12. Its cargo keeps each
package's artifacts under target/release/build/<package>/<hash>/ rather
than target/release/deps/, so the cache-priming step's
`rm -rf target/release/deps/authsvc*` stopped matching anything. The
placeholder built from `fn main() {}` stayed up to date, the second
`cargo build` finished in 0.04s without compiling src/, and the image
shipped the placeholder: a 418 KB binary that exits 0 at once.

`cargo clean --release -p authsvc` removes the placeholder's artifacts
wherever cargo keeps them, and a grep for the server's startup line fails
the build if the placeholder is ever shipped again. Built against the
current rust:1-alpine (1.100.0): a 1.2 MB binary that logs
"authsvc listening on ...", answers /_health with 200 and an /_auth
without a token with 401.

* aspnet-minimal_nginx, aspnet-minimal_caddy: build against aspnet-minimal's Services and Types

Both Dockerfiles still COPY aspnet-minimal's AppData.cs and Models.cs,
which #1002 (2026-07-18) split into Services/ and Types/, so neither image
has built since. They were the only failures in #1615's validation sweep,
and their published results have not been refreshed since July.

Copy Handlers.cs with Services/ and Types/, register DatabaseService,
DatasetService, ItemService and the AppJsonContext resolver the way
aspnet-minimal's Program.cs does, and give the nginx entry's /api routes
DatabaseService's Postgres pool in place of AppData.PgDataSource. The
caddy entry gains the StackExchange.Redis reference DatabaseService needs;
it connects to nothing unless REDIS_URL is set, which gateway-h3 never
does.

---------

Co-authored-by: Jacob Rothstein <hi@jbr.me>
@MDA2AV

MDA2AV commented Nov 13, 2026

Copy link
Copy Markdown
Owner

The authsvc fix is now on main in #1767. It's the same cargo clean --release -p authsvc change as your 6b2d03b, plus a build-time check that fails if the image ever ships the placeholder again.

Because both changes edit the same lines, this branch now conflicts with main in frameworks/_shared/authsvc/Dockerfile. A --save run merges main in first and would stop at that conflict. Taking main's version of that file when you merge or rebase main resolves it. Nothing else here conflicts with main. After that, your re-run will build the fixed authsvc.

# Conflicts:
#	frameworks/_shared/authsvc/Dockerfile
@MDA2AV

MDA2AV commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Update: I merged main into the branch (6bb3c3b) and took main's authsvc Dockerfile, so you don't need to resolve that conflict. Pull before your next push.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 3 | Test: all tests | Failed: 3

❌ tako

Test Conn RPS Rate CPU/req p99 p99.9 CPU Mem Δ RPS Δ Mem
baseline 4096 3,201,484 — — — — 6335.0% 124MiB -2.5% ~0%
pipelined 4096 20,799,968 — — — — 6767.5% 121MiB -7.1% -3.2%
limited-conn 4096 2,421,703 — — — — 6447.1% 154MiB -1.6% ~0%
json-comp 4096 204,795 — — — — 6478.6% 243MiB -3.0% -0.8%
json-comp 16384 204,554 — — — — 6501.0% 592MiB -1.9% -0.2%
json-tls 4096 1,001,696 — — — — 5949.9% 224MiB +8.5% -23.5%
8gbit 512 48,904 0.9781 ⚠️ 52.03us 135.0us 7498.0us 252.0% 93MiB -0.4% -7.9%
static-tls 1024 136,403 — — — — 3608.4% 160MiB +1.5% -1.2%
async-db 1024 235,674 — — — — 3547.8% 97MiB -2.8% ~0%
baseline-h3 64 534,875 — — — — 3061.3% 102MiB -1.0% +4.1%
static-h3 64 113,798 — — — — 3298.6% 172MiB -2.4% +16.2%
echo-ws 512 4,120,095 — — — — 6328.1% 40MiB +0.7% +5.3%
echo-ws 4096 4,094,900 — — — — 6056.1% 113MiB +0.8% -7.4%
echo-ws 16384 3,711,259 — — — — 6101.8% 394MiB +5.8% +1.0%
echo-ws-pipeline 512 57,394,140 — — — — 6602.8% 41MiB +897.7% +2.5%
echo-ws-pipeline 4096 55,968,937 — — — — 6260.2% 114MiB +831.6% -5.0%
echo-ws-pipeline 16384 51,593,811 — — — — 6078.2% 364MiB +784.2% -5.7%
echo-ws-limited 512 2,143,254 — — — — 5859.9% 50MiB -0.3% -2.0%
echo-ws-limited 4096 2,573,728 — — — — 6101.9% 139MiB +1.2% ~0%
latency-1m 1024 997,802 0.9978 24.09us 180.0us 3041.0us 2636.7% 49MiB ~0% -3.9%
latency-10k 1024 9,982 0.9982 27.90us 105.0us 141.0us 26.6% 50MiB ~0% ~0%
latency-500k-8cpu 1024 464,802 0.9296 ⚠️ 17.15us 1362432.0us 1405440.0us 795.7% 33MiB +0.9% -10.8%
async 32000 2,236,354 — — — — 5165.3% 921MiB -0.4% -5.0%

Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.

Log tail
  Throughput: 2.13M req/s
  Bandwidth:  239.29MB/s
  Status codes: 2xx=21267553, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 21267553 / 21267553 responses (100.0%)
[info] CPU 4812.2% | Mem 757MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     32000 (500/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   14.19ms   14.00ms   15.80ms   17.70ms   19.60ms

  22395542 requests in 10.00s, 22363542 responses
  Throughput: 2.24M req/s
  Bandwidth:  251.62MB/s
  Status codes: 2xx=22363542, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 22363542 / 22363542 responses (100.0%)
[info] CPU 5165.3% | Mem 921MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     32000 (500/thread)
  Pipeline:  1
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   14.20ms   14.00ms   15.80ms   17.70ms   19.80ms

  22376767 requests in 10.00s, 22344767 responses
  Throughput: 2.23M req/s
  Bandwidth:  251.41MB/s
  Status codes: 2xx=22344767, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 22344767 / 22344767 responses (100.0%)
[info] CPU 5180.0% | Mem 972MiB

=== Best: 2236354 req/s (CPU: 5165.3%, Mem: 921MiB) ===
[info] input BW: 102.37MB/s (avg template: 48 bytes)
[info] saved results/async/32000/tako.json
httparena-bench-tako
httparena-bench-tako
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 23 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 9 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536

❌ tako-grpc

No results captured.

Log tail
#13 unpacking to docker.io/library/httparena-tako-grpc:latest 0.0s done
#13 DONE 0.5s
[info] tuning host for benchmark runs
[info] CPU governor → performance
[info] setting kernel socket limits
[info] setting UDP buffer sizes for QUIC
[info] setting loopback MTU to 1500 (realistic Ethernet)
[info] restarting docker daemon
[info] dropping kernel caches
[info] skip: tako-grpc does not subscribe to baseline
[info] skip: tako-grpc does not subscribe to pipelined
[info] skip: tako-grpc does not subscribe to limited-conn
[info] skip: tako-grpc does not subscribe to json-comp
[info] skip: tako-grpc does not subscribe to json-tls
[info] skip: tako-grpc does not subscribe to 8gbit
[info] skip: tako-grpc does not subscribe to static-tls
[info] skip: tako-grpc does not subscribe to async-db
[info] skip: tako-grpc does not subscribe to fortunes
[info] skip: tako-grpc does not subscribe to baseline-h2
[info] skip: tako-grpc does not subscribe to static-h2
[info] skip: tako-grpc does not subscribe to baseline-h2c
[info] skip: tako-grpc does not subscribe to json-h2c
[info] skip: tako-grpc does not subscribe to baseline-h3
[info] skip: tako-grpc does not subscribe to static-h3
[info] skip: tako-grpc does not subscribe to gateway-64
[info] skip: tako-grpc does not subscribe to gateway-h3
[info] skip: tako-grpc does not subscribe to production-stack

==============================================
=== tako-grpc / unary-grpc / 256c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.

==============================================
=== tako-grpc / unary-grpc / 1024c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.

==============================================
=== tako-grpc / unary-grpc-tls / 256c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.

==============================================
=== tako-grpc / unary-grpc-tls / 1024c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
[info] skip: tako-grpc does not subscribe to echo-ws
[info] skip: tako-grpc does not subscribe to echo-ws-pipeline
[info] skip: tako-grpc does not subscribe to echo-ws-limited
[info] skip: tako-grpc does not subscribe to latency-1m
[info] skip: tako-grpc does not subscribe to latency-10k
[info] skip: tako-grpc does not subscribe to latency-500k-8cpu
[info] skip: tako-grpc does not subscribe to async
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 23 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 4 profile/connection measurements failed
[info] restoring loopback MTU to 65536

❌ tako-gateway

Test Conn RPS CPU Mem Δ RPS Δ Mem
gateway-h3 64 81,857 3477.9% 488MiB -1.4% +14.3%
gateway-h3 256 58,187 2636.9% 1.5GiB -23.9% +174.3%

Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.

Log tail
59
.

41.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h3
58. Stopping all clients.

finished in 5.04s, 58652.60 req/s, 626.06MB/s
requests: 293263 total, 301455 started, 293263 done, 293263 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 298418 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 3.06GB (3282372703) total, 21.17MB (22193977) headers (space savings 50.52%), 3.03GB (3258297580) data
UDP datagram: 94869 sent, 2356090 received
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :      982us       1.55s     68.18ms   539.61ms   799.05ms   134.06ms    172.77ms    87.08%
connect     :     4.20ms    129.26ms     14.28ms    72.38ms   103.43ms    27.78ms     23.83ms    83.59%
TTFB        :     6.70ms    194.78ms     49.98ms   146.65ms   178.92ms    59.31ms     40.35ms    70.70%
req/s       :     126.38      418.11      222.36     308.96     365.55     229.08       44.68    69.53%
min RTT     :       11us      1.60ms        50us      218us      919us       93us       151us    96.48%
smoothed RTT:    12.48ms     63.10ms     28.42ms    46.93ms    59.87ms    29.64ms      8.77ms    75.78%
packets sent:        295         464         373        414        442     372.58       27.53    68.75%
packets recv:       5008       16937        8923      12485      14782    9205.48     1822.22    69.53%
packets lost:          0           0           0          0          0       0.00        0.00   100.00%
GRO packets :          1          17           4         14         14       6.38        5.39    71.02%
[info] CPU 2636.9% | Mem 1.5GiB
[info]   proxy: 2062% 1.3GiB | server: 575% 130MiB

=== Best: 58187 req/s (CPU: 2636.9%, Mem: 1.5GiB) ===
[info] saved results/gateway-h3/256/tako-gateway.json

==============================================
=== tako-gateway / production-stack / 256c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.

==============================================
=== tako-gateway / production-stack / 1024c (tool=h2load) ===
==============================================
h2load >= 1.69.0 is required: older clients can run forever after -D expires during connection setup. Update H2LOAD or build docker/h2load.Dockerfile.
[info] skip: tako-gateway does not subscribe to unary-grpc
[info] skip: tako-gateway does not subscribe to unary-grpc-tls
[info] skip: tako-gateway does not subscribe to echo-ws
[info] skip: tako-gateway does not subscribe to echo-ws-pipeline
[info] skip: tako-gateway does not subscribe to echo-ws-limited
[info] skip: tako-gateway does not subscribe to latency-1m
[info] skip: tako-gateway does not subscribe to latency-10k
[info] skip: tako-gateway does not subscribe to latency-500k-8cpu
[info] skip: tako-gateway does not subscribe to async
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 25 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 4 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536

@dancixx

dancixx commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark-multiple -f tako,tako-grpc,tako-gateway -t baseline-h2,static-h2,baseline-h2c,json-h2c,unary-grpc,unary-grpc-tls,gateway-64,production-stack --save

Updated against main and fixed client setup in 598c95a. The previous run rejected the old native h2load before these profiles could start. Default setup now selects a supported native client (including h2load-h3 in HTTP/2 mode) or prepares the pinned HTTP/2 Docker image before host tuning. Explicit H2LOAD/H2LOAD_CMD overrides still require a supported version.

This reruns the blocked profiles and preserves the successful HTTP/1, HTTP/3 and WebSocket results from the previous run.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

…son-h2c,unary-grpc,unary-grpc-tls,gateway-64,production-stack) [skip ci]
@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Frameworks: 3 | Test: baseline-h2,static-h2,baseline-h2c,json-h2c,unary-grpc,unary-grpc-tls,gateway-64,production-stack | Failed: 1

✅ tako

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline-h2 256 872,980 4754.1% 208MiB +17.9% -41.2%
baseline-h2 1024 719,930 4770.7% 544MiB +5.3% -46.9%
static-h2 256 129,744 2846.6% 337MiB +0.6% +0.6%
static-h2 1024 129,095 3172.9% 977MiB +2.2% -4.6%
baseline-h2c 256 865,081 4817.3% 209MiB +16.3% -46.8%
baseline-h2c 1024 734,010 4776.7% 551MiB +10.1% -55.2%
baseline-h2c 4096 690,356 4818.1% 1.3GiB +5.2% -40.9%
json-h2c 1024 606,563 4065.6% 483MiB -16.1% +19.9%
json-h2c 4096 522,847 3813.1% 1.3GiB -15.8% +8.3%

Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.

✅ tako-grpc

Test Conn RPS CPU Mem Δ RPS Δ Mem
unary-grpc 256 578,661 5128.5% 201MiB +0.1% -44.0%
unary-grpc 1024 493,020 5171.3% 532MiB -4.0% -52.8%
unary-grpc-tls 256 575,891 5188.0% 226MiB -7.5% -28.9%
unary-grpc-tls 1024 498,325 5139.5% 584MiB -2.1% -43.0%

Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.

❌ tako-gateway

Test Conn RPS CPU Mem Δ RPS Δ Mem
gateway-64 512 76,150 3900.4% 2.6GiB -1.6% -7.1%
gateway-64 1024 70,030 3831.0% 5.4GiB +1.7% +1.9%
production-stack 1024 47,003 3375.5% 3.2GiB -0.4% +10.3%

Composite score unavailable: related measurements failed or did not finish; retained results are not a complete new run.

Log tail
21.
1.30Warm-up phase is over for thread #Main benchmark duration is started for thread #34.43.


.

TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2


0. Stopping all clients.

43

finished in 5.05s, 44929.80 req/s, 478.07MB/s
requests: 224649 total, 241033 started, 224649 done, 224649 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 225225 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 2.33GB (2506453027) total, 2.20MB (2305013) headers (space savings 93.02%), 2.33GB (2498621320) data
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :      167us       1.60s    342.57ms   705.71ms   837.61ms   345.31ms    223.65ms    60.89%
connect     :     4.33ms    899.65ms     49.59ms   280.85ms   556.28ms    99.70ms    112.52ms    88.96%
TTFB        :    10.78ms    992.33ms    199.53ms   580.38ms   806.34ms   245.08ms    169.53ms    70.41%
req/s       :      36.99       53.58       43.79      47.78      50.39      43.87        2.35    73.83%
=== h2load-writes ===
starting benchmark...
7.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2

finished in 5.00s, 2518.60 req/s, 35.16KB/s
requests: 12593 total, 13617 started, 12593 done, 12593 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 12593 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 175.80KB (180018) total, 55.14KB (56467) headers (space savings 91.85%), 0B (0) data
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :     1.30ms       1.36s    433.08ms   711.80ms   839.12ms   392.66ms    229.42ms    64.22%
connect     :     3.65ms     38.11ms     10.69ms    30.74ms    35.82ms    13.02ms      7.86ms    78.13%
TTFB        :     5.86ms    272.07ms     21.87ms    49.15ms   271.90ms    27.25ms     33.19ms    97.66%
req/s       :      15.60       23.60       19.79      22.20      23.20      19.67        1.49    71.09%
[info] CPU 3375.5% | Mem 3.2GiB
[info]   server: 239% 381MiB | cache: 0% 4MiB | authsvc: 303% 289MiB | edge: 2834% 2.6GiB

=== Best: 47003 req/s (CPU: 3375.5%, Mem: 3.2GiB) ===
[info] saved results/production-stack/1024/tako-gateway.json
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/tako.json - 16 new, 42 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 1 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536

@dancixx

dancixx commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f tako-gateway -t production-stack --save

The previous run completed HTTP/2, gRPC and gateway-64 successfully, but production-stack reads returned intermittent non-2xx responses. Logs now come from the real Compose services and survive failed measurements. If every attempt fails, an additional replay reports exact HTTP statuses and does not publish a score. Local isolated runs completed 842,812 requests without reproducing the failure.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ /benchmark --save cannot start: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.

@dancixx

dancixx commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f tako-gateway -t production-stack --save

Merged current main (35e43f2) into this branch. The previous precheck failed with fatal: refusing to merge unrelated histories: git fetch origin main --depth=1 truncated the ancestry after a full checkout. This PR also removes those shallow fetches, with real Git regressions covering compatible divergence and genuine conflicts. All 34 tests pass locally. The workflow correction will take effect once merged into main; the synchronized branch is ready for this retry.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: tako-gateway | Test: production-stack

No results captured

Full log
  | {"level":"info","ts":1794779855.6808484,"msg":"GOMEMLIMIT is updated","GOMEMLIMIT":243039891456,"previous":9223372036854775807}
  | {"level":"info","ts":1794779855.680851,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
  | {"level":"info","ts":1794779855.6808524,"msg":"adapted config to JSON","adapter":"caddyfile"}
  | {"level":"warn","ts":1794779855.680855,"logger":"admin","msg":"admin endpoint disabled"}
  | {"level":"info","ts":1794779855.6810577,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x6f32cf1f780"}
  | {"level":"info","ts":1794779855.681719,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
  | {"level":"info","ts":1794779855.6825063,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2"]}
  | {"level":"info","ts":1794779855.6903713,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
  | {"level":"info","ts":1794779855.6903763,"msg":"serving initial configuration"}
  | {"level":"info","ts":1794779855.6973524,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
  | {"level":"info","ts":1794779855.7004461,"logger":"tls","msg":"finished cleaning storage units"}
─── httparena-tako-gateway-production-stack-edge-1 — end of logs ───

─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
  | 7:C 15 Nov 2026 21:57:33.387 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  | 7:C 15 Nov 2026 21:57:33.387 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  | 7:C 15 Nov 2026 21:57:33.387 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 15 Nov 2026 21:57:33.387 * Configuration loaded
  | 7:M 15 Nov 2026 21:57:33.387 * monotonic clock: POSIX clock_gettime
  | 7:M 15 Nov 2026 21:57:33.388 * Running mode=standalone, port=6379.
  | 7:M 15 Nov 2026 21:57:33.389 * Server initialized
  | 7:M 15 Nov 2026 21:57:33.389 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — last 120 log lines ───
  | authsvc listening on 0.0.0.0:9090 (JWT HMAC-SHA256)
─── httparena-tako-gateway-production-stack-authsvc-1 — end of logs ───

[run 3/3]
=== h2load-reads ===
starting benchmark...
progress: Main benchmark duration is started for thread #23.
68% of clients started
25Main benchmark duration is started for thread #54.
1029..

.Main benchmark duration is started for thread #Main benchmark duration is started for thread #58.

53Main benchmark duration is started for thread #21.
.
.

TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2
.Main benchmark duration is started for thread #36.
56Warm-up phase is over for thread #Warm-up phase is over for thread #Warm-up phase is over for thread #Main benchmark duration is started for thread #3748.
45.
.
37.
42.

39Main benchmark duration is started for thread #19.
.

21. Stopping all clients.
21
53. Stopping all clients.
51

finished in 5.06s, 43010.00 req/s, 458.80MB/s
requests: 215050 total, 231434 started, 215050 done, 215043 succeeded, 7 failed, 0 errored, 0 timeout
status codes: 215513 2xx, 0 3xx, 7 4xx, 0 5xx
traffic: 2.24GB (2405445530) total, 2.13MB (2230165) headers (space savings 92.94%), 2.23GB (2397919323) data
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :      198us       1.95s    362.18ms   765.59ms      1.06s   355.10ms    243.52ms    68.55%
connect     :     4.35ms       1.01s     45.85ms   716.01ms   820.25ms   124.84ms    201.26ms    90.04%
TTFB        :    10.56ms       1.07s    196.11ms   791.74ms   838.79ms   279.40ms    235.88ms    77.83%
req/s       :      33.39       52.17       42.19      45.99      48.18      41.99        2.99    64.94%
=== h2load-writes ===
starting benchmark...
3.

TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2


finished in 5.01s, 2530.40 req/s, 35.37KB/s
requests: 12652 total, 13676 started, 12652 done, 12652 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 12652 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 176.83KB (181071) total, 55.69KB (57028) headers (space savings 91.80%), 0B (0) data
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :     1.17ms       1.90s    421.22ms   763.65ms      1.05s   382.92ms    254.21ms    65.80%
connect     :     3.76ms     64.11ms     16.76ms    49.66ms    61.82ms    19.91ms     14.33ms    75.78%
TTFB        :     6.81ms    390.56ms     25.17ms    74.46ms   388.33ms    39.25ms     62.74ms    95.31%
req/s       :      14.00       25.20       19.40      23.60      24.80      19.76        2.60    67.19%
[info] CPU 3242.4% | Mem 3.2GiB
[info]   server: 211% 343MiB | cache: 0% 5MiB | authsvc: 259% 257MiB | edge: 2771% 2.6GiB
invalid h2load measurement: unsuccessful requests: 7 failed, 0 errored, 0 timeout
[warn] run 3 failed: invalid load-generator output

─── httparena-tako-gateway-production-stack-edge-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-edge-1 — last 120 log lines ───
  | {"level":"info","ts":1794779855.6808357,"msg":"maxprocs: Leaving GOMAXPROCS=30: CPU quota undefined"}
  | {"level":"info","ts":1794779855.6808484,"msg":"GOMEMLIMIT is updated","GOMEMLIMIT":243039891456,"previous":9223372036854775807}
  | {"level":"info","ts":1794779855.680851,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
  | {"level":"info","ts":1794779855.6808524,"msg":"adapted config to JSON","adapter":"caddyfile"}
  | {"level":"warn","ts":1794779855.680855,"logger":"admin","msg":"admin endpoint disabled"}
  | {"level":"info","ts":1794779855.6810577,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x6f32cf1f780"}
  | {"level":"info","ts":1794779855.681719,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
  | {"level":"info","ts":1794779855.6825063,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2"]}
  | {"level":"info","ts":1794779855.6903713,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
  | {"level":"info","ts":1794779855.6903763,"msg":"serving initial configuration"}
  | {"level":"info","ts":1794779855.6973524,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
  | {"level":"info","ts":1794779855.7004461,"logger":"tls","msg":"finished cleaning storage units"}
─── httparena-tako-gateway-production-stack-edge-1 — end of logs ───

─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
  | 7:C 15 Nov 2026 21:57:33.387 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  | 7:C 15 Nov 2026 21:57:33.387 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  | 7:C 15 Nov 2026 21:57:33.387 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 15 Nov 2026 21:57:33.387 * Configuration loaded
  | 7:M 15 Nov 2026 21:57:33.387 * monotonic clock: POSIX clock_gettime
  | 7:M 15 Nov 2026 21:57:33.388 * Running mode=standalone, port=6379.
  | 7:M 15 Nov 2026 21:57:33.389 * Server initialized
  | 7:M 15 Nov 2026 21:57:33.389 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — last 120 log lines ───
  | authsvc listening on 0.0.0.0:9090 (JWT HMAC-SHA256)
─── httparena-tako-gateway-production-stack-authsvc-1 — end of logs ───
[warn] production-stack/1024: no valid measurements; no result saved
[diagnostic] replaying production traffic with per-request status logging; excluded from results
[diagnostic] reads: finished in 5.07s, 46023.60 req/s, 490.29MB/s
[diagnostic] reads: requests: 230118 total, 246502 started, 230118 done, 230095 succeeded, 23 failed, 0 errored, 0 timeout
[diagnostic] reads: status codes: 230315 2xx, 0 3xx, 23 4xx, 0 5xx
[diagnostic] writes: finished in 5.00s, 2683.20 req/s, 36.66KB/s
[diagnostic] writes: requests: 13416 total, 14440 started, 13416 done, 13416 succeeded, 0 failed, 0 errored, 0 timeout
[diagnostic] writes: status codes: 13416 2xx, 0 3xx, 0 4xx, 0 5xx
[diagnostic] reads: HTTP 200=230093, HTTP 499=23
[diagnostic] reads: start_us=1794779878861434 status=499 duration_us=293067
[diagnostic] reads: start_us=1794779878915319 status=499 duration_us=246077
[diagnostic] reads: start_us=1794779878922604 status=499 duration_us=250943
[diagnostic] reads: start_us=1794779879090821 status=499 duration_us=90297
[diagnostic] reads: start_us=1794779878814489 status=499 duration_us=371315
[diagnostic] writes: HTTP 204=13416

─── httparena-tako-gateway-production-stack-edge-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-edge-1 — last 120 log lines ───
  | {"level":"info","ts":1794779855.6808357,"msg":"maxprocs: Leaving GOMAXPROCS=30: CPU quota undefined"}
  | {"level":"info","ts":1794779855.6808484,"msg":"GOMEMLIMIT is updated","GOMEMLIMIT":243039891456,"previous":9223372036854775807}
  | {"level":"info","ts":1794779855.680851,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
  | {"level":"info","ts":1794779855.6808524,"msg":"adapted config to JSON","adapter":"caddyfile"}
  | {"level":"warn","ts":1794779855.680855,"logger":"admin","msg":"admin endpoint disabled"}
  | {"level":"info","ts":1794779855.6810577,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x6f32cf1f780"}
  | {"level":"info","ts":1794779855.681719,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
  | {"level":"info","ts":1794779855.6825063,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2"]}
  | {"level":"info","ts":1794779855.6903713,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
  | {"level":"info","ts":1794779855.6903763,"msg":"serving initial configuration"}
  | {"level":"info","ts":1794779855.6973524,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
  | {"level":"info","ts":1794779855.7004461,"logger":"tls","msg":"finished cleaning storage units"}
─── httparena-tako-gateway-production-stack-edge-1 — end of logs ───

─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
  | 7:C 15 Nov 2026 21:57:33.387 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  | 7:C 15 Nov 2026 21:57:33.387 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  | 7:C 15 Nov 2026 21:57:33.387 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 15 Nov 2026 21:57:33.387 * Configuration loaded
  | 7:M 15 Nov 2026 21:57:33.387 * monotonic clock: POSIX clock_gettime
  | 7:M 15 Nov 2026 21:57:33.388 * Running mode=standalone, port=6379.
  | 7:M 15 Nov 2026 21:57:33.389 * Server initialized
  | 7:M 15 Nov 2026 21:57:33.389 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — last 120 log lines ───
  | authsvc listening on 0.0.0.0:9090 (JWT HMAC-SHA256)
─── httparena-tako-gateway-production-stack-authsvc-1 — end of logs ───
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 2 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536

@dancixx

dancixx commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f tako-gateway -t production-stack --save

Retrying after #1810 was merged into main. The previous run captured HTTP 499 responses during the diagnostic replay; this rerun checks production-stack with the updated workflow.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ This run's results were not saved: the push to dancixx/HttpArena was refused, most likely a fork that does not allow edits from maintainers. Nothing was committed, and the board keeps the numbers it has. Save them with a new run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: tako-gateway | Test: production-stack

No results captured

Full log
  | 7:C 17 Nov 2026 23:06:43.439 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 17 Nov 2026 23:06:43.439 * Configuration loaded
  | 7:M 17 Nov 2026 23:06:43.439 * monotonic clock: POSIX clock_gettime
  | 7:M 17 Nov 2026 23:06:43.440 * Running mode=standalone, port=6379.
  | 7:M 17 Nov 2026 23:06:43.440 * Server initialized
  | 7:M 17 Nov 2026 23:06:43.441 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — last 120 log lines ───
  | authsvc listening on 0.0.0.0:9090 (JWT HMAC-SHA256)
─── httparena-tako-gateway-production-stack-authsvc-1 — end of logs ───

[run 3/3]
=== h2load-reads ===
starting benchmark...
58.
.Main benchmark duration is started for thread #13.

.



59.
.
43Main benchmark duration is started for thread #39.

48.
55Main benchmark duration is started for thread #29.
57.
.49Main benchmark duration is started for thread #57.Main benchmark duration is started for thread #33Main benchmark duration is started for thread #50.63Main benchmark duration is started for thread #6.
18..

.Main benchmark duration is started for thread #49.

.


progress: 9.

37% of clients started
.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2
23Main benchmark duration is over for thread #18. Stopping all clients.

46Main benchmark duration is over for thread #63. Stopping all clients.

37Stopped all clients for thread #

61Stopped all clients for thread #46
18. Stopping all clients.

54. Stopping all clients.
6Stopped all clients for thread #. Stopping all clients.
5Stopped all clients for thread #6


finished in 5.06s, 44573.40 req/s, 474.95MB/s
requests: 222867 total, 239251 started, 222867 done, 222859 succeeded, 8 failed, 0 errored, 0 timeout
status codes: 223638 2xx, 0 3xx, 8 4xx, 0 5xx
traffic: 2.32GB (2490129835) total, 2.19MB (2295841) headers (space savings 92.99%), 2.31GB (2482346513) data
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :      211us       1.83s    336.76ms   770.82ms      1.03s   348.16ms    242.45ms    65.08%
connect     :     3.81ms    468.78ms     49.39ms   186.87ms   265.77ms    72.96ms     63.47ms    77.05%
TTFB        :    11.57ms    626.95ms    141.51ms   389.78ms   437.62ms   177.65ms    102.62ms    70.02%
req/s       :      37.19       54.99       43.59      46.79      49.19      43.52        2.16    70.70%
=== h2load-writes ===
starting benchmark...
7Main benchmark duration is started for thread #2.
75% of clients started
.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Certificate: RSA 2048 bits
Negotiated Group: x25519
Resumption: no
Application protocol: h2

finished in 5.01s, 2572.40 req/s, 35.79KB/s
requests: 12862 total, 13886 started, 12862 done, 12862 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 12862 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 178.96KB (183259) total, 55.93KB (57274) headers (space savings 91.90%), 0B (0) data
                 min         max         median     p95        p99        mean         sd        +/- sd
request     :      848us       1.70s    421.87ms   798.18ms      1.05s   377.10ms    266.72ms    60.43%
connect     :     3.85ms     26.13ms     11.81ms    23.29ms    25.52ms    12.42ms      5.86ms    60.16%
TTFB        :     6.07ms    169.58ms     22.98ms    38.23ms    43.29ms    23.37ms     15.78ms    89.84%
req/s       :      15.60       24.00       20.00      22.60      23.59      20.09        1.49    66.41%
[info] CPU 3287.0% | Mem 3.0GiB
[info]   server: 176% 322MiB | cache: 0% 4MiB | authsvc: 292% 276MiB | edge: 2818% 2.4GiB
invalid h2load measurement: unsuccessful requests: 8 failed, 0 errored, 0 timeout
[warn] run 3 failed: invalid load-generator output

─── httparena-tako-gateway-production-stack-edge-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-edge-1 — last 120 log lines ───
  | {"level":"info","ts":1794956805.706405,"msg":"maxprocs: Leaving GOMAXPROCS=30: CPU quota undefined"}
  | {"level":"info","ts":1794956805.7064176,"msg":"GOMEMLIMIT is updated","GOMEMLIMIT":243039533875,"previous":9223372036854775807}
  | {"level":"info","ts":1794956805.70642,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
  | {"level":"info","ts":1794956805.7064219,"msg":"adapted config to JSON","adapter":"caddyfile"}
  | {"level":"warn","ts":1794956805.7064242,"logger":"admin","msg":"admin endpoint disabled"}
  | {"level":"info","ts":1794956805.7066183,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x1130cfb3d200"}
  | {"level":"info","ts":1794956805.7067747,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
  | {"level":"info","ts":1794956805.7074828,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2"]}
  | {"level":"info","ts":1794956805.7170897,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
  | {"level":"info","ts":1794956805.717095,"msg":"serving initial configuration"}
  | {"level":"info","ts":1794956805.7198782,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
  | {"level":"info","ts":1794956805.722965,"logger":"tls","msg":"finished cleaning storage units"}
  | {"level":"error","ts":1794956812.9652622,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: http: putIdleConn: too many idle connections for host","request":{"remote_ip":"127.0.0.1","remote_port":"10414","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"localhost:8443","uri":"/api/items/99","headers":{"User-Agent":["h2load nghttp2/1.69.0-DEV"],"Accept-Encoding":["br;q=1, gzip;q=0.8"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost","ech":false}},"duration":0.321880164,"status":502,"err_id":"zwq5y53mm","err_trace":"reverseproxy.statusError (reverseproxy.go:1776)"}
  | {"level":"error","ts":1794956812.968693,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: http: putIdleConn: too many idle connections for host","request":{"remote_ip":"127.0.0.1","remote_port":"11632","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"localhost:8443","uri":"/api/items/98","headers":{"User-Agent":["h2load nghttp2/1.69.0-DEV"],"Accept-Encoding":["br;q=1, gzip;q=0.8"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost","ech":false}},"duration":0.239717359,"status":502,"err_id":"7yhc6cmnw","err_trace":"reverseproxy.statusError (reverseproxy.go:1776)"}
─── httparena-tako-gateway-production-stack-edge-1 — end of logs ───

─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
  | 7:C 17 Nov 2026 23:06:43.439 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  | 7:C 17 Nov 2026 23:06:43.439 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  | 7:C 17 Nov 2026 23:06:43.439 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 17 Nov 2026 23:06:43.439 * Configuration loaded
  | 7:M 17 Nov 2026 23:06:43.439 * monotonic clock: POSIX clock_gettime
  | 7:M 17 Nov 2026 23:06:43.440 * Running mode=standalone, port=6379.
  | 7:M 17 Nov 2026 23:06:43.440 * Server initialized
  | 7:M 17 Nov 2026 23:06:43.441 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — last 120 log lines ───
  | authsvc listening on 0.0.0.0:9090 (JWT HMAC-SHA256)
─── httparena-tako-gateway-production-stack-authsvc-1 — end of logs ───
[warn] production-stack/1024: no valid measurements; no result saved
[diagnostic] replaying production traffic with per-request status logging; excluded from results
[diagnostic] reads: finished in 5.07s, 44023.00 req/s, 474.55MB/s
[diagnostic] reads: requests: 220115 total, 236499 started, 220115 done, 220084 succeeded, 31 failed, 0 errored, 0 timeout
[diagnostic] reads: status codes: 220423 2xx, 0 3xx, 31 4xx, 0 5xx
[diagnostic] writes: finished in 5.00s, 2375.20 req/s, 32.90KB/s
[diagnostic] writes: requests: 11876 total, 12900 started, 11876 done, 11876 succeeded, 0 failed, 0 errored, 0 timeout
[diagnostic] writes: status codes: 11876 2xx, 0 3xx, 0 4xx, 0 5xx
[diagnostic] reads: HTTP 200=220084, HTTP 499=31
[diagnostic] reads: start_us=1794956828946005 status=499 duration_us=240339
[diagnostic] reads: start_us=1794956828933236 status=499 duration_us=256628
[diagnostic] reads: start_us=1794956828913220 status=499 duration_us=288198
[diagnostic] reads: start_us=1794956828998009 status=499 duration_us=203487
[diagnostic] reads: start_us=1794956828752522 status=499 duration_us=452675
[diagnostic] writes: HTTP 204=11876

─── httparena-tako-gateway-production-stack-edge-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-edge-1 — last 120 log lines ───
  | {"level":"info","ts":1794956805.706405,"msg":"maxprocs: Leaving GOMAXPROCS=30: CPU quota undefined"}
  | {"level":"info","ts":1794956805.7064176,"msg":"GOMEMLIMIT is updated","GOMEMLIMIT":243039533875,"previous":9223372036854775807}
  | {"level":"info","ts":1794956805.70642,"msg":"using config from file","file":"/etc/caddy/Caddyfile"}
  | {"level":"info","ts":1794956805.7064219,"msg":"adapted config to JSON","adapter":"caddyfile"}
  | {"level":"warn","ts":1794956805.7064242,"logger":"admin","msg":"admin endpoint disabled"}
  | {"level":"info","ts":1794956805.7066183,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0x1130cfb3d200"}
  | {"level":"info","ts":1794956805.7067747,"logger":"http.auto_https","msg":"automatic HTTPS is completely disabled for server","server_name":"srv0"}
  | {"level":"info","ts":1794956805.7074828,"logger":"http.log","msg":"server running","name":"srv0","protocols":["h1","h2"]}
  | {"level":"info","ts":1794956805.7170897,"msg":"autosaved config (load with --resume flag)","file":"/config/caddy/autosave.json"}
  | {"level":"info","ts":1794956805.717095,"msg":"serving initial configuration"}
  | {"level":"info","ts":1794956805.7198782,"logger":"tls","msg":"cleaning storage unit","storage":"FileStorage:/data/caddy"}
  | {"level":"info","ts":1794956805.722965,"logger":"tls","msg":"finished cleaning storage units"}
  | {"level":"error","ts":1794956812.9652622,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: http: putIdleConn: too many idle connections for host","request":{"remote_ip":"127.0.0.1","remote_port":"10414","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"localhost:8443","uri":"/api/items/99","headers":{"User-Agent":["h2load nghttp2/1.69.0-DEV"],"Accept-Encoding":["br;q=1, gzip;q=0.8"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost","ech":false}},"duration":0.321880164,"status":502,"err_id":"zwq5y53mm","err_trace":"reverseproxy.statusError (reverseproxy.go:1776)"}
  | {"level":"error","ts":1794956812.968693,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: http: putIdleConn: too many idle connections for host","request":{"remote_ip":"127.0.0.1","remote_port":"11632","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"localhost:8443","uri":"/api/items/98","headers":{"User-Agent":["h2load nghttp2/1.69.0-DEV"],"Accept-Encoding":["br;q=1, gzip;q=0.8"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost","ech":false}},"duration":0.239717359,"status":502,"err_id":"7yhc6cmnw","err_trace":"reverseproxy.statusError (reverseproxy.go:1776)"}
  | {"level":"error","ts":1794956829.2462523,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: http: putIdleConn: too many idle connections for host","request":{"remote_ip":"127.0.0.1","remote_port":"46670","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"localhost:8443","uri":"/api/items/101","headers":{"User-Agent":["h2load nghttp2/1.69.0-DEV"],"Accept-Encoding":["br;q=1, gzip;q=0.8"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost","ech":false}},"duration":0.315783619,"status":502,"err_id":"jqfvf9vtx","err_trace":"reverseproxy.statusError (reverseproxy.go:1776)"}
  | {"level":"error","ts":1794956829.2474177,"logger":"http.log.error","msg":"net/http: HTTP/1.x transport connection broken: http: putIdleConn: too many idle connections for host","request":{"remote_ip":"127.0.0.1","remote_port":"51780","client_ip":"127.0.0.1","proto":"HTTP/2.0","method":"GET","host":"localhost:8443","uri":"/api/items/114","headers":{"User-Agent":["h2load nghttp2/1.69.0-DEV"],"Accept-Encoding":["br;q=1, gzip;q=0.8"],"Authorization":["REDACTED"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"localhost","ech":false}},"duration":0.196367304,"status":502,"err_id":"j3yn5mrmt","err_trace":"reverseproxy.statusError (reverseproxy.go:1776)"}
─── httparena-tako-gateway-production-stack-edge-1 — end of logs ───

─── httparena-tako-gateway-production-stack-server-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-server-1 — the container produced no output at all

─── httparena-tako-gateway-production-stack-cache-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-cache-1 — last 120 log lines ───
  | 7:C 17 Nov 2026 23:06:43.439 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
  | 7:C 17 Nov 2026 23:06:43.439 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
  | 7:C 17 Nov 2026 23:06:43.439 * Redis version=7.4.8, bits=64, commit=00000000, modified=0, pid=7, just started
  | 7:C 17 Nov 2026 23:06:43.439 * Configuration loaded
  | 7:M 17 Nov 2026 23:06:43.439 * monotonic clock: POSIX clock_gettime
  | 7:M 17 Nov 2026 23:06:43.440 * Running mode=standalone, port=6379.
  | 7:M 17 Nov 2026 23:06:43.440 * Server initialized
  | 7:M 17 Nov 2026 23:06:43.441 * Ready to accept connections tcp
  | [redis] seeded from /seed.txt
─── httparena-tako-gateway-production-stack-cache-1 — end of logs ───

─── httparena-tako-gateway-production-stack-authsvc-1 — status=running exit=0 oom=false error=
─── httparena-tako-gateway-production-stack-authsvc-1 — last 120 log lines ───
  | authsvc listening on 0.0.0.0:9090 (JWT HMAC-SHA256)
─── httparena-tako-gateway-production-stack-authsvc-1 — end of logs ───
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[warn] 2 profile/connection measurements failed
httparena-postgres
[info] restoring loopback MTU to 65536

@dancixx

dancixx commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

/benchmark -f tako-gateway -t production-stack --save

Merged current main and pushed it directly. The previous save failed because the bot lacked workflows permission to push the merged workflow files; maintainer edits were already enabled.

Added an unscored fixed-request control after failed production attempts: 100000 reads and 10000 writes, with in-flight requests allowed to finish. The original failure stays failed and the control never publishes a score.

Locally, Caddy 2.11.9 / Go 1.27.3 with Go-only backends reproduced HTTP 499 on requests whose own contexts were still live when other requests were canceled. This matches golang/go#82547. The no-cancellation control and the real h2load fixed-request control completed without errors. This rerun compares timed shutdown with request draining on the benchmark host.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants