Repository navigation
Fix token-usage report aggregation at production scale - #301
Conversation
Backend unit coveragePASS · minimum combined coverage 98.00%. Tests: 10580 passed, 0 failed, 0 skipped. Source files: 367.
Unit coverage includes every backend workspace. PostgreSQL/Redis integration tests run separately. Commit: 1f23bc6 · HTML, JSON, XML reports and job logs · attempt 1 |
PR performance resultsCurrent PR head: Token usage benchmarkSource head: PASS — Token usage benchmark: all 12 comparisons, accounting checks and production-volume gates passed. Compared head: Before = frozen pre-fix SQL; after = application SQL at the compared head. Improvement = Query runtime — median milliseconds
Validation
Runtime gate allows up to 20% or 20 ms noise; PASS does not mean every case got faster. Writer latency is observational and has no pass/fail gate. Its samples include writer startup and reports may finish before a subsequent write completes. Writer latency caveats — p95 regressions over 20%
p95 uses the nearest-rank sample. Small sample counts and startup outliers prevent a sustained-load latency conclusion. 30-day CPU, temporary writes and hash memory
DB CPU is the container CPU median, including concurrent ingestion; elapsed time is measured separately. Hash memory is the largest reported aggregate node, not total process memory. Temporary writes are PostgreSQL blocks from EXPLAIN. Production-volume coverage: retained history and 1/7/30-day counts
Production snapshot observed at: Production counts are the captured snapshot, not a live production benchmark. Full JSON artifact and job logs · attempt 1 Reader API performanceSource head: PR reader API performanceResult: INCONCLUSIVE 5 inconclusive · 3 within threshold. Zero request failures across all six runs. Base Three independent GitHub-hosted ARM64 runners, each measuring base and PR sequentially with alternating order and isolated resources for every sample. 16 users, completed bootstrap and 15 seconds of warmup before each full 60-second measurement, fixed journey seed 294, 1,000 synthetic articles, cache off, five PgBouncer slots. Latency comparisonAll latency values are in milliseconds; lower is better. Values are medians of three runs, with one-millisecond histogram bins.
Additional latency and throughput metricsEach cell shows base → PR. Latencies are in milliseconds.
All paired p95 samples and changesEach pair shares a validated runner. Cells show base → PR p95 (absolute and percentage change), in milliseconds. Slower/faster counts use the same strict 20% and 20 ms thresholds against that pair's own baseline.
Individual runs
How to interpret this reportRegressed / Improved: a p95 regression needs >20% and >20 ms increase in the medians and in at least two same-runner base/PR pairs. Improvements use the inverse threshold. Within threshold means the change did not meet both thresholds. Inconclusive: a within-revision p95 range >35% of its median and >20 ms is too noisy for a confident latency verdict. A material median change without two confirming pairs is also inconclusive; this does not prove unchanged performance. Inconclusive latency comparisons are non-blocking: the check passes without claiming an improvement. Confirmed regressions and test/report errors fail the PR gate. Throughput is descriptive: paced users do not measure maximum capacity. This uncached public API test does not establish production health, browser performance or worker capacity. Base and PR share a runner within each pair; three pairs use independent runners. Alternating order reduces host-assignment bias but cannot eliminate changes in host load over time. Full HTML/CSV reports and logs · attempt 1 |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Lighthouse ·
|
| Page | FCP | LCP | CLS | TBT | JS | Total |
|---|---|---|---|---|---|---|
Latest (/latest) |
1.21 s | 4.22 s | 0.001 | 283 ms | 361 KiB | 663 KiB |
Article preview (/articles/…) |
1.37 s | 3.95 s | 0.005 | 277 ms | 364 KiB | 685 KiB |
Topics (/topics) |
1.22 s | 3.83 s | 0.000 | 158 ms | 319 KiB | 471 KiB |
Sources (/sources) |
1.22 s | 3.97 s | 0.000 | 353 ms | 332 KiB | 488 KiB |
Public profile (/users/…) |
1.21 s | 2.90 s | 0.009 | 95 ms | 280 KiB | 428 KiB |
Leaderboard (/leaderboard) |
1.06 s | 2.15 s | 0.000 | 119 ms | 276 KiB | 383 KiB |
Mobile reader · Three cold loads per page · Median timings · Largest transfer size.
Needs attention
| Page | Finding | Measured | Target / limit |
|---|---|---|---|
Latest (/latest) |
4.22 s | 4.00 s | |
Latest (/latest) |
361 KiB | 330 KiB | |
Latest (/latest) |
4.22 s | 2.50 s | |
Latest (/latest) |
283 ms | 200 ms | |
Article preview (/articles/…) |
364 KiB | 330 KiB | |
Article preview (/articles/…) |
3.95 s | 2.50 s | |
Article preview (/articles/…) |
277 ms | 200 ms | |
Topics (/topics) |
3.83 s | 2.50 s | |
Sources (/sources) |
332 KiB | 330 KiB | |
Sources (/sources) |
3.97 s | 2.50 s | |
Sources (/sources) |
353 ms | 200 ms | |
Public profile (/users/…) |
2.90 s | 2.50 s |
Full reports and logs · 1f23bc6 · Attempt 1
|
What changes and why?
The token-usage report materialized every matching job and grouped a date expression derived from near-unique completion timestamps. PostgreSQL 17 overestimated daily groups and spilled long windows to temporary storage.
Stream two narrow projections into daily aggregation: extract the UTC date and bounded token text once, then validate/cast tokens once. Literal
OFFSET 0fences preserve those projections under custom and generic plans. Counts, durations, UTC boundaries and cumulative token accounting stay exact, with no schema or additional writer maintenance.The token benchmark and reader API comparison now update the oldest existing performance comment. A shared publisher serializes updates, preserves both sections and head/run provenance, and removes duplicate performance comments after saving their results. Token tables show before, after and signed improvement; resource/volume details expand below the validation totals and writer caveats.
Related issues
Closes #295.
Validation
reports/token-usage/; unsafe links/files are rejected. The disposable PostgreSQL port binds only to loopback with no embedded password, and runtime commands use the locked environment without package builds.Exact read-only production counts at 2026-10-24 10:05 UTC, schema 0034, calibrated every fixture population and its qualifying 1/7/30-day windows:
The completed native ARM64 PostgreSQL 17.11 run on
5f9dcbebee28810860c93d5e4068bb291d0e0890passed 12 exact accounting comparisons, 84 performance/plan gates and 26 production-volume checks. It uses 180 days of history, incompressible 4 KiB provider metadata, 4 MiBwork_mem, JIT, parameterized custom/generic plans and concurrent ingestion requested at 20 writes/second. No parallel workers actually launched. Complete JSON evidence.Measured 30-day query medians:
Monthly temporary writes fell 549 → 0 blocks at 100k and 2,868 → 0 at 300k. The largest reported aggregate-node peak fell 793 → 40 KiB; this is not whole-process memory. Database-container CPU includes concurrent writer/background work. Writer latency is observational: short samples include startup effects and do not prove steady-state ingestion latency. The current run exposes one writer p95 regression: 22.54 → 39.58 ms (−75.6%), from only 56/41 samples; it does not establish a sustained-load regression or improvement.
The existing results comment was updated in place by both workflows on the current head; exactly one performance comment remains. The reader API comparison completed all six runs without request failures; latency was inconclusive, with no confirmed regression. The native run and in-place comment update passed on
5f9dcbebee28810860c93d5e4068bb291d0e0890. Full CI also passed on that head in attempt 1: all six reader browser suites, all six database integration shards, all three CodeQL languages, SonarQube, all 12 native ARM64 image builds/scans/smoke tests, and CI required. Sonar reports 99.7% new-code coverage, no open issues and no security hotspots awaiting review. All automated review conversations are resolved.Rollout notes
No schema/API contract change, extra index, trigger or writer maintenance. Rollback is an application rollback. The query still reads matching histories once. Synthetic timings do not establish production latency; after deployment compare fresh statement statistics under matched traffic, report freshness, database CPU and ingestion latency. This PR does not deploy production.
Ready for review