[abhi1693]

What changes and why?

Publication reports repeatedly probe review indexes for every article. This change shares a narrow publication cohort and separates totals/median aggregation from autonomous classification so PostgreSQL can use semi/anti joins and first-match lookups.

The report's datetime bounds use SQLAlchemy's typed execution-time literals. This keeps cohort selectivity visible after generic prepared-plan adoption while retaining compilation caching and normal connection/server settings. Without this, warmed connections reproduced almost all of the original repeated work.

Manual precedence, inclusive per-article cutoffs, historical facts, distinct counts, empty periods, and duration semantics are preserved. Existing partial indexes remain in use.

The benchmark is covered by regression tests, confines atomic JSON output to reports/publication, and uses fresh masked credentials for each disposable CI database. Coverage scope and quality thresholds are unchanged.

Related issues

Closes #296.

Validation

Current head cc7d222c merges master f9b9bfe2. The gateway conflict preserves master's response handling, headers-only declared rejection, and actual chunked placement-overflow checks. The publication query is unchanged.

Merge validation passed: 225 focused backend tests, Python lint/format/types/version checks, 92 targeted gateway/body tests, web lint, and a production reader build with two real gateway fixture runs on Node 26.12.1. Hosted CI and native volume benchmarks are queued for this merge and must pass before merge. The hosted totals and measurements below record the previously validated 94676927 source.

• 10,416 hosted unit tests and 2,802 PostgreSQL integration tests passed, with backend unit statement/branch coverage of 98.01%.
• The profiler's 52 regression cases include two real PostgreSQL integration tests. Its combined hosted unit/integration coverage is 98.34%.
• Publication behavior coverage includes actual psycopg generic preparation, timestamp visibility, cache reuse, precedence/boundaries, duplicate facts, editorial changes, and the 60,340-article/603,400-tag-link fixture.
• Python lint, formatting, versions, mypy, workflow actionlint, and Prettier passed.
• All hosted security checks passed, including Python, Actions, and JavaScript/TypeScript CodeQL.
• Gateway overflow probes now verify declared-size rejection before uploading a body, avoiding a Node fetch/EPIPE race while preserving actual chunked overflow tests. Node 26.12.1 production build, two complete gateway fixture runs, 92 targeted tests, and web lint/types passed.
• CI required and the complete hosted workflow passed on 94676927, including all ARM64 image checks. Sonar reports 98.4% new-code coverage, zero open issues/hotspots, and a passing quality gate. GitHub received the clean Sonar export; all four original alerts are fixed, and the SonarCloud check passes.

Native ARM64 PostgreSQL 17.11 benchmarks passed on 94676927 at 100,000 and 300,000 articles, with 1/7/30-day periods, concurrent ingestion, and custom/generic/warmed automatic plans. The larger fixture has 935,808 reviews and 495,000 historical analysis jobs, approximately 3.2 times the issue's estimated production article count. Every comparison preserved results, and all broad-report buffer, correlated-subplan, and temporary-data budgets passed.

30-day steady-report medians and total shared-buffer accesses:

Articles	Before → after	Buffer accesses before → after
100,000	204.3 → 108.7 ms	230,628 → 20,920
300,000	683.1 → 426.0 ms	780,918 → 306,141

At 300,000 articles, serial backend CPU fell from 680 to 430 ms. Concurrent ingestion completed without errors; the concurrent 30-day report improved from 713.4 to 439.9 ms, and report plans generated zero WAL.

Measured tradeoffs: the 300,000-article 30-day report writes approximately 21 MiB of temporary data and reads it back, within its cohort budget. At 300,000 articles, the warmed seven-day automatic plan measured 132.0 ms versus 116.2 ms and uses more buffer accesses (272,153 versus 176,707). The steady seven-day report still improved from 134.5 to 110.1 ms; at the production-like 100,000-article size, warmed seven-day latency improved from 36.3 to 31.0 ms. Smaller historical job blobs and complete-workload limits are explicit in the evidence and reproduction guide, which records the earlier unchanged-query benchmark run.

Rollout notes

Application-only rollout and rollback; no migration, index, trigger, or database-wide planner change. After deployment, compare fresh statement-statistics deltas under matched reporting/ingestion workloads, including CPU, buffer work, temporary I/O, freshness, and ingestion health. Production was not modified or benchmarked by this PR.

Ready for review

• The description explains the final change, results, tradeoffs, and hosted checks.
• Regression tests and reproducible benchmark documentation are updated.

[github-actions]

Backend unit coverage

PASS · minimum combined coverage 98.00%.

Tests: 10635 passed, 0 failed, 0 skipped. Source files: 367.

Workspace	Lines	Branches	Combined
All backend code	98.67%	95.65%	98.00%
apps/admin-api	96.66%	89.65%	95.33%
apps/aggregator	97.99%	94.45%	97.18%
apps/api	99.49%	98.88%	99.38%
apps/article-enrichment-worker	80.00%	50.00%	71.43%
apps/cli	99.43%	90.76%	98.13%
apps/images-worker	80.00%	50.00%	71.43%
apps/mcp	99.86%	98.70%	99.65%
apps/notifications	100.00%	100.00%	100.00%
apps/partner-api	99.47%	95.51%	98.83%
apps/search-indexer	85.48%	85.71%	85.53%
apps/source-discovery-worker	80.00%	50.00%	71.43%
apps/user-api	99.88%	98.96%	99.72%
packages/core	99.09%	96.84%	98.54%
packages/http	99.47%	97.49%	99.03%

Unit coverage includes every backend workspace. PostgreSQL/Redis integration tests run separately.

Commit: cc7d222 · HTML, JSON, XML reports and job logs · attempt 1

[github-actions]

PR performance results

Current PR head: cc7d222c477cfdc3fc499bf78d180f2c468ddf04

Token usage benchmark

Source head: cc7d222c477cfdc3fc499bf78d180f2c468ddf04 · run 40124489803 · attempt 1

PASS — Token usage benchmark: all 12 comparisons, accounting checks and production-volume gates passed.

Compared head: cc7d222c477cfdc3fc499bf78d180f2c468ddf04 · Benchmark run · Architecture: aarch64.

Before = frozen pre-fix SQL; after = application SQL at the compared head. Improvement = (before − after) / before × 100: + means better, − means worse; n/a means missing evidence or a zero baseline. Lower is better for every performance metric.

Query runtime — median milliseconds

Articles	Window	Plan	Before (ms)	After (ms)	Improvement (+/−)
100,000	1d	custom	41.97	34.95	+16.7%
100,000	1d	generic	42.31	34.51	+18.4%
100,000	7d	custom	253.03	207.07	+18.2%
100,000	7d	generic	263.71	219.54	+16.7%
100,000	30d	custom	867.64	652.48	+24.8%
100,000	30d	generic	897.22	691.12	+23.0%
300,000	1d	custom	191.09	165.96	+13.1%
300,000	1d	generic	190.82	168.83	+11.5%
300,000	7d	custom	757.56	614.87	+18.8%
300,000	7d	generic	802.56	654.17	+18.5%
300,000	30d	custom	2,675.05	1,942.37	+27.4%
300,000	30d	generic	2,753.10	2,077.57	+24.5%

Validation

Check	Passed / expected
Runtime comparisons	12 / 12
Identical token accounting	12 / 12
Runtime, plan and spill gates	84 / 84
Production-volume gates	26 / 26
Measurement step	success

Runtime gate allows up to 20% or 20 ms noise; PASS does not mean every case got faster. Writer latency is observational and has no pass/fail gate. Its samples include writer startup and reports may finish before a subsequent write completes.

30-day CPU, temporary writes and hash memory

Articles	Plan	Metric	Before	After	Improvement (+/−)
100,000	custom	DB container CPU (ms)	914.95	690.27	+24.6%
100,000	custom	App CPU (ms)	0.83	0.78	+5.9%
100,000	custom	Temp writes (blocks)	549.00	0.00	+100.0%
100,000	custom	Largest hash aggregate (KiB)	793.00	40.00	+95.0%
100,000	generic	DB container CPU (ms)	941.95	729.13	+22.6%
100,000	generic	App CPU (ms)	0.79	0.72	+8.6%
100,000	generic	Temp writes (blocks)	549.00	0.00	+100.0%
100,000	generic	Largest hash aggregate (KiB)	73.00	40.00	+45.2%
300,000	custom	DB container CPU (ms)	2,856.17	2,067.49	+27.6%
300,000	custom	App CPU (ms)	1.03	0.95	+8.0%
300,000	custom	Temp writes (blocks)	2,868.00	0.00	+100.0%
300,000	custom	Largest hash aggregate (KiB)	793.00	40.00	+95.0%
300,000	generic	DB container CPU (ms)	2,997.13	2,266.89	+24.4%
300,000	generic	App CPU (ms)	1.13	0.98	+13.0%
300,000	generic	Temp writes (blocks)	2,868.00	0.00	+100.0%
300,000	generic	Largest hash aggregate (KiB)	121.00	40.00	+66.9%

DB CPU is the container CPU median, including concurrent ingestion; elapsed time is measured separately. Hash memory is the largest reported aggregate node, not total process memory. Temporary writes are PostgreSQL blocks from EXPLAIN.

Production-volume coverage: retained history and 1/7/30-day counts

Fixture	Production snapshot	100,000 articles	300,000 articles
Articles	92,598	100,000 (PASS)	300,000 (PASS)
article analysis / total	156,021	200,000 (PASS)	600,000 (PASS)
article analysis / one day	3,034	6,901 (PASS)	20,701 (PASS)
article analysis / seven day	16,692	17,695 (PASS)	53,085 (PASS)
article analysis / thirty day	50,402	59,098 (PASS)	177,296 (PASS)
topic analysis / total	159,575	200,000 (PASS)	600,000 (PASS)
topic analysis / one day	2,381	2,666 (PASS)	8,010 (PASS)
topic analysis / seven day	30,257	34,663 (PASS)	104,013 (PASS)
topic analysis / thirty day	128,657	157,331 (PASS)	472,000 (PASS)
research verification / total	67,014	73,000 (PASS)	219,000 (PASS)
research verification / one day	190	974 (PASS)	2,925 (PASS)
research verification / seven day	3,142	12,655 (PASS)	37,968 (PASS)
research verification / thirty day	45,316	57,426 (PASS)	172,277 (PASS)

Production snapshot observed at: 2026-10-24T10:05:40Z.

Production counts are the captured snapshot, not a live production benchmark.

Full JSON artifact and job logs · attempt 1

Reader API performance

Source head: cc7d222c477cfdc3fc499bf78d180f2c468ddf04 · run 40124490284 · attempt 1

PR reader API performance

Result: INCONCLUSIVE

4 inconclusive · 4 within threshold. Zero request failures across all six runs.

Base f9b9bfe2339bb69a82006066b7302b9709398f87 → PR cc7d222c477cfdc3fc499bf78d180f2c468ddf04

Three independent GitHub-hosted ARM64 runners, each measuring base and PR sequentially with alternating order and isolated resources for every sample. 16 users, completed bootstrap and 15 seconds of warmup before each full 60-second measurement, fixed journey seed 294, 1,000 synthetic articles, cache off, five PgBouncer slots.

Latency comparison

All latency values are in milliseconds; lower is better. Values are medians of three runs, with one-millisecond histogram bins.

Route	Base p95	PR p95	Change	Result
All requests	121	112	-9 (-7.4%)	Within threshold
Latest feed	152	139	-13 (-8.6%)	Within threshold
Feed pagination	117	114	-3 (-2.6%)	Inconclusive
Article detail	83	82	-1 (-1.2%)	Inconclusive
Source feed	106	111	+5 (+4.7%)	Inconclusive
Topic feed	114	113	-1 (-0.9%)	Within threshold
Feed filters	99	84	-15 (-15.2%)	Inconclusive
Sources	56	49	-7 (-12.5%)	Within threshold
Topics	62	50	-12 (-19.4%)	Within threshold

Additional latency and throughput metrics

Each cell shows base → PR. Latencies are in milliseconds.

Endpoint	p50	p99	Requests/s
Aggregated	38 → 36	164 → 160	28.53 → 28.65
/v1/feed [latest]	52 → 50	178 → 182	8.54 → 8.61
/v1/feed [page]	36 → 35	152 → 149	4.05 → 4.08
/v1/articles/[id]	24 → 23	143 → 139	5.80 → 5.81
/v1/feed [source]	34 → 35	134 → 155	2.00 → 2.00
/v1/feed [topic]	35 → 35	149 → 198	1.60 → 1.60
/v1/feed/options	25 → 26	136 → 128	2.18 → 2.18
/v1/sources	16 → 16	86 → 99	2.18 → 2.18
/v1/topics	16 → 18	91 → 109	2.18 → 2.18

All paired p95 samples and changes

Each pair shares a validated runner. Cells show base → PR p95 (absolute and percentage change), in milliseconds. Slower/faster counts use the same strict 20% and 20 ms thresholds against that pair's own baseline.

Route	Pair 1	Pair 2	Pair 3	Slower pairs	Faster pairs
All requests	112 → 112 (+0, +0.0%)	121 → 105 (-16, -13.2%)	122 → 116 (-6, -4.9%)	0	0
Latest feed	152 → 139 (-13, -8.6%)	156 → 130 (-26, -16.7%)	147 → 144 (-3, -2.0%)	0	0
Feed pagination	102 → 114 (+12, +11.8%)	117 → 81 (-36, -30.8%)	122 → 124 (+2, +1.6%)	0	1
Article detail	71 → 63 (-8, -11.3%)	83 → 82 (-1, -1.2%)	90 → 93 (+3, +3.3%)	0	0
Source feed	116 → 123 (+7, +6.0%)	77 → 111 (+34, +44.2%)	106 → 105 (-1, -0.9%)	1	0
Topic feed	86 → 121 (+35, +40.7%)	114 → 113 (-1, -0.9%)	119 → 105 (-14, -11.8%)	1	0
Feed filters	110 → 64 (-46, -41.8%)	78 → 96 (+18, +23.1%)	99 → 84 (-15, -15.2%)	0	1
Sources	56 → 49 (-7, -12.5%)	41 → 46 (+5, +12.2%)	58 → 50 (-8, -13.8%)	0	0
Topics	62 → 50 (-12, -19.4%)	54 → 47 (-7, -13.0%)	67 → 57 (-10, -14.9%)	0	0

Individual runs

Run	Requests	Failures	Exit code
Base 1	1713	0	0
Base 2	1719	0	0
Base 3	1707	0	0
PR 1	1720	0	0
PR 2	1723	0	0
PR 3	1711	0	0

How to interpret this report

Regressed / Improved: a p95 regression needs >20% and >20 ms increase in the medians and in at least two same-runner base/PR pairs. Improvements use the inverse threshold. Within threshold means the change did not meet both thresholds.

Inconclusive: a within-revision p95 range >35% of its median and >20 ms is too noisy for a confident latency verdict. A material median change without two confirming pairs is also inconclusive; this does not prove unchanged performance.

Inconclusive latency comparisons are non-blocking: the check passes without claiming an improvement. Confirmed regressions and test/report errors fail the PR gate. Throughput is descriptive: paced users do not measure maximum capacity. This uncached public API test does not establish production health, browser performance or worker capacity. Base and PR share a runner within each pair; three pairs use independent runners. Alternating order reduces host-assignment bias but cannot eliminate changes in host load over time.

Full HTML/CSV reports and logs · attempt 1

[github-actions]

Lighthouse · ⚠️ Targets exceeded

⚠️ 19 target warnings · Bold values need attention.

Page	FCP	LCP	CLS	TBT	JS	Total
Latest (/latest)	1.23 s	4.42 s	0.001	467 ms	361 KiB	663 KiB
Article preview (/articles/…)	1.37 s	5.02 s	0.005	544 ms	364 KiB	685 KiB
Topics (/topics)	1.23 s	4.02 s	0.000	309 ms	319 KiB	471 KiB
Sources (/sources)	1.23 s	4.20 s	0.000	534 ms	332 KiB	488 KiB
Public profile (/users/…)	1.22 s	2.67 s	0.009	125 ms	280 KiB	428 KiB
Leaderboard (/leaderboard)	1.07 s	2.38 s	0.000	201 ms	276 KiB	383 KiB

Mobile reader · Three cold loads per page · Median timings · Largest transfer size.

Needs attention

Page	Finding	Measured	Target / limit
Latest (/latest)	⚠️ Target · LCP	4.42 s	4.00 s
Latest (/latest)	⚠️ Target · TBT	467 ms	400 ms
Latest (/latest)	⚠️ Target · JS transfer	361 KiB	330 KiB
Latest (/latest)	⚠️ Target · LCP	4.42 s	2.50 s
Latest (/latest)	⚠️ Target · TBT	467 ms	200 ms
Article preview (/articles/…)	⚠️ Target · TBT	544 ms	400 ms
Article preview (/articles/…)	⚠️ Target · JS transfer	364 KiB	330 KiB
Article preview (/articles/…)	⚠️ Target · LCP	5.02 s	2.50 s
Article preview (/articles/…)	⚠️ Target · TBT	544 ms	200 ms
Topics (/topics)	⚠️ Target · LCP	4.02 s	4.00 s
Topics (/topics)	⚠️ Target · LCP	4.02 s	2.50 s
Topics (/topics)	⚠️ Target · TBT	309 ms	200 ms
Sources (/sources)	⚠️ Target · LCP	4.20 s	4.00 s
Sources (/sources)	⚠️ Target · TBT	534 ms	400 ms
Sources (/sources)	⚠️ Target · JS transfer	332 KiB	330 KiB
Sources (/sources)	⚠️ Target · LCP	4.20 s	2.50 s
Sources (/sources)	⚠️ Target · TBT	534 ms	200 ms
Public profile (/users/…)	⚠️ Target · LCP	2.67 s	2.50 s
Leaderboard (/leaderboard)	⚠️ Target · TBT	201 ms	200 ms

Full reports and logs · cc7d222 · Attempt 1

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
98.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud