Optimise DoclingTableRefiner: per-page Docling re-run for large PDFs

[smodee]

Background

The initial DoclingTableRefiner (issue #16) runs Docling on the whole PDF whenever the refiner is triggered. This is simple and reliable, but the eval data shows Docling takes ~22 s/page on CPU (no OCR, TableFormer FAST), with full-doc conversions ranging from ~1.5 min (5-page MMWR) to ~9 min (15-page WHO mpox sitrep, 12-page ECDC CDTR).

For longer reports — or for pipelines that touch many PDFs per cron run — this wall-clock may become a problem. Docling supports DocumentConverter.convert(source, page_range=(start, end)) to limit conversion to a page slice. Restricting Docling to just the pages that contain suspect tables would cut typical refinement cost from minutes to ~22 s.

The eval explored this: per-page cost test in scripts/eval_docling_ocr_cost.py confirmed page_range works and produces consistent per-page timings.

Scope

Once the main refiner is in production, gather some data and decide whether to switch.

1. Measure wall-clock distribution of refiner runs over real ingest traffic for ~2 weeks. Capture:
   • Per-doc total refiner time
   • Page count
   • Number of "suspect" table sections detected
   • Their page numbers
2. Decide the threshold: at what doc size does full-doc Docling become unacceptable? (Probably when median refiner time exceeds some user-facing budget — TBD.)
3. If the data justifies it, implement per-page mode:
   • For allowlist-triggered docs: still run full-doc (no in-tree table to identify suspect pages from).
   • For heuristic-triggered docs: collect the set of page numbers from suspect SectionContents, call convert(source, page_range=(min_page, max_page)) or per-page if non-contiguous.
   • Merge the page-scoped Docling result back into ParsedContent using the same page-number matching the main issue introduced.
4. Regression test: ensure per-page mode produces equivalent table content to full-doc mode on the eval PDFs (MMWR, cholera).

Definition of done

• Two weeks of refiner-time telemetry collected after the main issue ships.
• Decision documented in this issue: per-page or stay full-doc?
• If per-page: implemented behind a config flag (docling_per_page_mode: bool), regression-tested against eval PDFs.

Out of scope

• GPU/sidecar Docling host. That's a deployment decision, separate from the page-range optimisation.

References

• Main refiner issue: Hybrid PDF extraction: add DoclingTableRefiner step in ExtractionPipeline #16
• Per-page cost data: data/docling_eval/ocr/per_page_cost.json
• Eval write-up: data/docling_eval/FINDINGS.md

[smodee]

Telemetry hook added (step 1) — decision still gated

Implemented the refiner-time telemetry this issue's definition-of-done depends on, on branch feat/pipeline-tuning (commit 73bef15).

RefinerTelemetry (extraction/docling_refiner.py) — one record per refine() call (i.e. per PDF reaching the refiner), capturing exactly the fields step 1 asks for:

• convert_s — the Docling convert() wall-clock (the page-scaling cost page_range would reduce); total_s adds the cheap merge
• page_count, n_suspect_tables, suspect_pages — sizes the future page_range window
• fired / trigger / status — separates real runs from skipped_no_trigger / skipped_ocr and failure modes (converter_unavailable / conversion_failed / no_document)

Accumulated on the refiner, exposed via ExtractionPipeline.docling_telemetry, persisted to manifest["docling_refiner"], with a per-PDF console line. Measurement-only — refiner behaviour unchanged. Tests: TestRefinerTelemetry (5 cases).

Two blockers to the actual decision remain, so it stays correctly deferred:

1. No refiner traffic to measure yet. Across every saved run: 0 PDF documents, 0 docling chunks — the benchmark corpus is all-HTML, so the refiner has essentially never fired in replay. Collecting the 2 weeks of data needs live mode or a deliberately PDF-heavy eval. The ~18–22 s/page figure is still only from the offline data/docling_eval/ fixtures.
2. per_page_cost.json is not missing — it's gitignored. .gitignore:24 ignores data/docling_eval/* except FINDINGS.md + sources/, so the per-page cost JSON is a deliberately-uncommitted generated artifact (reproduce with scripts/eval_docling_ocr_cost.py). Noting it here since the reference looked broken from a fresh checkout.

Write-up: data/investigations/track-f-docling-perf.md.