[jmandel]

SNOMED CT: optional memory-mapped cache files, chosen per edition in config.json: run the full tx.fhir.org source list in 16 GB, or less

With #329 merged, tx.fhir.org's full source list needs about 9 GB of process memory, so it already fits a 16 GB machine. This PR adds an opt-in setting that memory-maps the SNOMED CT cache files instead of reading them into the heap. That brings the process to about 3 GB plus reclaimable page cache, which leaves room for more editions and VSAC refreshes on 16 GB, and makes 8 GB workable. The default behaviour is unchanged.

Decisions for you

#	What we chose	Alternative	Why
1	Opt-in: with no snomed block, every edition loads as today (read)	Map by default	Mapping makes the first pass after a restart 28–60% slower, so it should be a choice
2	The load mode is set per edition in config.json	An attribute on each source in the library yml	The yml says which content to serve; config.json says how this machine loads it
3	One new optional dependency, koffi, to call mmap()	A small native addon compiled at install	Prebuilt for every platform, including the Alpine image, with nothing compiled. If it's missing, FHIRsmith reads the files as today
4	A prewarm mode alongside read and mmap	Only read and mmap	The default edition gets most unversioned traffic; prewarming removes its cold-read penalty
5	When any edition is mapped (on Linux), the memory guard leaves file-backed pages out of its measure	Count them as now	Mapped pages are reclaimable page cache; counting them refuses requests on a healthy server

What it does

• A snomed block in the tx module's config.json says how each edition's cache file is loaded: read (today's behaviour, and the default when the block is absent), mmap, or prewarm (mapped, then read through once in the background so its pages are already in the shared page cache). Details below. The library yml still says only which content to load.
• mmap: SnomedFileReader maps each .cache file read-only and wraps the mapping in a Buffer. The structures already read by offset, so nothing else changes. Only the pages that requests touch become resident, as ordinary page cache the kernel can drop and re-read, and shared by every process that maps the same file.
• Memory guard: once any edition is mapped, the 503/422 thresholds use RSS minus file-backed pages (RssFile in /proc/self/status). Otherwise mapped pages that have been read would count as used memory and trip the guard on a machine with plenty left.

12 files, +509 / −14 (not counting package-lock.json). One new optional dependency, koffi (MIT, prebuilt binaries for Linux glibc and musl including the Alpine image, macOS, Windows and the BSDs; nothing is compiled). If it fails to install, FHIRsmith is unaffected.

Platforms. Maps on Linux (x64/arm64, glibc or musl) and macOS. On Windows, 32-bit platforms, or if koffi is missing or mmap() fails, it logs one warning and reads the file exactly as today.

Measured

Full tx.fhir.org source list (tx/tx.fhir.org.yml, VSAC pre-loaded), replaying the IG Publisher's real terminology traffic (the calls recorded in build.fhir.org qa-tx.html transcripts), us-central1, pd-balanced disk at production's limits. "99% within" = 99 of 100 calls finished within this time. Cold = first pass after a restart; warm = second pass.

Same 4 vCPUs, half the memory: e2-standard-4 (16 GB), about $103/month vs $137 for e2-highmem-4 (list on-demand prices, us-central1, Cloud Billing catalog, 1 Nov 2026, with a 50 GB pd-balanced disk; no discounts; single runs)

	process memory	99% within, cold / warm	calls/s, cold / warm	startup
main before #329	14.2 GB	refuses every request (memory guard)	—	—
main (with #329)	9.0 GB	613 / 590 ms	77 / 86	92 s
main + all editions mmap	3.0 GB	630 / 571 ms	71 / 91	55 s

On today's machine (e2-highmem-4, 32 GB), mmap against in-memory (3 repeats)

	process memory	99% within, cold / warm	calls/s, cold / warm	startup
main (before #329)	16.3 GB	772 / 784 ms	78 / 82	98 s
main + mmap	4.5 GB	796 / 769 ms	74 / 85	75 s

The cost is cold reads. On the production disk, SNOMED CT calls took 28–60% longer in total on the first pass after a restart (14,666 calls, 3 repeats; the typical call 6–7 ms → 8–12 ms), while pages were read from disk. Once those pages are resident, mmap is the same speed or slightly faster. About 1.4 GB of SNOMED CT stayed resident under this traffic.

Smaller still. With mmap the same source list runs on an e2-standard-2 (8 GB, about $54/month): 99% within 682 / 614 ms, but about 23% fewer calls per second on 2 vCPUs. Main is OOM-killed during startup there, with or without #329. The floor is about 4 GB; below that V8's heap limit and the memory guard take over.

Per-edition load mode, set in config.json

How SNOMED CT is loaded is a deployment decision, so it is set in config.json, in the tx
module's snomed block. The library yml still says only what content to load.

"tx": {
  "snomed": {
    "load": "mmap",
    "prewarm": ["sct_intl_20250201.cache", "sct_us_20250901b.cache"],
    "read": []
  }
}

• load sets the mode for every edition: "read" (the default, today's behaviour) or
  "mmap".
• prewarm lists editions to map and also read into the page cache once at startup. The
  read runs in the background through a 4 MB buffer, so it costs no process memory. Those
  pages are shared by every process that maps the file, so the first requests don't wait for
  the disk. madvise(MADV_WILLNEED) was tried first, but it read almost none of the file.
• read lists editions to read into the process even when load is "mmap".
• An entry can be:
  • a cache file name,
  • an edition version URI (http://snomed.info/sct/731000124108/version/20250901), or
  • an edition URI, which covers every loaded version of that edition.
• read wins over prewarm.
• These are warned about at startup: entries that match no loaded edition, an entry in both
  lists, and bad values.
• With no block at all, every edition is read, as today.
• The memory guard leaves out file-backed pages once any edition is actually mapped.

Which editions are busy

The use counts on https://tx.fhir.org/r4/ (2 Nov 2026, after 2 days 2 hours of uptime)
show that two editions take almost all SNOMED CT traffic:

edition	use count
US 20250901	804,897
International 20250201	661,818
UK 20230412	31,981
US 20230301	6,555
US 20240301	325
NL 20240930	23
Test set 20250909	14
International 20240201	9
IPS 20241216	0

The top two are 97.4% of the total. In the IG Publisher corpus, the calls that pin a version
point at the same two editions (Intl 20250201: 9,497 calls; US 20250901: 2,563). Unversioned
calls go to the yml's default edition, which is International 20260701 in
tx/tx.fhir.org.yml. The production server's page doesn't list that edition yet.

Measured: read, mmap, mixed and prewarmed, with 1, 2 and 4 server processes

Setup.

• Machine: e2-highmem-4 (4 vCPU, 32 GB), pd-balanced 50 GB, matching production.
• Source list: the full tx/tx.fhir.org.yml, VSAC included.
• "All read" is main with SNOMED CT: hold each loaded edition once, not twice (use cache sections in place) #329 (sections in place).
• Workload: the 14,666 SNOMED CT calls of the IG Publisher corpus, with no version
  repinning.
  • The calls were split evenly between the processes.
  • 4 concurrent clients in total.
  • The cold pass ran right after a start with the page cache dropped; the warm pass ran
    second.
• "Hot" editions: Intl 20250201 and US 20250901.
• Memory:
  • "Private" is the processes' anonymous memory, added up.
  • "Shared page cache" is the SNOMED CT cache file pages resident in the page cache, counted
    once.
  • "Whole cgroup" also counts every other file page the processes touched, including pages
    left in the page cache by reading files.
• Repeats: two runs per row, shown as a range.

processes	configuration	startup s	private memory GB	shared page cache GB	whole cgroup after cold pass GB	cold: total s / 99% ms	warm: total s / 99% ms
1	all read	88–90	8.2–8.3	5.9	15.9–16.5	517–683 / 507–719	427–448 / 338–353
1	hot editions read, rest mmap	56–79	3.8	1.5	8.5–8.7	586–625 / 628–655	450–569 / 377–565
1	all mmap	52–55	2.4–2.5	0.1	6.2–6.7	626–650 / 656–756	481–602 / 387–508
1	all mmap, hot editions prewarmed	58–64	2.1–2.4	1.5	7.0–7.1	600–608 / 590–639	450–610 / 368–480
2	all read	106–146	15.8–16.2	5.9	23.9–24.2	356–515 / 357–498	268–363 / 238–386
2	hot editions read, rest mmap	58–74	7.6–7.7	1.5	12.3–12.4	385–392 / 374–452	308–320 / 331–339
2	all mmap	50–59	4.4–4.5	0.1	8.7	410 / 454–469	277–460 / 249–468
2	all mmap, hot editions prewarmed	57	4.8–5.0	1.5	9.7–9.9	362–369 / 386–416	289–297 / 281–299
4	all read	out of memory at a 30 GB cap during startup
4	hot editions read, rest mmap	90–95	14.8–15.1	1.5	19.5–19.8	329–333 / 346–354	282–310 / 280–358
4	all mmap	89–91	8.5–9.0	0.1	12.0–12.8	343–377 / 386–477	252–289 / 256–262
4	all mmap, hot editions prewarmed	88–89	9.0–9.5	1.5	13.7–14.2	320–337 / 357–386	263–291 / 265–276

What this shows.

• Memory grows with the process count only for what each process reads itself.

  • Each process holds about 2.1–2.5 GB of its own: VSAC, LOINC and the other sources,
    which are loaded into every process.
  • On top of that, each process holds a private copy of every SNOMED CT edition it reads:
    6 GB for all nine, 1.4 GB for the two hot ones.
  • Mapped editions are held once, in the shared page cache, whatever the number of
    processes.

• Four processes on this 32 GB machine:

  configuration	private memory	shared page cache	outcome
  all read	—	—	does not start
  Grahame's mix (hot editions read)	15 GB	1.5 GB	runs
  all mmap, hot editions prewarmed	9.0–9.5 GB	the same 1.5 GB	runs

• Prewarming the hot editions brings first-pass latency back to the level of the mix.

  • With two or four processes, the cold pass took 320–369 s in total with prewarm, against
    329–392 s for the mix and 343–410 s for all-mmap.
  • With one process the four configurations overlap. Run-to-run variation here is
    15–30%, as the two "all read" runs at N=1 show (517 and 683 s).

• Faults on the cold pass: prewarm cut major page faults from about 17,700 (all mmap) to
  about 11,300, the same as the mix.

  • The rest come from the editions that are still mapped cold.
  • Most of them are the yml's default edition, Intl 20260701, which the unversioned calls
    use.
  • Prewarming that edition too cut the faults to about 7,500.
  • It did not measurably change latency: 366–441 s total with two processes, 387–398 s
    with four.
  • The default edition is worth prewarming wherever unversioned traffic is large.

• Warm, every configuration performs the same, within the run-to-run variation.

• Startup:

  • Mapped editions load in milliseconds.
  • Reading all nine editions takes about 50 s of the 88–146 s startup on this disk.

Single-process runs with 3 repeats, against all-read on the same box, give the same picture
for Grahame's mix. These runs used an earlier build that set the mode per line in the yml,
with the same load behaviour.

• Cold pass, total time for the 14,666 calls:
  • all read: 521–539 s
  • hot editions read: 587–629 s
  • Intl 20260701 and US read: 579–694 s
  • all mmap: 601–636 s
• Calls more than 2× and more than 50 ms slower than all-read:
  • hot editions read: 226–279
  • all mmap: 309–357
  • noise floor, all-read against all-read: 122–186
• Warm pass: the same in every configuration.

pin (mlock). This was not implemented.

• The memlock limit is 8 MB by default, and Ubuntu's systemd units have the same default,
  so locking a 0.6–0.9 GB edition would need LimitMEMLOCK= raised for the service.
• Locked pages can't be reclaimed under memory pressure, which gives up what mmap is for.
• Prewarmed pages that stay in use stay resident anyway. The kernel only evicts them when
  memory runs short, and then evicting them is the right thing to do.

Correctness

• HL7 tx-ecosystem IG tests (all modes, including snomed, sct-ecl and tx.fhir.org): 4,565 / 4,565 with no snomed block, with all editions mmap, and with load: mmap plus prewarm.
• SNOMED CT jest suites: identical results in every mode. tests/tx/snomed-load-config.test.js covers the config parsing and matching; tests/library/memory-guard.test.js covers the guard change.
• Replaying 20,000 IG Publisher calls, every call got the same HTTP status with and without mmap.
• Nothing writes to a loaded structure (the setters are used only by the importer), and a loaded edition is never unloaded, so mappings are never unmapped. A stray write would fault, not corrupt.

The full write-up, with per-query footprints and the cost table, is available on request.

🤖 Generated with Claude Code

[grahamegrieve]

Thanks - this looks good to merge. Two small things:

1. The tests don't show that anything was actually mapped. The Library test in
   snomed-load-config.test.js loads snomed-testing.cache with load: mmap, but it only checks
   factory.loadMode and that there are concepts. If mapping isn't available, mapFile falls back
   to reading the file and the test still passes. Could you add a check that the buffer really is
   a mapping (e.g. a mapped flag on the reader/factory, or RssFile going up on Linux), and that
   a mapped load gives the same data as a read load? Related: the fallback warnings in mmap.js go
   to console.warn rather than the server log, so a server that meant to map but couldn't
   wouldn't show it in the log. It would be good to route those through the library's log.

2. The prewarm wording in tx/README.md ("mapped, with the kernel asked to read the whole file
   into the page cache at startup") and in the SnomedFileReader constructor comment ("have the
   kernel read it into the page cache straight away") still describe the madvise approach. Maybe:
   "prewarmed: mapped, then read through once in the background at startup, so its pages are
   already in the page cache when requests arrive. Startup doesn't wait for it."

[jmandel]

Thanks, both done (093cfe3, 1c1e11a):

1. Mapping is proven, and logged.

   • The reader and factory now have a mapped flag, true only when the buffer really is a mapping.
   • The startup log says, per edition, whether it was mapped or fell back to a read.
   • The memory guard switches to leaving out file-backed pages only when something actually is mapped. Before, it switched whenever the config asked for mapping.
   • The fallback warnings go through the library's log, with console.warn only where there's no logger.

   New tests check that:

   • load: mmap gives mapped === true, and on Linux the file shows up in /proc/self/maps;
   • a mapped load and a read load are byte-identical in all 10 structure sections, with the same descriptions, relationships and parents for all 2,176 test concepts;
   • a simulated mmap failure leaves mapped false and logs why.

2. Prewarm wording: I used your text in tx/README.md and the reader comment. I also fixed the same leftover description in load-config.js and config-template.json.

[grahamegrieve]

ok thanks