[jmandel]

Register row K14.

Reworked after review. The MAX_INCLUDE_CODE_LOOKUPS workaround is gone. The page now lists 500 codes per include by default, with a link to list them all, and the codes it lists are looked up in one call per include instead of one per code. Three commits:

1. resolveCode builds one code system provider per system and version per request, and awaits locate(). This is unchanged from the first version.
2. Provider.resolveCodes(opContext, system, version, codes) looks up many codes of one code system together.
3. The ValueSet page lists the first 500 codes of each include or exclude, then a line like "… and 32,841 more codes. Show all".

What was wrong

The HTML page of VSAC's Medication Brand Name (2.16.840.1.113883.3.88.12.80.16, 33,341 RxNorm codes, all in one include) opened one RxNorm SQLite connection for each code. The connections stay open until the response is finished, at about 50 KB each. On main the server went from 3.6 GB to 5.1 GB and was killed by a 6 GB limit in 3 of 3 runs.

curl -s -o /dev/null 'http://localhost:19844/r4/ValueSet/vsac-2.16.840.1.113883.3.88.12.80.16-20240608?_format=html'

resolveCode also called csp.locate(code) without await. A Promise is always truthy, so every code counted as found, and the page didn't wait for the queries, which kept running after it was sent.

What changed

Truncation (renderer.js, tx-html.js). Renderer.renderInclude lists the first MAX_RENDERED_INCLUDE_CODES (500) codes, then … and N more codes. with a "Show all" link. That link is the same URL plus _showAll=true, which lists every code. I chose the underscore because the other parameters that control how a result is shown (_format, _summary, _elements) use one. It also keeps the parameter apart from the $expand parameters, since the page passes its query string to the expansion it shows. A value set shown in a bundle links to its own page. A page from a POST gives the count with no link. Every code is still checked for validity, including codes not shown. The limit is a constant, because the renderer isn't given any configuration.

Batched look-ups (provider.js). resolveCodes finds the code system once and takes its provider from the per-request cache from commit 1. It locates the codes eight at a time and returns them in order. resolveCode now calls it with a single code. No CodeSystemProvider in main has a lookup for many codes at once, so the lookups run concurrently up to a limit. A prefetchCodes step (#334) could be added here later. The renderer makes one resolveCodes call per include, for the codes it shows.

It also skips code systems that can't link a code. A code from a factory gets a link only if the factory is iteratable() (US states) or overrides codeLink(), and no other factory does. For RxNorm, LOINC, SNOMED CT and the rest, whatever locate() found was thrown away, so these factories are no longer built or asked. The results are the same. This matters once locate() is awaited. RxNorm looks for a code it doesn't have in RXNATOMARCHIVE, which has no index on RXCUI, and each search takes 36 ms. 4,529 of Medication Brand Name's codes aren't in the current RxNorm. Without the skip, show-all took 241–246 s, and the default page took 11.5 s.

Why not an internal expand. An expansion gives displays, but not the links that make up most of what a look-up adds. Those come from codeLink(), the x- page for an iteratable code system, or a CodeSystem resource's anchors, so the locates would still be needed. An expansion also applies its own rules (inactive codes, excludes, size limits). This page's own expansion of Medication Brand Name is refused as "too many codes to produce (>1000)".

Elsewhere. An expansion's contains rows do no lookup per code: their $lookup links are built as strings, and the expansion is already bounded by the expand limits. I didn't change the CodeSystem page.

Before and after

The setup was a local server with RxNorm 03/2026 and a VSAC snapshot (a private hard-linked copy, refreshIntervalHours 500), under a 6 GB memory limit, three runs each.

GET		Time	Page	Resident memory during the request	RxNorm connections	Server afterwards
Medication Brand Name, ?_format=html	main	9.0–9.3 s	2.48 MB, 33,341 rows	3.6 → 4.9–5.1 GB	30,949–33,341	killed, 3 of 3
	this PR	7.8–8.0 s	54 KB, 500 rows + "… and 32,841 more codes. Show all"	+13–14 MB	2	up
Medication Brand Name, &_showAll=true	this PR	7.7–8.0 s	2.48 MB, byte-identical to main's page	+9 MB	2	up
Administrative Gender (3 codes), HTML	both	under 0.1 s	10,768 bytes, identical	no change	0	up
Acute Cluster Headache Medication (small RxNorm set), HTML	both	under 0.1 s	19,876 bytes, identical	no change	0	up

The two remaining RxNorm connections and nearly all of the 8 s come from the expansion that the page also runs, which is refused above 1,000 codes. This PR doesn't change that. Administrative Gender still links its codes to THO, for example <a href="/r4/CodeSystem/tho-v3-AdministrativeGender#F">F</a>: Female.

Tests

• tests/tx/renderer-include-codes.test.js (9 tests):
  • 33,341 codes list 500 rows and … and 32,841 more codes. with the link, and the link is left out when the page has none to give;
  • with no limit, all 1,200 codes are listed and looked up;
  • at exactly 500 codes, nothing is cut;
  • an invalid code past the limit still fails;
  • 2,000 codes make one resolveCodes call, for the 500 codes shown;
  • a resolver without resolveCodes is asked one code at a time;
  • a small include renders exactly as on main.
• tests/tx/html-valueset-show-all.test.js (5 tests):
  • the default page keeps the other query parameters in the link;
  • _showAll=true lists all 33,341 codes;
  • in a bundle, the link goes to the value set's own page;
  • a POST gets no link;
  • a small set is unchanged.
• tests/tx/provider-resolve-code.test.js (12 tests, 5 new):
  • one build for 100 codes, with locates overlapping but never more than 8 at once, and results in order;
  • one build per system and version across overlapping calls;
  • unknown codes come back as null in their place;
  • no codes means no build;
  • a factory that can't link a code is never built, and a CodeSystem resource still links its codes.

22 of the 26 tests fail on main. The touched and nearby suites (tests/tx/renderer*, provider*, html-*, read*, cs-provider-list), run under a 6 GB limit: 19 suites, 201 tests, all pass.

Not changed here: renderCoding (renderer.js, around line 376) reads desc and url from what resolveCode returns, but it returns description and link, so a coding there is never linked. This is the same on main. It's a one-line fix if you'd like it.

🤖 Generated with Claude Code

[grahamegrieve]

MAX_INCLUDE_CODE_LOOKUPS seems like workaround here. We control the renderer, so wouldn't it be better for us to do a deeper fix and batch up fetching the displays? We could run an expand internally?

But there's a wider question here: if the value set is that big, do we really want to render it fully? I think the answer is no, we cut it off at X codes, and only render it fully if the user asks. Which is also a much deeper fix

[jmandel]

Agreed on both, and reworked:

• The page lists the first 500 codes of each include. Then it says "… and 32,841 more codes. Show all", which links to the same URL with _showAll=true. MAX_INCLUDE_CODE_LOOKUPS is gone.
• The listed codes are looked up together. The new Provider.resolveCodes uses one provider per code system per request and runs 8 lookups at a time.
• I didn't use an internal expand. It gives displays but not the links, and this set's own expansion is refused above 1,000 codes anyway.
• Code systems that can't produce a link are no longer looked up. That's everything but US states, since no factory overrides codeLink, so their results were always thrown away. It matters: for each code RxNorm doesn't have, it scans its unindexed archive table (36 ms), which made show-all take 4 minutes. The archive index is in Small fixes: 422s instead of 500s, a request that never got an answer, LOINC data, and two robustness fixes #337 too.

Medication Brand Name (33k codes):

• main: 9 s, 33k open connections, and the server killed at 6 GB in 3 of 3 runs.
• Now, default page: 8 s and 54 KB.
• Now, show-all: 8 s and 2.48 MB, byte-identical to main's page.
• Both use +9–14 MB and 2 connections. Small value sets give byte-identical pages.

[grahamegrieve]

ok thanks