Repository navigation
fix(recv): do not cache Buffer bodies, which replay as JSON - #256
Merged
Merged
Conversation
isCacheable accepted any non-empty, non-stream body, and the store
serialises every entry with JSON.stringify({ data }). A Buffer stringifies
to { type: "Buffer", data: [...] } and nothing turns it back, and the
handler's Content-Type is not stored either. So the first caller got the
CSV or file the handler sent, and every caller after them got a JSON array
of byte values until maxAge.
Refuse a Buffer body in isCacheable, the way a stream already is: the
response goes out untouched and nothing is stored. Storing binary
faithfully would mean a new entry format and storing headers, which is a
much larger change than the bug. Only Buffer is refused because it is the
only body Koa sends as raw bytes; it JSON-encodes any other typed array
itself, and that round-trips through the store unchanged, so what is
cached for every other body stays exactly as it was.
Keeps both category probe routes and strategy entries: buffer (this branch) and throwing-body (#247).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
The failure mode
A handler that returns a
Buffergets corrupted cache hits. Typical examples are a CSV export, a proxied file, or anything that setsctx.typeandctx.body = Buffer.from(...). The first caller gets the right bytes. Every caller after that, untilmaxAge, gets this:It goes wrong in three steps:
isCacheablelets the Buffer through. It accepts any truthy body that is not a stream.cacheStore.setwrites JSON. Every entry is stored asJSON.stringify({ data }), and aBufferstringifies to{ type: "Buffer", data: [...] }.JSON.parsenever turns that back into aBuffer.recvreplays it without a content type. The entry is sent back as a plain object, so Koa serves it as JSON. The handler'sContent-Type(heretext/csv) was never stored, so it is lost too.The fix
isCacheablenow refuses aBufferbody with the reasonbinary (Buffer) body, the same way it already refuses a stream. The response goes to the client untouched and nothing is stored. Withdebugon, the log line names the reason.Storing binary bodies correctly would need a new entry format plus stored response headers. That is a much bigger change than this bug, and it would touch both providers. Refusing to store a Buffer is the minimal correct fix.
The list of uncacheable responses gains a row in
docs/guide/caching/index.mdand a bullet indocs/guide/reference/config.md.Why this is not a breaking change
AGENTS.md says a fix must keep what gets cached the same for existing configurations. The only responses that change are those with a
Bufferbody, and every cache hit for those was already corrupt. Refusing them replaces a broken response with the correct one. It does not change any correct behaviour.The check is
Buffer.isBuffer, not "any typed array", on purpose. Koa 2.16 (the version Strapi uses) sends only aBufferas raw bytes. Any other typed array, such as a plainUint8Array, Koa serialises as JSON itself, and that JSON survives the store unchanged. Those bodies cache faithfully today and keep caching. Keys,Cache-Controland hitpass are untouched.Fixtures
These come from the bug-hunt patch:
GET /categories/probe/buffer→category.buffer, which setsctx.type = "text/csv"andctx.body = Buffer.from("a,b\n1,2\n");/api/categories/probe/bufferunderapi::category.category, deliberately configured for caching so the refusal is actually exercised;bufferadded to the public permissions for category, like the other probes.Before / after
pnpm run test:e2e:memory -- buffer-bodyBefore (main's
isCacheable, same spec and fixtures):After:
The spec asserts the first request is a
MISSwith the CSV bytes and atext/csvtype. It asserts the second request gets the same bytes and the sameContent-Type, and is also aMISSbecause nothing was stored.Full memory suite:
Test Suites: 24 passed, 24 total/Tests: 137 passed, 137 total.Checklist
shared/tests/that fails before the change.pnpm run test:e2e:memorypasses.pnpm run test:lintpasses.admin/? Not touched.pnpm run test:esmpasses.docs/guide/caching/index.mdanddocs/guide/reference/config.mdlist Buffer bodies as never cached.main.Notes for the reviewer
test:e2e:rediswas not run. The fix is inisCacheable, before any provider is reached.Content-Type. That would be a feature with a format migration, not a bug fix.shared/files, so whichever merges second will need a trivial rebase.