Repository navigation
static, celeris: a Root of '.', './' and '/' serves the tree below it (v1.6.0, Refs #888) - #1152
Merged
Merged
Conversation
…the test that pinned the defect is replaced (v1.6.0, Refs #888) Red on main: TestRootSpellings{ServeTheTree,ConditionalRequests,FollowSymlinksInside, RefuseSymlinksOut,TraversalMatchesTheAbsoluteRoot,BrowseAndSPA,OverTheWire}888, TestRootSlashHasNoOutside888, TestRoot304DotRootDoesNotFollowSymlinkOut888 (flipped: a path below the root now reaches the containment check and is refused with a 400) and the root package's TestFileFromDirDotAndSlashBases888. TestRootDotAndSlashServeOnlyTheRootIndex888, which pinned the defect, is removed.
…ow it, as Context.FileFromDir does for those bases (v1.6.0, Refs #888) The containment check compared the cleaned path with root + separator: no path below '.' starts with './' (filepath.Join drops it) and none below '/' starts with '//'. So static passed every request below these roots on, and Context.FileFromDir refused every file below them with a 400. One comparison now (ctxkit.PathWithin) serves the five sites: static's lexical guard, its browse and 304 containment checks, and FileFromDir's lexical and symlink checks. A symlink with an absolute target that is resolved under a relative base is compared with the base made absolute (ctxkit.ResolvedWithin). Release note: static Root '.', './' and '/' now serve the tree; before they served nothing below the root. Context.FileFromDir (and so Server.Static and Group.Static) takes the same bases.
…' root is refused (kills the recheck mutant)
…e baseDir takes an in-tree absolute symlink (Refs #888)
… it as '.' (Refs #888) filepath.Clean("") is ".", which this PR makes servable: Server.Static and Group.Static pass their root on unchecked, so an unset root would have served the working directory where it was a 400 for every file before.
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
… names: matrix over root spellings, requests and entry points (Refs #888) Red on the PR head: TestDotDotRootsMiddleware888, TestDotDotRootsFileFromDir888 and TestDotDotRootsOverTheWire888 (std and epoll; middleware, Server.Static, Group.Static) for the roots '..', '../', './..', 'x/../..', 'a/../..', '../..', '../../', '../x/..' (GET /../TOPSECRET.txt answers 200 with a file outside the directory the root resolves to).
… above the tree, '../..' rows, HEAD and Range rows (Refs #888)
…s is the directory it resolves to, and nothing outside it is served; docs for the upgrade hazard and Static (Refs #888) PathWithin special-cased '.' and not '..': for a base of '..', '../../etc/passwd' starts with '..' and a separator, so it was 'within', and Root '..' or '../..' (and FileFromDir, so Server.Static and Group.Static) served any file the process could read to a request that climbs. The old prefix test had the same flaw: it is on main since v1.3.3. A relative base that starts with '..' and the path are now made absolute against the working directory with its symlinks resolved, and compared as absolute paths. ResolvedWithin uses the same step when the lexical comparison refuses, which also serves a symlink that climbs through the root's own name (a link '../c/a' under Root '.' in a directory named c). Docs: Config.Root, doc.go, FileFromDir, Server.Static and Group.Static state that '.' serves the working directory with its dotfiles, that '.' is the whole filesystem when the working directory is '/', and that a configuration inert since v1.3.3 starts to serve after the upgrade; the package has no dotfile rule today and a default deny is being decided.
…s, with no behaviour yet (the API and its golden only; Refs #888)
…nown, root names, Index, Browse, SPA, symlinks, FS, HEAD/Range/conditional, the wire (Refs #888) Red on the head before the rule (against the Dotfiles field with no behaviour): TestDotfiles{Policies,HeadRangeConditional,RootNameIsNotHidden,Index,Browse,SPA,Symlinks,FS, OverTheWire}888 fail (142 FAIL lines, 244 PASS: the passes are the visible rows and the Allow rows); TestFileFromDirServesDotfiles888 passes and pins that Context.FileFromDir is unchanged.
… default; Config.Dotfiles (DotfilesIgnore, DotfilesAllow, DotfilesDeny) (v1.6.0, Refs #888) A request whose path below the root has a segment that starts with a dot (.env, .git, .ssh, .htaccess) is answered as if the file did not exist (the next handler; the index in SPA mode), whether or not it exists: DotfilesIgnore, the default. DotfilesDeny answers 403; DotfilesAllow serves them as before. The exact segment .well-known is served (RFC 8615); its children are unless their own names start with a dot. The segments are those of the cleaned request path (a/../.env is .env) with one level of percent-decoding; the name of the root never counts. The rule follows symlinks: a visible name that leads to a hidden file or directory of the tree is refused (also for a 304), and a listing (Browse) leaves hidden entries out. The Index file is the operator's choice and is exempt. It applies to Config.Root, Config.FS, Server.Static and Group.Static; Context.FileFromDir, whose caller chooses the path, is unchanged. Exported API added: DotfilesPolicy, DotfilesIgnore, DotfilesAllow, DotfilesDeny, Config.Dotfiles. Release note: static: files and directories whose name starts with a dot are not served by default (404); set Dotfiles: Allow to restore the old behaviour; .well-known is served by default; this also makes the new Root '.', './' and '/' support safe for an upgrading project.
…e rows run with Dotfiles: Allow and are passed on by default; the matrix tree has no dotfile (Refs #888)
…O, socket, devices, symlinks to them) over Root, '.', '/', os.DirFS and the explicit APIs; the symlink rule under a real os.DirFS; a backslash is not a separator; a hidden .gz link; DotfilesPolicy validation; Server.Static 404 for a missing file; the cost of a '..' root (Refs #888) Red on f7c79a1: TestNonRegularFilesAreNotServed888 (every FIFO/socket cell hangs: no answer in 5 s), TestNonRegularFilesExplicitAPIs888, TestDotfilesDirFS888 (a visible link to .env is served under os.DirFS), TestDotfilesCompressVariantLink888, TestDotfilesPolicyIsValidated888, TestServerStaticMissingIs404_888, TestDotDotRootCostsAboutADotRoot888, and the new rows of TestDotfilesPolicies888 (a literal backslash name). The device cells (TestDeviceFiles*888) run in a memory-capped container, see the next commits.
…dotfile rule follows symlinks under os.DirFS; a backslash is a name character; a hidden .gz link is no variant; Dotfiles is validated; Server.Static answers a missing file with 404; a '..' root is resolved once (Refs #888) Non-regular files (R2-F7): fileFromDir, Context.File, FileFromFS, serveOS and serveFS refuse anything that is neither a regular file nor a directory, before the open (open(2) of a FIFO waits for a writer) and again on the open descriptor, and the buffered reads are bounded by the existing 100 MB cap (io.LimitReader). The middleware answers a missing file; Context.File, FileFromDir and FileFromFS return an HTTPError 404 'not a regular file' (wrapping ctxkit.ErrNotRegular). Symlinks are followed to a regular file only. R2-F1: serveFS resolves symlinks for an FS that implements fs.ReadLinkFS (os.DirFS) and applies the hidden rule to where the path leads. R2-F2: HiddenPath splits on '/' only (and on '\\' on Windows); a segment holding a backslash is hidden if a piece of it starts with a dot. R2-F3: a variant that is a hidden link or not regular is skipped. R2-F6: a root made of '..' segments is resolved once at New and at Server.Static; R2-F5: Server.Static and Group.Static map a missing file to 404.
…the API and its golden; the tests that serve a filesystem root set it) (Refs #888)
…efused at New and at Server.Static / Group.Static unless Config.AllowFilesystemRoot is set (Refs #888) Red against the field with no behaviour (docker --cpus 2 -race): TestFilesystemRootIsRefused888 (12 spellings: '/', '//', '/.', '.' at '/', '..' from /usr, '../../..' up to and beyond '/', symlinks to '/'), TestStaticRoutesRefuseAFilesystemRoot888, ctxkit TestIsRootOf (Windows drive and UNC rows) and TestFilesystemRoot: 16 FAIL lines, 5 PASS (the ordinary roots and the acknowledged cases, which pass before and after).
…fused at New, Server.Static and Group.Static unless Config.AllowFilesystemRoot is set; docs: not a secrets filter, not a sandbox, non-regular files, FS symlinks, the root is resolved at New (Refs #888) Release note: Root resolving to the filesystem root is refused unless Config.AllowFilesystemRoot is set; nothing in any released version served such a root.
… (kills the serveOS regular-file mutant) (Refs #888)
…ave an FS, the walk's bound, a FIFO under fs.Sub, Server.Static's body for a FIFO, a size-0 pseudo-file, the open descriptor's own check (Refs #888) Red on 72615c2 (docker --cpus 2 -race, 1 GiB): TestDotfilesFSLinksLeavingTheFS888 (a hidden file reached through two planted links, or through an absolute link, is served under Ignore and Deny, under os.DirFS and fs.Sub), TestFSLinkWalkIsBounded888, TestFIFOUnderSubFS888 (blocked 5 s), TestServerStaticNotRegularIsNotFound888, TestZeroSizePseudoFileIsServedEmpty888 (the binary was killed: /proc/self/pagemap read to the 100 MB cap). TestServeOpenFileChecksTheOpenDescriptor888 passes and pins the check the mutant run found untested; it uses the openForServe seam.
…gnore and Deny; the walk is bounded; a FIFO under fs.Sub is found; size-0 pseudo-files are served empty; the open is non-blocking; Server.Static answers a FIFO as not found; docs (Refs #888) R3-F1: ctxkit.ResolveFSLinks walks an FS path link by link (at most 8 links and 256 steps) and reports ok=false when it leaves the FS (an absolute target, a target that climbs above the FS root) or cannot finish: the middleware then refuses the path as hidden (404 as missing under Ignore, 403 under Deny); Allow follows as before. R3-F3: an FS that is a ReadLinkFS but no StatFS (fs.Sub) is Lstat-ed at the resolved name before the open, in the middleware and in FileFromFS. R3-F4: the buffered read is bounded by the size the open file reports. R3-F5: the file is opened with O_NONBLOCK where the platform has it. R3-F6: Server.Static maps a non-regular file to the 404 of a missing one. R3-F2 and the cases that reach a filesystem root unprotected are in the godoc.
…sed variant under os.DirFS and fs.Sub, and the link cases around the two that cannot be checked (Refs #888)
…one check: a FIFO as an index file or a pre-compressed variant is not opened; docs for what an FS cannot do (Refs #888) ctxkit.NotRegularFS looks at a name without opening it (fs.Stat for a StatFS; the resolved name's Lstat for an FS that reads links, an fs.Sub); the middleware's main open, its index probe and its variants, and FileFromFS use it. Two cases cannot be checked and are documented: a link that leaves an fs.Sub under DotfilesAllow, and a FIFO swapped in after the check (an FS cannot open non-blocking). The godoc says a path through a link that leaves the FS is refused whatever it leads to, and that /proc/self/environ is served empty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
middleware/staticwithRootset to.,./or/served nothing below the root. This fixes it, in static and inContext.FileFromDir, which static serves through and which had the same flaw.Refs #888 (items 35 and 46; the finding is D2 of the L16 review of #1131).
Release notes
.env,.git/,.ssh/,.htaccess) is answered as if the file did not exist (next handler / router 404; the index in SPA mode), whether or not it exists; directory listings omit such entries; a visible name that is a symlink to a hidden file is refused too (forRoot; forConfig.FSwhen the FS can read links, asos.DirFScan, and only for links that stay inside the FS: links that leave the FS are not followed under Ignore and Deny: a path through one is refused like a hidden one, whatever it leads to, a visible target included, so acurrent -> /srv/releases/42layout under an FS needsDotfilesAllowor aRoot)..well-known(exact name) is served. Applies toConfig.Root,Config.FS,Server.StaticandGroup.Static. New:static.DotfilesPolicywithDotfilesIgnore(default),DotfilesAllow(the old behaviour) andDotfilesDeny(403), set withConfig.Dotfiles; any other value panics atNew.Server.Statichas no opt-out: use the middleware.Context.FileFromDiris unchanged. The rule is not a secrets filter: a file with no dot segment in its path is served.Root/Server.Staticroot "." and "./" now serve the working directory (they served nothing below it since v1.3.3). Upgrade hazard: a configuration that was inert starts serving the working directory (sources, configs, without their hidden files). Name a directory that holds only public files.static.New,Server.StaticandGroup.Staticpanic whenRootresolves to "/" (or a Windows drive / UNC root): "/", "//", "/..", a symlink to "/", "../../.." reaching it, and "." or "./" when the working directory is "/" (systemd services, containers withoutWORKDIR): such an app, which served nothing before, now panics at startup with a message naming the resolved root. To serve a filesystem root on purpose use the middleware with the newConfig.AllowFilesystemRoot: true(it then serves every file the process can read,/etc/passwd,/procand/sysincluded). The check is made once, atNew/ registration (the Root should exist then); a laterchdiror symlink retarget is not covered.Config.FS(e.g.os.DirFS("/")) andContext.FileFromDirare not checked."..","../..","./..","x/../..") no longer serves files outside the directory it names (GET /../secretreturned files above it since v1.3.3, forstatic.Config.Root,Server.Static,Group.StaticandContext.FileFromDir). For the middleware and the static routes such a root is resolved against the working directory once, atNew/ registration; prefer an absolute root.GET /dev/zerounder a root that reached it read without bound and a FIFO hung the handler): the middleware answers them as a missing file;Context.File,FileFromDirandFileFromFSreturn anHTTPError404 "not a regular file". A symlink is followed to a regular file only. A file is read for as many bytes as it reports (a size-0 pseudo-file such as/proc/self/pagemapor/proc/self/environis served empty), and aRootfile is opened without blocking. ForConfig.FSandFileFromFSevery name of the tree is looked at before it is opened (anos.DirFSand anfs.Subof one; an index file and a pre-compressed variant included), but an FS cannot open non-blocking, so a FIFO swapped in after that look can still block one request, and a link that leaves anfs.Sub(followed only underDotfilesAllow, or byFileFromFS) ends where the FS cannot look.Server.Static/Group.Staticanswer a missing file, and a FIFO or socket, with 404 (was 500 for a missing file).Context.FileFromDiraccepts a baseDir of ".", "./" and "/" (was 400; "/" serves every file the process can read to the caller-chosen path, and is not refused); an empty baseDir is still refused. A relative base now serves an in-tree file reached through a symlink with an absolute target (was 400).static.DotfilesPolicy,static.DotfilesIgnore,static.DotfilesAllow,static.DotfilesDeny,static.Config.Dotfiles,static.Config.AllowFilesystemRoot.Dotfiles (the maintainer's decision after the security review)
Exported API added (
api/middleware/static.txt, regenerated withmage api;mage apiCheckmatches):type DotfilesPolicy uint8,const DotfilesIgnore(0, the default),DotfilesAllow,DotfilesDeny, andConfig.Dotfiles DotfilesPolicy(appended last, so the field order of existing fields is unchanged). Naming follows the repo (ClientPolicyDrop,HubPolicy...).Semantics, as Express serve-static's
dotfilesoption except where noted:.well-known, exactly, is not hidden (RFC 8615);.well-known-x,.well-knownx,.Well-Knownare. Its children are served unless a child segment is itself hidden (/.well-known/.secret,/.well-known/.sub/xare hidden).a/../.envis.env(cleaned asfilepath.Joincleans what is opened),./,//and\separators handled, one level of percent-decoding (the engine hands the middleware the raw path, so%2eenvnames a file literally called%2eenv; a segment that decodes to a dot name is hidden anyway, and%252edecodes once only and names nothing). The root's own name never counts: Root.config,.or..inside a.configdirectory, and a Prefix such as/.assetsserve their files.Index: ".index.html"is served for a directory request (and in the SPA fallback); asked for by its path it is hidden.link -> .env,link -> .git) is refused like it, also for a conditional request (no 304 oracle) and for a listing; a hidden name is hidden whatever it links to; a link to.well-knownis served. The reason: the point of the rule is that a dotfile is not served however it is reached. The cost: alatest -> .builds/42layout needsDotfilesAllowor a visible directory name; it is in the godoc. The containment check still governs targets outside the root.Config.FS),.well-knownis listed; a hidden directory is not listed at all. HEAD, Range, If-None-Match, If-Modified-Since, If-Range on a hidden path behave as on the missing path (no 206, no 304, no validators); Deny 403 without validators.Config.Root,Config.FS(the request rule, the listing filter and, for anfs.ReadLinkFSsuch asos.DirFS, the symlink rule: see Round 2),Server.StaticandGroup.Static(directory servers; a hidden path is a 404HTTPError; they have no option, so a project that serves hidden files uses the middleware withAllow). Finding, not changed here:Server.Staticanswers a missing file with 500 (FileFromDirreturns the rawlstaterror), while a hidden path is 404; this does not reveal whether a hidden file exists (a hidden path is 404 whether or not it does).Context.FileFromDiris unchanged (see the analysis in the report): an explicit API whose caller chooses the path. In this repository its only users areServer.Static,Group.Staticand the static middleware, which now pass a hidden-path predicate through an internal parameter; the public method passes none.TestFileFromDirServesDotfiles888pins it.Failing-first: the API-only commit (field and constants, no behaviour) first, then the tests, red against it (docker golang:1.27.3
--cpus 2-race):TestDotfiles{Policies,HeadRangeConditional,RootNameIsNotHidden,Index,Browse,SPA,Symlinks,FS,OverTheWire}888fail, 142 FAIL lines, 244 PASS lines (passes are the visible rows and the Allow rows);TestFileFromDirServesDotfiles888passes (it pins the unchanged method). Green with the rule: 386 PASS lines, 0 FAIL in that selection; at the final head the static selection of all the root tests is 791 PASS lines, 0 FAIL. The earlier root tests were updated where they asserted that a dotfile is served (their dotfile rows run withAllowand are checked to be passed on by default; the..matrix tree has no dotfile). Also:internal/ctxkittables forHiddenSegment,HiddenPath(zero allocations on a plain path) andRelWithin.Mutants, each turned a named test red: rule removed (
TestDotfilesPolicies888, wire, FS...: 5 failing tests), only the last segment checked (7),.well-knownexemption by prefix (5), the exemption removed (7), the root's own name treated as hidden (TestDotfilesRootNameIsNotHidden888), no percent-decoding (5), symlink-target rule off (5,TestDotfilesSymlinks888), listing unfiltered (TestDotfilesBrowse888,TestDotfilesFS888), Index exemption off (TestDotfilesIndex888), the 304 oracle (TestDotfilesHeadRangeConditional888,TestDotfilesSymlinks888), Deny answered as Ignore (5), theServer.Staticrule off (TestDotfilesOverTheWire888), "." segments not dropped (TestDotfilesPolicies888).Runs: the new tests
-race -count=10ok; wholemiddleware/static,internal/ctxkitand the root package under-raceonce ok (the root package 53 s); the native darwin whole static and ctxkit packages ok; golangci-lint v2.14.0 0 issues (linux, darwin);go vetwindows, darwin, linux clean;mage apiCheckok.Round 4 of the security review
ctxkit.NotRegularFS(fsys, name), looks at a name without opening it (fs.Statfor aStatFS; the link walk and anLstatof the resolved name for an FS that reads links but has noStat, anfs.Sub) and everyOpenof a name from the tree is preceded by it: the main open, the directory index probe, the pre-compressed variants andFileFromFS(a grep ofOpen,ReadFileandReadDirinstatic.goandcontext_response.gofinds no other: the sniff re-open of the original file inservePreCompressedFSis of a name the main check already passed, andReadDirreads directories).TestFIFOIndexAndVariantUnderFS888: a FIFO asd/index.html(with and without Browse, SPA) and asx.js.br(Compress,Accept-Encoding: br), underos.DirFSandfs.Sub, Ignore/Deny/Allow: red onc73407din a 512 MiB container (10 FAIL lines: everyfs.Suband index cell "no answer in 5 s"), green now;TestNotRegularFSin ctxkit (StatFS, link-only and open-only FS values). Not checkable, so documented rather than "fixed": (3) a link that leaves anfs.Subfollowed underDotfilesAllow, and (4)FileFromFS(sub, name)of such a name: the FS cannot say what is beyond the link, and refusing it would break theAllowsetups that exist to follow such links. A FIFO behind such a link blocks one request; the godoc says so. (Under Ignore/Deny these paths are refused before anything is opened, and anos.DirFS, aStatFS, follows the link withstat;TestFIFOLeavingLinksAreRefusedUnderIgnore888.)Rootfiles are opened withO_NONBLOCK.Config.FSandFileFromFS; a path through a link that leaves the FS is refused whatever it leads to (an absolute link to a visible file, a re-entry by the FS root's own name, a chain of more than 8 links:current -> /srv/releases/42under an FS needsAllowor aRoot); theAllowFilesystemRootgodoc says/proc/self/environand/proc/self/statusare served empty and that/sysfiles and other pseudo-files that report a size are served for the bytes they hold.FileFromFScheck off (3), the resolved-nameLstatreplaced by the link's own (2:TestDotfilesFSLinksLeavingTheFS888,TestNotRegularFS).-race -count=10ok; the whole static package-vunder-racein a 1 GiB container 1458 PASS lines, 0 FAIL, 0 SKIP; ctxkit ok; the whole root package under-raceok (49 s); golangci-lint v2.14.0 0 issues (linux, darwin);go vetwindows, darwin, linux clean;mage apiCheckok (no new exported identifier).AllowFilesystemRoot, HEAD of a/sysfile reportsContent-Length: 4096while GET sends 6 bytes (the engine's HEAD takes the stat size; pre-existing and opt-in); a chain of 9 to 40 links to a visible file is served under aRootand refused under an FS (fail closed, documented as 8).Round 3 of the security review
os.DirFS/fs.Sub): fixed, failing closed. The FS symlink walk stopped at the first link that left the FS, so a hidden file reached through two planted links (a relative link that climbs out of the FS and back,/out-parent-envserving the.envabove the public directory; an absolute link and a link behind it) was served under Ignore and Deny.ctxkit.ResolveFSLinksnow reports "cannot vouch" for an absolute target, a target that climbs above the FS root, more than 8 links on a path (a loop included) and a walk of more than 256 steps, and the middleware refuses such a path like a hidden one (a missing file under Ignore, 403 under Deny; Allow follows as before). A link whose target stays inside the FS is judged by where it ends. That includes an absolute link to a visible file outside (the FS cannot vouch for it).TestDotfilesFSLinksLeavingTheFS888has the review's six rows plus: out and back, the FS root's parent (up -> ..), chains of 2 and 3 links, loops, an absolute link to a visible file, underos.DirFSandfs.Sub(os.DirFS), Ignore/Deny/Allow, with the served rows (v1 -> v2 -> a.txt,in-visible): red on72615c2(36 FAIL lines across the new round-3 tests), green now. The godoc (Config.Dotfiles,doc.go), not only this body, says that links that leave the FS are not followed.TestFSLinkWalkIsBounded888: under 1500 allocations against 24,574 measured by the review). The no-link path no longer allocates a path per segment (oneLstatof each prefix of the name).BenchmarkDirFSRequest888(docker--cpus 2, os.DirFS, deep path of 6 segments): 42 -> 27 allocations, 7.0-10.6 us -> 6.1 us per request; shallow 15 -> 12 allocations, 3.9 -> 3.6 us, against 9 allocations and 3.2 us with the rule off (Allow). The remaining cost is oneLstatper segment, needed to see a link.fs.Sub(os.DirFS)hangs): fixed. Anfs.Subis anfs.ReadLinkFSbut nofs.StatFS; the pre-open check nowLstats the resolved name for such an FS, in the middleware and inFileFromFS. An FS that is neither is opened as it is, and the godoc says so.TestFIFOUnderSubFS888(FIFO, socket, links to them,FileFromFS): blocked 5 s on72615c2, green now./proc/self/pagemapreads 100 MB per request): fixed. The buffered read (serveOpenFile,FileFromFS) is bounded by the size the open file reports, so a size-0 pseudo-file is served empty, as under epoll's sendfile and under an FS (the cap of 100 MB still bounds the size).TestZeroSizePseudoFileIsServedEmpty888(Linux; with the heap guard, in a container): the test binary was killed on72615c2; green now. TheAllowFilesystemRootgodoc says a filesystem root also serves/procand/sys.New, that the Root should exist then, and that a Root that is or contains a symlink is followed at each request, so a later retarget (acurrentsymlink, or a link to "/" made afterNew) is not covered. A missing Root is not refused (existing behaviour: it serves nothing).os.DirFS("/"),os.DirFS(".")at a working directory of "/",FileFromDir("/"), a root retargeted afterNew, a bind mount of "/".O_NONBLOCKwhere the platform has it (a FIFO swapped in after the stat is caught by the descriptor check instead of blocking inopen(2);TestOpenNonblockDoesNotWaitOnAFIFO888, killed by the mutant that opens blocking); the descriptor check is now tested (TestServeOpenFileChecksTheOpenDescriptor888, through anopenForServeseam, it was the mutant that survived);Server.Staticanswers a FIFO or socket with the body of a missing file (TestServerStaticNotRegularIsNotFound888).Mutants (each turned a named test red): an absolute link followed (2 failing tests), a climb above the FS root ignored (1), the prefix scan skipped (2), the
fs.SubFIFO check off in the middleware (1) and inFileFromFS(1), the read bound back to the cap (1, the container kills the binary),Server.Static's FIFO body (1),O_NONBLOCKoff (1), a walk that cannot vouch accepted (2). Not mutated: the two walk bounds separately (they overlap: a loop is cut by either; removing both hangs the request).Runs: the new tests
-race -count=10ok (one process at a time, 1 GiB container); the wholemiddleware/staticpackage-vunder-racein a 1 GiB container: 1444 PASS lines, 0 FAIL, 0 SKIP (the device and FIFO cells ran only in the container; the Mac runs skip them);internal/ctxkitok; the whole root package under-raceok (55 s, uncapped, as before); golangci-lint v2.14.0 0 issues (linux, darwin);go vetwindows, darwin, linux clean;mage apiCheckok (no new exported identifier this round).Findings, not fixed here (unchanged): the Windows drive-relative roots
C:/C:..and 8.3 short names underos.DirFS(reasoned, not executed); the existence oracles that need a planted symlink; TOCTOU swaps (hard links, a directory component swapped to a symlink) beyond the non-blocking open and the descriptor check; absolute targets judged by their own segments no longer apply (they are refused under an FS); a warning atNewfor/proc,/sys,/etc-like roots was recommended and not done;FileFromFSgodoc does not say it serves hidden names.Round 2 of the security review (and the filesystem-root decision)
Exported API added in total (
api/middleware/static.txt,mage api;mage apiCheckmatches):DotfilesPolicy,DotfilesIgnore,DotfilesAllow,DotfilesDeny,Config.Dotfiles, andConfig.AllowFilesystemRoot bool(zero value = refuse a filesystem root). Both fields are appended last.fileFromDir,Context.File,FileFromFS,serveOSandserveFSrefuse anything that is neither a regular file nor a directory, before the open (open(2) of a FIFO waits for a writer; anfs.StatFSsuch asos.DirFSis stat-ed first) and again on the open descriptor; the size-0 check that let/dev/zerothrough no longer matters, and the buffered reads useio.LimitReaderat the package's 100 MB cap as defence in depth (untested: it needs a file that grows during the read). Failing-first: the three device tests (TestDeviceFilesStatic888,TestDeviceFilesExplicitAPIs888,TestDeviceFilesOverTheWire888; Root "/" and "." at cwd "/",os.DirFS("/"),c.File,FileFromDir("/"),FileFromFS, std and epoll) were run on the headf7c79a1indocker run -m 512m --memory-swap 512m --cpus 2: 3 of 3 killed the test binary (signal: killed) within about a second; with the fix they pass (0.00 s, 0.00 s, 0.11 s). In process there is a 256 MiB heap guard that exits the binary; the cells do not run on macOS (a regression would write 8 GB/s into a developer's machine;L21_DEVICE_CELLS=1forces them), and I never ran them there.TestNonRegularFilesAreNotServed888(FIFO, socket, symlinks to them,/dev/null, under an absolute Root, ".", "/",os.DirFS, SPA, plus a conditional request) andTestNonRegularFilesExplicitAPIs888run everywhere: red onf7c79a1(every FIFO cell: "no answer in 5 s"), green now.Config.FS): fixed for anfs.ReadLinkFS(os.DirFS): the path is walked withLstat/ReadLink(links, chains, relative links through "..", absolute targets judged by their own segments, loops refused) and the hidden rule is applied to where it leads, also for the index file, the listing and a.gzvariant. An FS that cannot resolve links (embed.FS, aMapFSwithout links) has none to follow; the godoc says so.TestDotfilesDirFS888runs the whole table over a realos.DirFS(file link, dir link, chain, link into.well-known,.well-known/g -> ../.git, a hidden name linking out,lnkrel,lnkabs): red onf7c79a1, green now. The cost is oneLstatper path segment, only for an FS that implementsReadLinkFS(not measured).HiddenPathsplits on "/" only (and on "" on Windows); a segment that holds a backslash is hidden if any piece of it starts with a dot, so/.git/..\xand/.s\..\y(real files in the test tree) are 404/403..gzvariant that links to a hidden file): fixed: it is no variant, the identity file is served (Root and FS).Server.Static/Group.Staticmapfs.ErrNotExistto 404, which also closes the 500-on-a-dangling-link difference.Newand at route registration (a later chdir does not move it; documented).TestDotDotRootCostsAboutADotRoot888pins the allocations to an absolute root's (about 104, against 388 on the review's measurement; an absolute root costs about 100).Dotfilesvalue panics atNew(TestDotfilesPolicyIsValidated888).ctxkit.FilesystemRootmakes the (cleaned) root absolute against the working directory with symlinks resolved, then asks whether it is the root of its volume (isRootOf: a single separator after the volume name, or a UNC share with nothing after it). Failing-first (against the field with no behaviour):TestFilesystemRootIsRefused888(12 spellings: "/", "//", "/.", "/../..", "." and "./" at "/", ".." from /usr, "../../.." up to and beyond "/", symlinks to "/"),TestStaticRoutesRefuseAFilesystemRoot888, ctxkitTestIsRootOf(with the Windows drive, UNC and device rows, which run on every OS) andTestFilesystemRoot: 16 FAIL lines, 5 PASS (the ordinary roots, the acknowledged cases and the example, which pass before and after); all green now. Alink/..root is cleaned lexically first, as the package cleans every root, so it names the link's parent directory and is not refused. FS configs have no root and are unaffected;Context.FileFromDiris unchanged (its godoc warns that a base of "/" serves every file the process can read).Mutants (each turned a named test red): filesystem root: symlinks not resolved (3 failing tests), not made absolute (3), acknowledgement ignored (2), Static/Group.Static not checked (1), the check applied to an FS config (1), any path counted a root (3), UNC root missed (1, the ctxkit table); non-regular: the
fileFromDircheck off (1), theserveOScheck off (1, after I added the conditional-request row: it was an equivalent mutant before), theserveFSpre-stat off (1), theContext.Filepre-stat off (1); symlinks under FS not resolved (1), backslash pieces off (3), the variant error not skipped (1), the 404 mapping off (1), the ".." root not resolved once (1),Dotfilesnot validated (1).Runs: the new tests
-race -count=10ok; the wholemiddleware/staticandinternal/ctxkitunder-raceok (the static package in a 1 GiB container); the whole root package under-raceok (49 s; it was killed by a 3 GiB container cap that earlier uncapped runs did not need, so it ran uncapped); the static selection 1105 PASS lines, 0 FAIL, 0 SKIP at the last full run; the native darwin whole static and ctxkit packages ok (the device cells skip there by design); golangci-lint v2.14.0 0 issues (linux, darwin);go vetwindows, darwin, linux clean (the FIFO helper is behind build tags);mage apiCheckok.Findings, not fixed here
C:..(climbsdoes not strip the volume: the round-1 flaw in drive-relative spelling) and 8.3 short names underConfig.FSwithos.DirFS(no resolution of short names;RootandServer.Staticresolve them throughEvalSymlinks).DotfilesDenya visible link to an existing.envis 403 and a dangling link is 404. Writing into the served tree already gives an attacker more; documented as "the served tree must not be writable by untrusted parties".Dotfilesgodoc anddoc.go.os.Rootwould close the containment half and is a larger change.Context.FileFromFSkeeps serving hidden names (caller-chosen, likeFileFromDir); its godoc does not say so yet.LimitReaderdefence is not covered by a test.The ".." roots (F1 of the security review)
PathWithinspecial-cased "." and not "..". For a base of "..",../../etc/passwdstarts with ".." and a separator, so it was within; the oldHasPrefix(p, base+"/")had the same flaw, so main is equally affected. Measured over the wire by the reviewer (std engine, raw request line):Root: "../.."withGET /../TOPSECRET.txtanswers 200 with a file outside the root;Root: ".."withGET /../../TOPSECRET.txtalso;FileFromDir("..", "../TOPSECRET.txt")returns 200, soServer.StaticandGroup.Staticare affected; identical on origin/main. The new matrix reproduces it on the std and the epoll engine for the middleware,Server.StaticandGroup.Static.Who was exploitable on v1.5.x: only a root (static
Root, or therootofServer.Static/Group.Static/FileFromDir) whose cleaned form is made only of ".." segments:"..","../","./..","../..","x/../..","../x/.."and so on. A named relative root that climbs first ("../public") was not affected: a request cannot leave it, because its cleaned form../public/..no longer starts with../public/.".","./"and"/"were inert, not exploitable. A ".."-rooted config is plausible only for a binary started from a subdirectory (cmd/server,bin/) that serves its project root; no use of it was seen in this repository.The fix: a relative base that starts with ".." and the path are made absolute against the working directory with its symlinks resolved (a logical working directory has a different parent than the operating system climbs to), and compared as absolute paths.
ResolvedWithinuses the same step when the lexical comparison refuses, so a symlink that climbs through the root's own name (../c/aunder Root "." in a directory named c) is served (F4) and one that leaves is refused.Failing-first for this part (docker golang:1.27.3,
--cpus 2,-race), on the head before the fix (8dd6afaplus the merge of main):TestDotDotRootsMiddleware888,TestDotDotRootsFileFromDir888,TestDotDotRootsOverTheWire888(std and epoll x middleware, Server.Static, Group.Static) red for the roots "..", "../", "./..", "x/../..", "a/../..", "../x/.." (39 FAIL lines, 94 PASS in that selection; the "../.." and "../../.." roots were added with a file above the tree after that count),PathWithinrows red in ctxkit. The matrix: 21 root spellings (".", "./", "x/..", "sub/..", "..", "../..", "../../..", "../x/..", "../c", "../other", "sub", the absolute dir with and without a trailing slash, "/") x every file of a tree with secrets outside the root (above it, in its parent, in a sibling) x requests (the file's path relative to the resolved root, which climbs for an outside file; extra climbs,./,//,x/../, encoded%2e%2eand..%2f, backslashes, NUL raw and encoded, absolute paths) x entry points. The oracle is independent of the code: the resolved directory is the working directory joined with the root. Final head: 405 PASS lines, 0 FAIL in the static selection;-race -count=10of the matrix ok. Mutants, all killed: the absolute step reverted (5 failing tests across the packages), the "." climb test removed (9), the relative fallback ofResolvedWithinreduced to the absolute-only case (4).Also added: HEAD and Range rows per spelling,
in-up(the symlink that climbs through the root's own name), theerrcheckfix in the wire harness (golangci-lint 2.15 flags a bare<-done; the CI pin 2.14 did not), and the Server.Static / Group.Static / FileFromDir / Config.Root / doc.go documentation of the three spellings, the ".." rule and the upgrade hazard.Cause
Two comparisons of the cleaned path with
root + "/". No cleaned path below "." starts with "./" (filepath.Joindrops it), and none below "/" starts with "//". SoserveOSpassed every request below these roots on (guard atstatic.go:179), andfileFromDir(context_response.go, lexical check and the symlink recheck) refused every file with a 400. The #1131 testTestRootDotAndSlashServeOnlyTheRootIndex888pinned this.The fix
One comparison,
ctxkit.PathWithin(base, p)(internal):pis below ".", when it is relative and does not climb ("..", "../x"; a file named "..a" is fine); below a base that ends in a separator ("/",C:\) when it starts with the base itself; else the oldbase + septest.ctxkit.ResolvedWithindoes the same forEvalSymlinksresults and handles one more case: under a relative base a resolved path can be absolute (a symlink with an absolute target) and still inside, so the base is made absolute and resolved, only when the cheap comparison failed. It serves the five sites: static's lexical guard, its browse recheck,resolvesInside(lexical and resolved), andfileFromDir's lexical check and symlink check.resolvesInside'scleanRoot == "."branch andpathUnderare gone. Exported API unchanged (mage apiCheck: matches 53 packages;ctxkitis internal).Tests (failing first)
Test commit first, red on main (docker golang:1.27.3,
--cpus 2,-race): static-run 'TestRootSpellings|TestRootSlash|TestRoot304': 160 FAIL lines, 100 PASS (the passes are the controls and the unchanged 304 tests); the root packageTestFileFromDirDotAndSlashBases888: red (every file below "." and "/" a 400). The same selection on the fix: 263 PASS lines, 0 FAIL at the final head (static,-v).TestRootDotAndSlashServeOnlyTheRootIndex888is removed;TestRoot304DotRootDoesNotFollowSymlinkOut888is flipped in the same commit (a path below "." now reaches the containment check and answers 400 with no validators, where it was "passed on").root_spellings_888_test.go, for each of ".", "./" and "/" (the last by absolute path; skipped on Windows), always against the bytes of the 200: serves the tree (index, nested,/suband/sub/,/./x,/sub/../x, dotfiles and dot directories); conditional requests (304 with the tag, includingGET /); in-tree symlinks (relative, absolute, to a directory) served and 304; escaping symlinks (absolute, relative, directory, climbing index) refused with 400, no validators, with and without If-None-Match,*, If-Modified-Since; a differential traversal test against the same tree under an absolute Root (..,%2e%2e,..%2f,%5c, backslashes,//, NUL bytes, also with Browse and SPA): equal answers, never the secret; Browse of a symlinked directory outside refused; and the same requests over the wire (real std server, raw requests, nothing cleans the path first).file_from_dir_root_888_test.go:FileFromDirwith ".", "./", "/": served, symlinks,.., absolute path, NUL, backslash, directory; the empty base; a relative base with an absolute in-tree symlink.internal/ctxkittable tests forPathWithinandResolvedWithin.Mutants, all killed (each run on the static, root and ctxkit selections): "." branch off (11 red), the climb test removed (8), the separator-base branch removed (9),
ResolvedWithinalways true (9), no absolute fallback (3), the "." absolute test removed (6), the old guard back inserveOS(9),resolvesInside's lexical check bypassed (2), the old guard infileFromDir(8), the old symlink check infileFromDir(8), the browse recheck off (1, killed by the Browse test added for it, which is also red on main:/in-plain/is passed on there).Runs: static and root selections
-race -count=10ok;middleware/staticandinternal/ctxkitwhole packages once under-raceok; the whole root package once under-race(51 s) ok, at the first fix head; the selection re-run natively on darwin (where the temp dir is under/var, a symlink to/private/var, so the absolute-symlink slow path runs with a logical working directory) ok. Windows: vet only (the "/" rows skip there). golangci-lint v2.14.0 0 issues (linux and darwin);go vetwindows, darwin, linux clean.Security
Config.AllowFilesystemRootis set, see Round 2). Probe in the container with Root "/" (before that decision):GET /etc/passwd200 (839 bytes),GET /proc/self/environ200 (the process's environment, 226 bytes),/proc/self/cmdline200,/../../etc/passwd200 (".." stops at the root). Everything the process can read is served; that is what the config asks for. The containment checks hold by construction, they just have nothing to exclude;TestRootSlashHasNoOutside888pins it.Config.Root's godoc anddoc.gosay so, and/proc/self/environis named in the godoc.Server.StaticorGroup.Static(see the section above);.envand.git/configanswered 200 under "." before this rule (measured by the review).Allowrestores it.Root: "."config was inert; after it, it serves the working directory (sources,.env,.git). The godoc onConfig.Rootstates it and tells the reader to name a directory that holds only what is public...escapes, absolute paths (joined as relative), symlinks (file, directory, climbing index, with and without validators, Browse), encoded slashes and dots (c.Path()is the raw path, so they name files that do not exist), backslashes, NUL and doubled slashes: same answer as under an absolute Root in the differential test, none returns the secret, under "." and "./". Under "/" the traversal rows are the "nothing is outside" rows above.FileFromDir(""):filepath.Clean("")is ".", which would have made an unset root (s.Static("/f", os.Getenv("DIR"))) serve the working directory. It is refused with a 400 as before (a test and a commit of its own).EvalSymlinks) runs only when a relative-root request crosses a symlink with an absolute target; the common path adds one comparison.fileFromDirstill resolves the working directory per request for a relative base, as the OS does when it opens the path; achdirafterNewchanges what "." means (the doc states it).