Skip to content

Fail fast on reftable repositories, and pin clone to files ref storage - #2121

Open
Tyrie Vella (tyrielv) wants to merge 1 commit into
microsoft:vnextfrom
tyrielv:tyrielv/reftable-repositoryformat-fix
Open

Tyrie Vella (tyrielv) wants to merge 1 commit into
microsoft:vnextfrom
tyrielv:tyrielv/reftable-repositoryformat-fix

Conversation

@tyrielv

@tyrielv Tyrie Vella (tyrielv) commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Problem and Context

VFS for Git assumes the "files" ref storage format (loose ref files plus
packed-refs). It does not populate refs through git's ref backend during a
clone. CloneVerb.TryInitRepo writes .git/packed-refs directly from the refs
the GVFS protocol returns (GitRefs.ToPackedRefs). Other code also reads and
writes loose ref files and the .git/logs reflog directly (for example
GitHeadRepairJob).

A repository initialized with git's newer "reftable" ref backend ignores
packed-refs entirely. Its refs live in .git/reftable/. So the refs GVFS
writes during a clone are invisible to git, and the enlistment is broken. This
is verifiable with stock git: in a git init --ref-format=reftable repo, a
hand-written .git/packed-refs entry does not appear in git for-each-ref.

A user who sets init.defaultRefFormat=reftable globally gets a reftable repo
from gvfs clone today, and reftable is the direction git is moving for its
default ref backend. Until now the failure was not a clear error.
RequiredGitConfig force-writes core.repositoryformatversion = 0 on clone, on
every mount, and on repair. Applied to a reftable repo, that leaves
repositoryformatversion = 0 next to the extensions.refstorage key that
requires repo-format v1, which git then refuses to parse at all
(fatal: repo version is 0, but v1-only extension found: refstorage). Because
mount re-applies the value, a hand edit back to 1 is undone on the next mount.

This is the ref-storage axis of #2119. The object-format (sha256) axis is
handled separately by #2116, which uses the same fail-fast pattern this PR
mirrors.

Changes

  • Pin git init to --ref-format=files during clone, mirroring the existing
    --object-format=sha1 pin in GitProcess.Init, so a user's global
    init.defaultRefFormat=reftable no longer produces a reftable enlistment from
    gvfs clone. The --ref-format option and the reftable backend both first
    shipped in git 2.45, so the pin is gated on the installed git version: older
    git cannot create a reftable repo and would reject the unknown option. The
    clone flow already resolves and validates the installed git version
    (GVFSVerb.CheckGitVersion), so GitProcess.Init takes that version as a
    parameter rather than shelling out for git --version again. If the version
    cannot be determined, the pin is omitted and the detection below still catches
    the repo.
  • Add GVFS.Common.Git.RefStorage. It reads extensions.refstorage and reports
    whether the repository uses the reftable backend, with the same value-parsing
    and config-read shape as the object-format helper (a missing key means the
    default "files" format; a genuine config-read failure is surfaced distinctly
    from "not reftable").
  • Fail fast with an actionable message
    (RefStorage.UnsupportedReftableErrorMessage) before any ref write or git
    config mutation, at the call sites that would otherwise reach the corrupting
    core.repositoryformatversion = 0 force-write. This covers pre-existing
    reftable repos (created outside gvfs clone) and the case where the clone pin
    was skipped because the git version was undetermined:
    • FastFetchVerb (FastFetch can target an arbitrary pre-existing repo).
    • CloneVerb.TryInitRepo, right after git init, before the direct
      packed-refs write (defense-in-depth behind the pin).
    • InProcessMount, before TrySetRequiredGitConfigSettings.
    • GitConfigRepairJob.TryFixIssues, before it rebuilds the config.
  • Treat a genuine config-read failure as fatal at mount, clone, and FastFetch. A
    missing extensions.refstorage key already reports the repository as
    files-format, so it never takes the read-failure path; a stderr-bearing read
    failure means the config is anomalous, and continuing would risk the same
    brick this guards against. Mount passes the git error to FailMountAndExit as
    a format argument, so a brace in git stderr cannot throw a FormatException
    instead of failing the mount cleanly.
  • Repair is the deliberate exception to fail-closed: it surfaces the read error
    and still proceeds, because repair exists to rebuild a corrupt config. It
    refuses a reftable repo detected either by extensions.refstorage or by the
    physical .git/reftable/ directory — the directory check still identifies a
    reftable repo when the config is too corrupt for git config to read, so a
    corrupt reftable repo is not silently rebuilt as a files-format config while
    its refs remain in reftable storage.
  • GitConfigRepairJob.HasIssue now reports a reftable enlistment as an
    unfixable issue, so gvfs diagnose / gvfs repair no longer declare it
    healthy.
  • Add tests: the version-gated --ref-format=files pin in GitProcess.Init
    including the release-candidate boundary (GitProcessTests), and RefStorage
    value parsing and config-read cases including the genuine read-failure path
    (RefStorageTests).

Notes for reviewers

@tyrielv
Tyrie Vella (tyrielv) force-pushed the tyrielv/reftable-repositoryformat-fix branch 2 times, most recently from 2dcf7fd to 07aedb9 Compare September 24, 2026 20:46
@tyrielv Tyrie Vella (tyrielv) changed the title Fail fast on reftable repositories in mount, clone, repair, and FastFetch Fail fast on reftable repositories, and pin clone to files ref storage Sep 24, 2026
@tyrielv
Tyrie Vella (tyrielv) force-pushed the tyrielv/reftable-repositoryformat-fix branch from 07aedb9 to 4f5e064 Compare September 24, 2026 21:06
@tyrielv
Tyrie Vella (tyrielv) marked this pull request as ready for review September 24, 2026 21:16

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the strongest of the six GVFS PRs currently out for review, and it is worth saying why explicitly, because it sets the bar for #2116:

  • Fails closed. A config-read failure is fatal at FastFetch, mount and clone, rather than warn-and-continue.
  • Physical fallback. The .git/reftable/ directory check still identifies a reftable repo when the config is too corrupt for git config to read — which is exactly the state gvfs repair runs in. That is the detail that makes the repair path actually safe.
  • Both halves of the repair job. HasIssue returns IssueType.CantFix as well as TryFixIssues failing, so gvfs diagnose will not call a reftable enlistment healthy.
  • Format-string safety. Passing git stderr as a format argument to FailMountAndExit rather than concatenating it is correct, and for the reason stated: FailMountAndExit only has a params object[] overload, so it does reach string.Format and a { in a crafted .git/config would otherwise throw instead of failing the mount cleanly. Nice catch.
  • Tests. 234 lines covering the version gate, the pin, the null-version fallback, the Microsoft/Git 2.45.0.vfs.0.1 shape, and the release-candidate edge. Zero AI-cruft hits and no stray .github artifacts in the diff.

I verified the version gate by hand against GitVersion.CompareVersionNumbers: it ignores Platform entirely and orders on Major/Minor/Build, then release candidate, then Revision/MinorRevision — so 2.45.0.vfs.0.1 and a bare 2.45.0 both compare as supporting --ref-format, and 2.44.x does not. The gate is correct.

Five findings, ranked — nothing blocking.


1. Medium — coordinate with #2116

You are editing the same four files at the same insertion points on the same base: GVFS/FastFetch/FastFetchVerb.cs (~204), GVFS/GVFS.Mount/InProcessMount.cs (~280), GVFS/GVFS/CommandLine/CloneVerb.cs (~857) and GVFS/GVFS/RepairJobs/GitConfigRepairJob.cs. These will conflict, and whichever merges second needs a rebase.

More importantly, please reconcile the read-failure policy: this PR is fatal, #2116 is warn-and-continue, and after both land those two checks sit adjacent inside MountWithLockAcquired and TryInitRepo with opposite semantics. I would keep yours and change #2116.

2. Low — GVFS/GVFS.Common/Git/GitProcess.cs, Init(Enlistment)

Every gvfs clone now pays an extra git --version subprocess before init. Negligible in isolation, but GVFSVerb already resolves and validates the installed git version earlier in the clone flow — worth threading that through to the new internal Init(enlistment, installedGitVersion) if it is cheap to plumb, rather than re-shelling.

3. Low — GitConfigRepairJob.ReftableBackendDirectoryExists

Path.Combine(WorkingDirectoryBackingRoot, GVFSConstants.DotGit.Root, "reftable") assumes .git is a directory. In a linked worktree .git is a file, so the fallback silently returns false — relevant given #2113 is in flight on worktree paths. Probably fine because repair targets the primary enlistment, but worth a one-line comment saying that is the assumption.

4. Low — scoping note, no action

This PR deliberately does not touch RequiredGitConfig force-writing core.repositoryformatversion=0, which is the shared root cause behind both the reftable and the SHA256 corruption, tracked in #2119. That scoping is correct and clearly stated in #2116's description — just flagging that the pre-check pattern is now replicated at four call sites across two PRs, so when #2119 lands there will be eight of them to unwind. Worth a note on #2119 so that cleanup is not forgotten.

5. Low — praise, keep doing this

SupportsRefFormatOption_IsFalseForReleaseCandidateOf245 documenting a known-conservative gate as "a missed optimization, not a correctness problem" is exactly the right kind of test comment: it records a deliberate decision and stops a future reader "fixing" it. Same for Init_DoesNotPinRefFormat_WhenGitVersionUnknown pinning down the fallback that makes the CloneVerb check defense-in-depth rather than redundant.

@tyrielv

Copy link
Copy Markdown
Contributor Author

Note

🤖 Machine-drafted, reviewed and approved by Tyrie Vella (@tyrielv) before posting.

Thanks for the depth here — especially verifying the version gate by hand against CompareVersionNumbers and confirming the FailMountAndExit format-string reasoning. All five addressed; pushed in 67268643.

1. Coordinate with #2116. Agreed, and agreed on which way to reconcile. I've moved #2116 back to draft; this PR lands first and #2116 rebases on top. When it does, I'll change #2116's SHA256 read-failure handling from warn-and-continue to fail-closed so the two adjacent checks in MountWithLockAcquired and TryInitRepo share one semantics (yours). One asymmetry worth noting: reftable has a physical .git/reftable/ fallback that lets repair stay safe when the config is unreadable; objectformat lives only in config, so SHA256 repair has no equivalent signal — I'll keep that limitation explicit rather than pretend to cover it.

2. Extra git --version on Init. Fixed — threaded through. GitProcess.Init now takes the resolved GitVersion, and CloneVerb passes the version the clone flow already validated (enlistment.GitVersion) instead of re-shelling. Collapsed the two Init overloads into one; the old 1-arg test that leaned on a real git --version spawn failure is gone.

3. .git-as-file in a linked worktree. Switched to this.Enlistment.DotGitRoot — the abstraction the whole codebase already uses to encode ".git is a directory at WorkingDirectoryBackingRoot\.git" — and added a comment stating the assumption and that a linked-worktree .git file would need gitdir resolution first. As you said, fine in practice because repair targets the primary enlistment.

4. Scoping note. Confirmed — no PR touches the RequiredGitConfig force-write; #2116 and #2121 only guard the two known axes (objectformat, refstorage). I filed the note on #2119 so the eight pre-checks (four sites × two PRs) get unwound when the root-cause fix lands.

5. Appreciated — I'll keep documenting deliberate-conservative decisions in the test comments.

VFS for Git assumes the "files" ref storage format. It does not populate refs
through git's ref backend during a clone; it writes .git/packed-refs directly
from the refs the GVFS protocol returns (CloneVerb.TryInitRepo and
GitRefs.ToPackedRefs). A reftable-backed repository ignores packed-refs, so
those refs are invisible to git and the enlistment is broken. Other code reads
and writes loose ref files and the .git/logs reflog directly.

A user who sets init.defaultRefFormat=reftable globally gets a reftable repo
from 'gvfs clone' today, and reftable is git's future default ref backend.
Until now that produced no clear error: RequiredGitConfig force-writes
core.repositoryformatversion=0 on clone, on every mount, and on repair, which
leaves repositoryformatversion=0 next to a v1-only extension that git then
refuses to parse at all.

Pin 'git init' to --ref-format=files during clone, mirroring the existing
--object-format=sha1 pin, so init.defaultRefFormat=reftable no longer produces
a reftable repo. The option and the reftable backend both first shipped in git
2.45, so the pin is gated on the installed git version; older git cannot create
a reftable repo and would reject the unknown option. The clone flow already
resolves and validates the git version (GVFSVerb.CheckGitVersion), so
GitProcess.Init takes that version as a parameter rather than shelling out for
'git --version' again. If the version is unknown, the pin is omitted and
detection still catches the repo.

Add GVFS.Common.Git.RefStorage to detect extensions.refstorage=reftable and
fail fast with an actionable message before any ref write or config mutation,
at the call sites that would otherwise reach the corrupting force-write:
FastFetch, clone (TryInitRepo, right after git init), mount (before
TrySetRequiredGitConfigSettings), and repair (before the config rebuild). This
mirrors the object-format (sha256) fail-fast helper and covers pre-existing
repos and the git-version-undetermined case that the clone pin cannot.

A genuine config-read failure is fatal at mount, clone, and FastFetch: a
missing key already reports the repository as files-format, so a read error
means the config is anomalous and continuing risks the very brick this guards
against. Mount passes the git error to FailMountAndExit as a format argument so
a brace in git stderr cannot throw a FormatException instead of failing the
mount cleanly. Repair instead surfaces the read error and proceeds, because
repair exists to rebuild a corrupt config; it refuses a reftable repo detected
either by config or by the physical .git/reftable/ directory, so a corrupt
reftable repo is not silently rebuilt as files. GitConfigRepairJob.HasIssue also
reports a reftable enlistment as an unfixable issue so 'gvfs diagnose'/'gvfs
repair' no longer call it healthy.

Add unit tests for the version-gated ref-format pin (including the release
candidate boundary and the version-unknown fallback) and for RefStorage value
parsing and config reads, including the genuine read-failure path.

Assisted-by: Claude Opus 4.8
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
@tyrielv
Tyrie Vella (tyrielv) force-pushed the tyrielv/reftable-repositoryformat-fix branch from 6726864 to 2548e41 Compare September 25, 2026 23:58
Tyrie Vella (tyrielv) added a commit to tyrielv/VFSForGit that referenced this pull request Sep 26, 2026
PR microsoft#2121 adds an equivalent fail-fast check for reftable repositories
(extensions.refstorage) at the same four call sites this PR already
checks for SHA256 repositories (extensions.objectformat): FastFetch,
'gvfs clone', 'gvfs mount', and 'gvfs repair'. Reviewing both together
surfaced a mismatch: this PR's ObjectFormat check warned and continued on
a genuine config-read failure, while microsoft#2121's RefStorage check fails
closed. Once both checks sit side by side at each call site, having one
warn-and-continue and the other fail-closed for the same class of error
(an unreadable config, as opposed to a missing key) is inconsistent and
confusing. Align this PR's policy with microsoft#2121's, since fail-closed is the
more defensible default: a missing extensions.objectformat key already
reports the repository as SHA1 through the success path, so the error
path only fires when the config is genuinely anomalous, and continuing
in that state risks the very corruption this check exists to prevent.

- FastFetch, clone (TryInitRepo), and mount: a config-read failure is now
  fatal, matching a missing/successfully-read config being the only
  non-fatal outcomes.
- Mount passes the read error to FailMountAndExit as a format argument
  ("{0}") rather than concatenating it into the message string:
  FailMountAndExit routes through ITracer.RelatedError(string, params
  object[]), which runs string.Format, so a literal '{' in git's stderr
  (possible with a crafted or corrupt .git/config) would otherwise throw
  a FormatException instead of failing the mount cleanly.
- Repair (GitConfigRepairJob) keeps its existing exception to this
  policy: a config-read failure there is surfaced as a warning but does
  not block the repair, because repair's whole purpose is to rebuild a
  corrupt config. Unlike ref storage format, SHA256 has no physical
  on-disk marker independent of git config, so - unlike microsoft#2121's
  reftable check, which falls back to checking for a .git/reftable/
  directory when the config is unreadable - a SHA256 repo whose config is
  also corrupt cannot be distinguished from an ordinary corrupt SHA1 repo
  here. This is an inherent limitation of the object-format axis, not
  something to fix in this change.
- Added a HasIssue check to GitConfigRepairJob so a SHA256 repository is
  reported as an unfixable issue (matching microsoft#2121's structure), instead of
  'gvfs diagnose'/'gvfs repair' treating it as healthy.

Added a unit test locking in that the non-Try IsSha256Repo overload still
swallows a genuine read failure and reports "not SHA256" (only the Try
overload distinguishes it), mirroring the equivalent RefStorage test.

Assisted-by: Claude Opus 4.8
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Tyrie Vella (tyrielv) added a commit to tyrielv/VFSForGit that referenced this pull request Sep 26, 2026
PR microsoft#2121 adds an equivalent fail-fast check for reftable repositories
(extensions.refstorage) at the same four call sites this PR already
checks for SHA256 repositories (extensions.objectformat): FastFetch,
'gvfs clone', 'gvfs mount', and 'gvfs repair'. Reviewing both together
surfaced a mismatch: this PR's ObjectFormat check warned and continued on
a genuine config-read failure, while microsoft#2121's RefStorage check fails
closed. Once both checks sit side by side at each call site, having one
warn-and-continue and the other fail-closed for the same class of error
(an unreadable config, as opposed to a missing key) is inconsistent and
confusing. Align this PR's policy with microsoft#2121's, since fail-closed is the
more defensible default: a missing extensions.objectformat key already
reports the repository as SHA1 through the success path, so the error
path only fires when the config is genuinely anomalous, and continuing
in that state risks the very corruption this check exists to prevent.

- FastFetch, clone (TryInitRepo), and mount: a config-read failure is now
  fatal, matching a missing/successfully-read config being the only
  non-fatal outcomes.
- Mount passes the read error to FailMountAndExit as a format argument
  ("{0}") rather than concatenating it into the message string:
  FailMountAndExit routes through ITracer.RelatedError(string, params
  object[]), which runs string.Format, so a literal '{' in git's stderr
  (possible with a crafted or corrupt .git/config) would otherwise throw
  a FormatException instead of failing the mount cleanly.
- Repair (GitConfigRepairJob) keeps its existing exception to this
  policy: a config-read failure there is surfaced as a warning but does
  not block the repair, because repair's whole purpose is to rebuild a
  corrupt config. Unlike ref storage format, SHA256 has no physical
  on-disk marker independent of git config, so - unlike microsoft#2121's
  reftable check, which falls back to checking for a .git/reftable/
  directory when the config is unreadable - a SHA256 repo whose config is
  also corrupt cannot be distinguished from an ordinary corrupt SHA1 repo
  here. This is an inherent limitation of the object-format axis, not
  something to fix in this change.
- Added a HasIssue check to GitConfigRepairJob so a SHA256 repository is
  reported as an unfixable issue (matching microsoft#2121's structure), instead of
  'gvfs diagnose'/'gvfs repair' treating it as healthy.

Added a unit test locking in that the non-Try IsSha256Repo overload still
swallows a genuine read failure and reports "not SHA256" (only the Try
overload distinguishes it), mirroring the equivalent RefStorage test.

Assisted-by: Claude Opus 4.8
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants