Reduce initial full-index work on large codebases - #5390
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Initial full indexing spends substantial work repeating declaration probes, receiver-scope analysis and reference-source SQL for large repositories. This PR reduces that work across 10 commits:
Names, extracted records, resolution rules, database layout, transaction boundaries and rollback behavior remain unchanged.
Validation
dotnet test CodeIndex.sln -c Release -p:UseSharedCompilation=false: .NET 8 12,733 passed / 9 skipped; .NET 9 12,239 passed / 435 skipped. Zero failures, errors, timeouts or aborted tests.origin/main..HEAD: No blocking/actionable issues found.Measurements
On the same C#-heavy source snapshot, the first two batches each had their own alternating Release .NET 8 before/after measurements: mean 43.6 → 37.7 seconds (~13.6%) and 37.40 → 34.51 seconds (~7.7%). These are separate comparisons, not a combined final-branch speedup.
The final batch has standalone Debug .NET 8 observations of 39.014 / 40.010 seconds, with no matched Release before/after timing; no percentage improvement is claimed for it. Details and allocation observations are in
docs/initial-index-performance.md.Documentation and changelog
Updated
TESTING_GUIDE.mdand the bilingual performance documentation. Added these bilingual fragments:changelog.d/unreleased/+fresh-canonical-source-probes.changed.mdchangelog.d/unreleased/+csharp-receiver-scope-reuse.changed.mdchangelog.d/unreleased/+csharp-declaration-prefix-pruning.changed.mdchangelog.d/unreleased/+fresh-reference-flag-work.changed.mdchangelog.d/unreleased/+csharp-reference-probe-pruning.changed.mdchangelog.d/unreleased/+csharp-header-probe-pruning.changed.mdchangelog.d/unreleased/+csharp-lookahead-copy-reuse.changed.mdchangelog.d/unreleased/+csharp-callable-receiver-cache.changed.mdchangelog.d/unreleased/+structural-mask-code-spans.changed.mdFollow-up candidates
Collect matched Release timings for the final batch and extend measurements to large corpora with a broader language mix.