Repository navigation
Reach for Go and Rust methods: receiver types from dependency code, and targets inside units - #53
Conversation
…OU-545) A Rust file with use smallvec::smallvec made the link step die with a stack overflow. The use binds smallvec. The walk of the path read the first smallvec through the same binding, and that binding led back to the same use, with no end. Rust resolves the first name of a use path against the crates and the modules, never against the binding of the same use. resolveRust now keeps the paths that each file resolves now. A repeat of a path gives no answer, so the walk tries the next candidate and ends. The path then names the crate, as smallvec//smallvec. The crate selectors 0.22.0 showed a second cycle of the same kind. An impl of Display for the enum Component holds use self::Component::*. The base Display can resolve through that wildcard, which asks for the external base of Component again. findExternalBase now keeps the classes that it reads, and a repeat has no external base. A link test runs use smallvec::smallvec, use a::a, use a::a::a and use self::a as a, and a second one the wildcard of the class in its impl. Each one overflowed the stack before the fix. The case rust/use-crate-name links a crate with the use and finds SmallVec::new. No graph of an existing case changes, so the version stays, and the hash rows of 11/11 take the new case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
…ncertain (FOU-542) The word "often" had no number behind it. This change gives the number and the rows that later fixes must flip. receivers_test.go has one row per form of the receiver of a method call of a dependency type, for Go and for Rust: a declared variable, a parameter, a struct field, a first-party return, an external return, a multi-return, a chained call and an interface return, and for Rust a reference and generic parameter, an associated function, a clone, a trait object and a generic bound. Small synthetic units under testdata/receivers give the dependency code. Each row names its answer. A row that fails today names the issue that fixes it, FOU-543, FOU-544 or FOU-546, and its answer today, so go test stays green. methods_test.go measures real projects under the projectgraph tag: 10 Go and 10 Rust projects that depend on a module or a crate with method symbols in the Go vulndb or in RustSec. corpus.txt pins each project and each dependency, and advisories.json pins the advisories. Every symbol of an advisory that affects a pinned dependency is a target, and the code of the dependency is a unit. scripts/fetch_method_corpus.sh fetches the code. The test counts found, not-found, uncertain by cause and no-such-target, per language and per symbol kind. report.md has the first numbers. 0.227 (294/1298) of the Go method targets and 0.342 (13/38) of the Rust ones end as unknown-receiver. The first run found a second stack overflow in the Rust link step, in selectors 0.22.0. The FOU-545 commit holds that fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
…n-receiver (FOU-546) html.NewTokenizer(r).Next() with no dependency code gave no-such-target for golang.org//x//net//html//Tokenizer//Next and left no unresolved record. The walk took Next as a member of the external key of the called name, so the call made an edge to html//NewTokenizer//Next, a key that names no symbol. With no record, a later rule that reads no-such-target as not called for an imported package would hide a real call. The link step now adds an unknown-receiver record with the method name when it takes the method of a call on a call result as a member of the key of the called name. The edge stays, because a Go call of a type name converts a value and the method of the type then runs. A construction gives an instance of the called class, so new Yaml().load() and Yaml().load() of Python or Kotlin get no record. The reach query then answers uncertain at the site of the call through the method scope of the record. Python, JavaScript and Rust chains get the same record. reach answers no-such-target as before when no uncertainty concerns the target. The graph keeps no record of the imports of an external package, so the query cannot tell an imported package from another one, and this change does not turn no-such-target into not-found. The case go/chained-external-call checks the record, the kept edge and the uncertain answer. The cases go/interface-method-set, javascript/commander-action and javascript/references-argument gain the record of a call on a call result. The Go chained rows of the receiver table move to unknown-receiver, and FOU-543 now fixes them. projectgraph.Version is 12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
…he data of tables (FOU-547) A -deps run with golang.org/x/net@v0.61.0 printed seven truncations of the package-clauses limit, a budget line for idna/tables15.0.0.go and unresolved calls of _test.go files. Each gen.go has //go:build ignore and package main, so the link step saw a second package clause in its directory. The table file hit the depth limit of the index with a string of thousands of parts. Each truncation and budget turned a reach answer for the module into uncertain, so the dependency code made the answer worse. The index step now leaves out a Go file of a dependency unit only when no Go build reads it by default: a _test.go file, a file with //go:build ignore, and a file whose constraint needs a tag that the go command never sets, as purego. go/build decides for each GOOS and GOARCH with the bytes that the run read. The index keeps the files of every platform, because a reach answer must hold for each platform that a project builds for. An index of one platform dropped the darwin twin of a function, and a call that only darwin makes read as no-such-target. Result.Excluded lists the files. A file of the project stays as it is. A generated Go file of a unit, or a tables*.go file, whose index hits a limit gets a second index from its skeleton: each top-level var value that holds only data blanked, with the lines and the columns kept. Function bodies and values with a call or a function literal stay, so the calls of a generated file stay in the graph. A skeleton without bodies dropped every call of a .pb.go file and marked it complete. The index keeps the hash of the file, so the store serves it on the next run. A skeleton that still hits a limit stays truncated. The same x/net run now prints no truncation and no budget line. An indexer test checks a unit with gen.go, a _test.go file, a purego file, a //go:build windows file, a _linux.go file and a large generated table whose body calls a function. The cases go/dependency-build-files, go/dependency-generated-code and go/dependency-platform-files check the graph and the reach answers. fileindex.Version is 12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
…l functions and methods (FOU-543) z := html.NewTokenizer(r); z.Next() gave uncertain with unknown-receiver even with the code of x/net as a unit. The link step read the result type only of a first-party function, and only its first result. The Go index records the type of each result by position, in Results, and each name of an assignment or a short variable declaration that one call binds, with the position in Typed.Index. The blank name binds nothing. The link step reads the definition of a function in the project or in the unit that gives its package, and binds the type of the result to the name. A type of a unit becomes its external key, as golang.org//x//net//html//Tokenizer, so the call gets the key that a target names, and a package edge joins the key to the definition. A walk through an external key takes the result type of each function or method of a unit in turn, so pkg.F().M() and a.B().C() resolve. An interface result keeps the cha edges, and an implementation in a unit gets its edge through its external key. With no implementation the record has the new reason interface-receiver. A result of a package that no unit gives has the new reason no-dependency-source, and so has the record of a call on a call result from FOU-546. The standard library is a unit too: GOROOT/src as go:std@<version> gives each import path whose first element has no dot. To make that unit usable, the index step now also leaves out the files of a Go unit that are not Go source, the files under testdata and under a directory that starts with _ or ., and the cmd module of the std unit. The skeleton of a generated table blanks const values too, and an older DO NOT EDIT line or a name that starts with table marks such a file. The real GOROOT then links with no truncation, and http.NewRequest followed by req.Write finds net//http//Request//Write. The Go rows of the receiver table now pass. The case go/dependency-result-types checks the forms against a unit and a std unit, and go/dependency-implements-interface now finds Impl.Run by its external key. report.md has the numbers after this change. The govulncheck set still finds 9 of 9 stacks, and pycg-micro keeps 0.954. fileindex.Version is 13 and projectgraph.Version is 13. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
…t, reference and generic types (FOU-544) A -deps run with a crate printed "the link step does not link the rust code of cargo:smallvec@1.13.2", and the link step knew a Rust receiver type only from Type::new. So a call on a parameter, a field, a result or a clone of a type of a crate read as unknown-receiver. The link step now links the Rust code of a cargo: unit. Each space has its own crates, and the root module of a crate of a unit is the unit, so two versions of one crate stay apart. A path of another crate, with or without a use, records its crate as the origin of the key, and the unit that gives the crate links it. The external key of an item of a unit is the path that another crate writes, with the shortest pub use of the root. A Rust unit has a surface, so a summary lists its symbols. The Rust index records the return type of each function and method, with Self as the type of the impl. Option<T> and Result<T, E> keep T in Returns and the wrapper in the new field Wraps. unwrap and expect give T, and so does a ?, which the index does not record. Another method of the wrapper gives no type. A declared type of a parameter, a field or a local that a unit gives binds the receiver, and an associated function binds its result. clone, to_owned, default and from on a type of a unit keep the type also when the crate derives them. A trait of a unit, as a trait object or a generic bound, gets the cha edges to its impls through their external keys, or a record with the new reason trait-receiver. The Rust rows of the receiver table now pass, and each probe of the real smallvec 1.13.2 finds SmallVec::insert_many. The case rust/dependency-crate checks a crate with a private module, a Result that unwrap opens and a generic bound. report.md has the numbers after this change. The govulncheck set still finds 9 of 9 stacks. fileindex.Version is 14 and projectgraph.Version is 14. The cache of the type that a unit gives for a key now skips an answer that the cycle guard of unitDefinitions shaped. An outer walk of the same name made it miss a definition, and the cache kept the miss for later queries, so the answer depended on the order of the walk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
…ns (FOU-555) A call inside a unit of dependency code goes to the definition, as go:example.com/dep@v1.0.0//dep.go//Vuln, and not to the external key example.com//dep//Vuln that a target names. So the search never met the target. A target that only the code of its unit calls read as no-such-target, or as not-found when an unreached call of the project named it. This is how most vulnerable symbols are called, as the methods of the HTTP/2 server of x/net. With dependency units, vet would say "not called" for code that runs. The link step now sets Node.External on each function, method and constructor of a unit: the key that an importer writes for it. reach matches a target against the external node of its key and against each definition with that external key, also for a wildcard target. The call sites of a target leave out the package edges, which join a key to its definition. The match showed a second gap. A Rust library crate with a build.rs had the build script as its only root, so the library-surface roots were off, and a call of its public API read as not-found. cookie calls rand::thread_rng from Key::generate this way. A build script, an example or a benchmark of a Rust crate is now a helper program: its root stays, and it does not turn off the library-surface roots. The cases go/dependency-internal-call, rust/dependency-internal-call and rust/library-build-script check both. The method measure now finds 331 of 1,298 Go method targets (30 before), 51 of 256 Go function targets (13) and 16 of 38 Rust method targets (0), and no target is not-found. projectgraph.Version is 15 and reach.Version is 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>
SafeDep Report SummaryPackage Details
This report is generated by SafeDep GitHub App |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
vet Summary ReportThis report is generated by vet Policy Checks
Malicious Package AnalysisPackages were checked against SafeDep's known malicious packages database Malicious Package Analysis Report
Changed PackagesChanged Packages
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #53 +/- ##
==========================================
+ Coverage 76.49% 76.71% +0.22%
==========================================
Files 165 167 +2
Lines 15629 16126 +497
==========================================
+ Hits 11955 12371 +416
- Misses 2964 3020 +56
- Partials 710 735 +25 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e0a52c4dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Important Review skippedToo many files! This PR contains 237 files, which is 137 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configuration
📒 Files selected for processing: 237
You can disable this status message by setting the
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A Cargo.toml can rename a dependency, as helper = { package = "kit" }.
A path of the crate writes helper, and the unit has the crate kit, so no
package edge reached the unit and a target of kit was not-found. The
link step now reads the renames and gives each origin the module of the
package.
Two platform files of a Go package can define one function with two
result types. The walk took the type of the first file, so a method of
the other type was not-found. A result is now known only when every
definition and its twins give the same type. Else the call is a record
with the reason unknown-receiver.
projectgraph.Version is 16. New cases: rust/dependency-renamed and
go/dependency-platform-results.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm
Signed-off-by: Claude <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PFQbqPqd9xn1fT6hBsyoUm Signed-off-by: Claude <noreply@anthropic.com>




What this changes
vet v2 reachability needs to know whether a project calls the vulnerable function of an advisory. 74% of the Go vulnerability database symbols are methods, as
Tokenizer.Next. Most of them are called from inside the dependency itself. This PR makes the call graph answer for those symbols. One commit per Linear issue of the milestone "v2.2a Reachability MVP":use smallvec::smallvec;, or on a second cycle found in selectors 0.22.0test/projectgraph/report.md)html.NewTokenizer(r).Next(), writes anunknown-receiverrecord and does not read as no-such-target//go:build ignoreand custom tags. It keeps the files of every platform. A generated table file blanks only its data values and keeps its function bodiesGOROOT/srcas a unit?,unwrap, traits and associated functionsResults on real projects
No target of the measure reads
not-found. Every other target isuncertainwith a cause. The share of Go method targets that end asunknown-receiverfalls from 0.227 to 0.119.html.NewTokenizer(r)followed byz.Next(), with the realgolang.org/x/net@v0.61.0as a unit, is nowfound, with no truncation and no budget line.Review
A sub-agent review of the first six commits found two honesty regressions in FOU-547 and a pre-existing hole. All are fixed, with a case for each:
.pb.gocall read as no-such-target. Casego/dependency-generated-code.GoPlatformoption and the-go-platformflag are gone. Casego/dependency-platform-files.go/dependency-internal-callandrust/dependency-internal-call. Fixing it showed that a Rust library with abuild.rshad no library-surface roots. Caserust/library-build-script.Two findings stay open as FOU-556: false edges from the Rust
?heuristic, andno-dependency-sourcenoise on chained builtins. Neither hides a call.Versions
fileindex.Version11 to 14,projectgraph.Version11 to 15,reach.Version2 to 3. Each bump has its rows intest/projectgraph/version_test.go.Checks
go build ./...,go vet ./...andgo test ./... -count=1pass at each commit. At the head,go test -tags projectgraph ./test/projectgraph/andgolangci-lint run ./...pass with 0 issues. The measure runs withscripts/fetch_method_corpus.shandPROJECTGRAPH_METHOD_CORPUS.Generated by Claude Code