Skip to content

Make NuGet walk tests ignore the host package cache - #2037

Merged
Mikola Lysenko (mikolalysenko) merged 2 commits into
mainfrom
ci-janitor/nuget-crawler-hermetic
Nov 9, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 2 commits into
mainfrom
ci-janitor/nuget-crawler-hermetic

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #2023.

Problem

crawlers::nuget_crawler::tests::solution_roots_and_unrestored_projects_keep_the_walk fails every time on a machine with any package in ~/.nuget/packages. It passes in CI only because the runners' NuGet cache is empty. Issue #2023 has the repro.

While reproducing it I found a second test with the same dependence, an_uncovered_sub_project_keeps_the_walk. That one is also order-dependent: neither test was #[serial], so both ran alongside the serial tests in the same module that point NUGET_PACKAGES and HOME at their own warm folders. With one host package planted (~/.nuget/packages/zz.hostpkg/9.9.9), 30 loops of cargo test -p socket-patch-core --lib nuget_crawler on the partial fix gave:

  • 15 failures of solution_roots_… (left had an extra pkg:nuget/zz.hostpkg@9.9.9)
  • 8 failures of an_uncovered_…, which passed only in runs where a concurrent serial test happened to have NUGET_PACKAGES set

Root cause

An unscoped project-mode crawl (a solution root, or a project with no parsed restore) always adds the global packages folder NuGet would restore into as search root 3 (crawl_roots, via global_packages_folder → nuget_home()). The tests built their packages in a temp shared-cache but left that folder pointing at the process env and home, which belong to the host and to whichever serial test is running at the time.

Fix (test-only)

  • New `crawl_local(cwd)` test helper. For the duration of the crawl it sets `NUGET_PACKAGES`, under `crate::test_env::EnvGuard`, to a tempdir holding exactly one package, `global.only@1.0.0`. All six project-mode crawl tests use it and are now `#[serial]`. The two unscoped-walk tests must list `global.only`, which shows root 3 is walked. The four scoped tests must leave it out, which shows scoping excludes the global folder; before this PR they passed only because `seen` dedups by purl. Mutation check: with root 3 dropped from `crawl_roots`, 3 tests fail.
  • `test_nuget_home_env_var` and `test_nuget_home_empty_env_var_falls_back_to_default` move from bare `set_var`/`remove_var` onto `EnvGuard`, so a failed assertion can't leak `NUGET_PACKAGES` into later tests.

The crawler itself doesn't change. The unscoped walk the tests pin (#427 / #1183) stays the same.

Not fixed here, left as follow-ups from `/code-review high`:

  • `NugetEnv` still hand-rolls a restore-on-drop that duplicates `EnvGuard`.
  • `tests/crawler_nuget_e2e.rs` still saves and restores `NUGET_PACKAGES` by hand around an `unwrap()`.
  • `crawl_local` doesn't isolate the user-level NuGet.Config. No current walk test depends on it.

Proof

setup main this PR
host ~/.nuget/packages holds zz.hostpkg/9.9.9 and newtonsoft.json/13.0.3 solution_roots_… FAILED every run, an_uncovered_… intermittent 100/100 module runs green (host also holding `serilog/3.1.1`)
no ~/.nuget (CI-like) 150/150 green 150/150 + 50/50 green

So in CI this is latent: no failure reproduced in 150 CI-like runs. The fix makes the tests hermetic for dev machines and removes the cross-test env race. cargo clippy -p socket-patch-core --all-features -D warnings (CI's command) is clean, and rustfmt shows no diff on the touched file. With --all-targets, my local clippy reports 39 lints that main has too; none are in this file.

Tests moved or removed

None. Every test still runs where it ran before (test jobs, PR and merge queue).

https://claude.ai/code/session_011f9aeRytjzpRh8AHn2i5Qh


Generated by Claude Code

Two project-mode crawl tests that check an unscoped walk
(solution_roots_and_unrestored_projects_keep_the_walk and
an_uncovered_sub_project_keeps_the_walk) also crawl search root 3,
the global packages folder from NUGET_PACKAGES / DOTNET_CLI_HOME /
~/.nuget/packages. They failed on any machine with a package in
~/.nuget/packages (#2023). They weren't #[serial] either, so they
raced the serial tests that point NUGET_PACKAGES and HOME at their
own warm folders: with a host package planted, the walk test passed
or failed depending on which serial test ran alongside it.

Run both crawls through a crawl_local helper. It points
NUGET_PACKAGES at an empty tempdir under EnvGuard, and both tests are
now #[serial]. Also move the two nuget_home env tests from bare
set_var/remove_var onto EnvGuard, so a failed assertion can't leak the
variable into later tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeRytjzpRh8AHn2i5Qh
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Nov 8, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Review follow-up. An empty global packages folder made the two walk
tests stop checking that search root 3 is walked. crawl_local now
seeds it with one package, global.only@1.0.0: unscoped walks must
list it, and scoped crawls must leave it out. Dropping root 3 from
crawl_roots now fails three tests.

The four scoped crawl tests now also go through crawl_local and are
#[serial]. They were still reading the host or a parallel test's
global folder, and passed only because `seen` dedups by purl.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeRytjzpRh8AHn2i5Qh
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit d089d69. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 8, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Labeled Ready for review by the burn-down agent.

  • Head: d089d69f
  • CI: 50/50 check runs green (31 success, 19 skipped by path filters); mergeable, no CHANGELOG.md change.
  • Bugbot: reviewed d089d69f, no findings; no unresolved review threads.
  • Reviewer note: test-only change to crates/socket-patch-core/src/crawlers/nuget_crawler.rs (hermetic NUGET_PACKAGES + #[serial] for project-mode crawl tests). The PR body lists three follow-ups left out of scope.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. A test-only fix for #2023. The project-mode NuGet crawl tests now point NUGET_PACKAGES (search root 3, the global packages folder) at a tempdir holding one known package, global.only@1.0.0, and run #[serial]. So they no longer read the developer's ~/.nuget/packages or race other tests that change NUGET_PACKAGES/HOME. The production crawler is unchanged.

Risk: low. Only the #[cfg(test)] module of one file changes. #[serial] on six more tests adds well under a second to the run.

Look here

  • crawl_local helper: seeds the fake global folder and sets the env through EnvGuard
  • global_packages_folder: NUGET_PACKAGES takes precedence over NuGet.Config, so a user-level config can't get around the helper
  • unscoped walk tests: they now expect GLOBAL_ONLY, which proves root 3 is walked
  • scoped tests and project_local_packages_folder_is_still_walked: they must leave GLOBAL_ONLY out, which proves scoping excludes root 3
  • nuget_home env tests: moved onto EnvGuard, so a failed assert can't leak the var

Verified

  • I planted a host package at ~/.nuget/packages/zz.hostpkg/9.9.9. With it, cargo test -p socket-patch-core --lib nuget_crawler on this head passed 56/56 three times out of three. On main's version of the file, the same run fails both solution_roots_and_unrestored_projects_keep_the_walk and an_uncovered_sub_project_keeps_the_walk, so the tests do catch the bug.
  • CI: ci-ok and clippy are green on d089d69; the burn-down counted 50/50 check runs. Bugbot found nothing on d089d69, and no review threads are unresolved.
  • Mergeable. CHANGELOG.md is untouched. There's no scope creep, and the three follow-ups listed in the body are reasonable to leave out.

Changes I made: none.

Open questions: none.

Auto-merge is armed, so approving sends this PR straight to the merge queue.


Generated by Claude Code

@mikolalysenko
Mikola Lysenko (mikolalysenko) added this pull request to the merge queue Nov 9, 2026
Merged via the queue into main with commit 1505e0d Nov 9, 2026
50 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the ci-janitor/nuget-crawler-hermetic branch November 9, 2026 01:05
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Nov 9, 2026
#2031 landed on main at 01:23 UTC, after the f5b0374 cut-off these
notes were written against, and changes user-visible Go behavior: it
completes the go.work handling #2011 introduced (bypassed replaces stay
refs; an unread root go.work no longer withholds the root go.mod) and
stops the in-memory hosted engine from probing the process cwd for Go
workspaces. The other PRs merged since f5b0374 (#2002, #2034, #2035,
#2037) change no shipped behavior and need no note.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeRVgqkQL2WXrHEqeDJ
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Nov 9, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NuGet crawler test crawls the host's ~/.nuget/packages and fails on dev machines

3 participants