Skip to content

Rebuild gem native extensions from a relative crawl root - #2048

Merged
Mikola Lysenko (mikolalysenko) merged 2 commits into
mainfrom
fix/gem-native-relative-gem-home
Nov 10, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 2 commits into
mainfrom
fix/gem-native-relative-gem-home

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

5.1.0 release blocker found while smoke-testing the headline feature (agent-mode gem native-extension rebuild, #1996/#1998) on a real nokogiri.

Agent-mode apply rebuilds a patched gem's C extension by running Ruby from the project dir, and RubyGems runs extconf.rb from the gem's ext/ dir. The copy's paths came straight from the crawl root, so they were relative:

  • Default --cwd . with Bundler path vendor/bundle (the common CI setup): GEM_PATH=./vendor/bundle/ruby/3.4.0 names nothing from the ext dir. nokogiri's extconf.rb then can't activate mini_portile2 from the same home: Could not find 'mini_portile2' (~> 2.8.2). Apply reports native_extension_build_failed and puts the original back, so the patch is never applied.
  • Relative --cwd proj: Ruby runs from proj and can't load proj/vendor/.../nokogiri-1.19.7.gemspec, so the patch is refused with native_extensions_unsupported.

GemCopy::of now makes the copy's path absolute (relpath::absolute_lexically: std::path::absolute with .. dropped lexically, as Ruby's File.expand_path does), so the spec path, GEM_PATH, gem dir and extension dirs are absolute for both the probe and the build. The build marker still records extension dirs relative to the home, so marker matching is unchanged.

Test plan

  • New capstone native_patch_builds_from_a_relative_crawl_root: the patch changes extconf.rb to activate a helper gem installed in the copy's home (as nokogiri does with mini_portile2), then applies with --cwd proj and with the default root. It fails on main and passes with the fix.
  • cargo test -p socket-patch-cli --test e2e_gem_native_ext_build -- --include-ignored: 8/8 pass locally (Ruby 3.4.10, macOS arm64).
  • cargo test -p socket-patch-core --lib gem_ext: 18 pass. cargo fmt --check and the CI clippy invocation are clean.
  • Manual check with nokogiri 1.19.7 (ruby platform, packaged libxml2): a hand-written ext/nokogiri/nokogiri.c patch rebuilds in about 28s, Ruby loads the patched constant, apply --check is in sync, and rollback restores the original build.

/code-review high

I fixed: the Windows rooted-path break in the unit tests, .. normalization to match Ruby's File.expand_path, the fallback when making the path absolute errors, the misplaced doc comment, the helper gem being built inside the crawled tree, the stale ext_dir_label doc, and the missing apply --check/rollback coverage on the default root. I didn't fix:

  • Make RubyEnv.cwd / the crawl root absolute at the top: the fix belongs at the copy's paths, which are what the build runs on. ruby_candidates compares canonical paths first, so its relative fallback only applies when canonicalize fails. Changing the CLI-wide --cwd spelling would change reported paths in every ecosystem, which is out of scope for a release blocker.
  • Absolute home in two refusal/restore details: these are error details for a host-local install, and an absolute path is clearer there. package_path keeps the user's spelling.
  • Generalize write_manifest for the two-file record: this test is the only multi-file case, and the inline closure keeps the existing helper unchanged.

🤖 Generated with Claude Code


Note

Medium Risk
Touches core gem native rebuild path resolution; incorrect absolutization could break builds or refuse valid installs, but scope is localized to GemCopy::of and is covered by new unit and e2e tests.

Overview
Fixes gem native-extension rebuilds when the crawl root is relative (--cwd ., Bundler-style vendor/bundle, or --cwd proj from a parent). GemCopy::of now resolves non-rooted package paths with absolute_lexically (dropping .. like Ruby's File.expand_path) so spec path, GEM_PATH, and gem dirs stay absolute for Ruby runs from the project dir and extconf.rb from ext/, allowing helper gems in the same home (e.g. nokogiri-style mini_portile2 loads).

Adds an ignored capstone native_patch_builds_from_a_relative_crawl_root (helper gem + patched extconf.rb) covering --cwd proj and default in-project apply/rollback/check, plus unit test gem_copy_of_a_relative_path_is_absolute. Marker matching doc for ext_dir_label is narrowed to symlink spelling, not relative roots.

Reviewed by Cursor Bugbot for commit 5b1417c. Configure here.


Generated by Claude Code

Agent-mode apply rebuilds a patched gem's C extension by running Ruby
from the project dir, and RubyGems runs extconf.rb from the gem's ext
dir. The copy's paths came from the crawl root as given, so with the
default `--cwd .` (Bundler `path vendor/bundle`) GEM_PATH was a relative
`./vendor/bundle/ruby/<abi>`. From the ext dir that names nothing, so an
extconf.rb that activates a gem from the copy's own home failed.
nokogiri's (`mini_portile2`) fails this way, and apply reported
native_extension_build_failed and put the original back. A relative
`--cwd proj` failed one step earlier: Ruby, run from `proj`, could not
load `proj/vendor/.../x.gemspec`, so the patch was refused as
native_extensions_unsupported.

GemCopy::of now makes the copy's path absolute, so the spec path,
GEM_PATH, gem dir and extension dirs are absolute for both the probe and
the build. The marker still records extension dirs relative to the home.

Found by smoke-testing 5.1.0. Before this fix, a hand-written patch to
ext/nokogiri/nokogiri.c on nokogiri 1.19.7 (ruby platform, packaged
libxml2) failed to rebuild under plain `socket-patch apply`. With it,
the extension rebuilds in about 28s, Ruby loads the patched constant,
`apply --check` is in sync, and rollback restores the original build.

The new capstone native_patch_builds_from_a_relative_crawl_root patches
extconf.rb to activate a helper gem installed in the same home, then
applies with `--cwd proj` and with the default root. It fails without
this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Keep a rooted path as is, so Windows doesn't put a drive on `/h` and
  break the gem_ext unit tests. Make only a relative root absolute, with
  relpath::absolute_lexically, which drops `..` the way Ruby's
  File.expand_path does and falls back to the given path on error
  instead of reading the copy as "not a RubyGems install".
- Unit test: a relative `..` path becomes absolute and normalized.
- Capstone: build the helper gem outside the crawled vendor/bundle tree,
  and also run `apply --check` and `rollback` on the default root.
- Move walk_files' doc comment back onto it, and drop the now-stale
  "relative crawl root" reason from ext_dir_label's doc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review November 10, 2026 00:45
@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 10, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Labeled Ready for review by the burn-down agent.

  • Head: 5b1417c31c2999121bc6d2aa38eb1ddee6f17a2c
  • CI: every check on this head passed or was skipped by the dispatcher (ci-ok, clippy, coverage, all e2e jobs including e2e_gem_native_ext_build green). No main-wide failures.
  • Bugbot: reviewed this head, no findings; no open review threads.
  • Mergeable, no conflicts, doesn't touch CHANGELOG.md.
  • For reviewers: the change is GemCopy::of in crates/socket-patch-core/src/patch/gem_ext.rs making the copy's paths absolute. chore(release): complete 5.1.0 notes for every PR since v5.0.1 #2047 says this is a 5.1.0 release blocker that should land before Release is dispatched.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. Agent-mode apply rebuilds a patched gem's C extension by running Ruby from the project dir, and extconf.rb from the gem's ext/ dir. When the crawl root was relative (the default . with Bundler vendor/bundle, or --cwd proj), the spec path and GEM_PATH were relative too, so they pointed nowhere from those dirs and nokogiri-style builds failed. GemCopy::of now makes the copy's path absolute (and drops .. the way Ruby's File.expand_path does), so every path used for the probe and the build is absolute. The PR description calls this a 5.1.0 release blocker.

Risk: low. One function changes, in agent-mode gem native rebuilds only. Rooted paths are kept as they are. The build marker still records extension dirs relative to the home, so existing markers still match.

Look here

  • gem_ext.rs:127-140: the fix in GemCopy::of.
  • gem_ext.rs:506-522: ruby_candidates checks whether the Ruby prefix is inside the project. The home is now absolute while project may be relative. Canonical paths are compared first, so the raw-path fallback matters only when canonicalize fails, and then prefix/bin/ruby doesn't exist either.
  • gem_ext.rs:1651: new unit test (relative path with .. becomes absolute).
  • e2e_gem_native_ext_build.rs:360: new capstone. The patched extconf.rb loads a helper gem from the same home. It runs apply/rollback with --cwd proj and with the default root.

Verified

  • Read the full diff (2 files, +144/−1) and checked it against the description. CHANGELOG.md is untouched.
  • cargo test -p socket-patch-core --lib gem_ext: 19/19 pass.
  • Ran the new capstone locally with real Ruby 3.3 and cc. It passes on this head. With gem_ext.rs reverted to main it fails with native_extensions_unsupported … cannot load proj/vendor/bundle/…/sp_native-1.0.0.gemspec, so the test does exercise the fix.
  • CI: 32/32 checks that ran passed (the other 50 were skipped), including ci-ok and clippy. The CI gem leg (Ruby 3.4) log shows native_patch_builds_from_a_relative_crawl_root ... ok. Cursor Bugbot: success.
  • No unresolved review threads.

Changes I made. No code changes. I fixed one sentence in the PR description: it said std::path::absolute, but the code uses relpath::absolute_lexically.

Open questions. None.

Auto-merge (squash) is armed, so approving sends this straight to the merge queue.


Generated by Claude Code

Merged via the queue into main with commit 17ec1e7 Nov 10, 2026
82 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the fix/gem-native-relative-gem-home branch November 10, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants