From a2b1f231620ab7b5b0622f7ba924adf27ad654a7 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Fri, 20 Nov 2026 12:42:52 -0500 Subject: [PATCH 1/3] Test hosted Cargo mode on a rules_rust hub that names no cargo config The real-Bazel Cargo flows run on a hub that names the project's cargo config (`cargo_config = "//:.cargo/config.toml"`). The hub of the rules_rust README names none: crate.from_cargo( name = "crates", cargo_lockfile = "//:Cargo.lock", manifests = ["//:Cargo.toml"], ) cargo-bazel links what stands beside each manifest such a hub names into a directory of its own and writes a root manifest of its own there. It then removes every cargo config it finds there and puts back only the file the hub names (`symlink_roots`, `write_root_manifest` and `setup_cargo_config` in crate_universe/src/splicing/splicer.rs, rules_rust 0.56.0 and 0.75.0). Hosted mode names a registry of its own on the dependency in Cargo.toml and defines it in `.cargo/config.toml`, so on this hub the cargo that cargo-bazel runs has never heard of that registry. `cargo::hosted_from_cargo_without_config` holds what the release does there: the build passes on the upstream crate; `scan --mode hosted --yes` exits 0, reports the crate applied and rewrites `.cargo/config.toml`, `Cargo.lock` and `Cargo.toml`; and the next `bazel build //:consumer` fails in the hub with cargo's registry index was not found in any configuration: `socket-patch-` It passes on Bazel 9.3.2 (rules_rust 0.75.0) and 7.7.2 (rules_rust 0.56.0) with the product code as it is. The next commit changes the product and these expectations. The flow is the fixture of the other two with one per-test edit: the `cargo_config` line is taken out of the era's MODULE.bazel, and the checkout has no cargo config. The hermetic test of the fixture holds that every era's hub is the README's but for that line. The crate under test still comes from the loopback stand-in and from no other host. The README says how a hub without a config reaches a private registry: with `CARGO_BAZEL_ISOLATED=false` it runs its cargo with the cargo home of the machine, and the test's cargo home names the mirror in its `config.toml`. cargo-bazel itself reads no cargo config the hub does not name. For a crate the lock takes from crates.io it goes by what the cargo home keeps of that index: `config.json` for the download URL, and with rules_rust 0.56.0 the crate's cached file of the index as well. A cargo behind a mirror fetches neither, so the test writes both into its cargo home, as a home holds them whose cargo fetched the crate from crates.io itself. `cargo_env` takes the isolation switch as an argument for that; the two existing flows pass `true`, as before. bazel-cargo.yml: the header names the third flow. A leg still holds two workspace copies at a time (`--test-threads=2`). Co-Authored-By: Claude Fable 5.2 --- .github/workflows/bazel-cargo.yml | 19 ++- .../tests/e2e_bazel_build/cargo.rs | 150 ++++++++++++++++-- 2 files changed, 152 insertions(+), 17 deletions(-) diff --git a/.github/workflows/bazel-cargo.yml b/.github/workflows/bazel-cargo.yml index e637162ff..931c7dbac 100644 --- a/.github/workflows/bazel-cargo.yml +++ b/.github/workflows/bazel-cargo.yml @@ -3,7 +3,8 @@ name: Bazel patch compatibility (Cargo) # The Cargo flows of the real-Bazel tests (`cargo::` in `e2e_bazel_build`): # hosted and vendored mode for a crate that rules_rust's crate_universe # resolves from a Cargo workspace (`crate.from_cargo`, with the project's -# cargo config and no cargo-bazel lock), on each Bazel release below. +# cargo config and no cargo-bazel lock), and hosted mode on the hub that +# names no cargo config, on each Bazel release below. # bazel-compatibility.yml holds the legs and what every ecosystem shares; # this file says when the Cargo legs run and on which releases. # @@ -22,13 +23,15 @@ name: Bazel patch compatibility (Cargo) # No leg asks crates.io about the crate under test. cargo-bazel's cargo # reads no downloader config, so the test's workspace names a mirror of # crates.io in its `.cargo/config.toml`, the loopback stand-in, and the -# crate has a name crates.io cannot hold. A crate file under test passes -# through the repository cache as well: Bazel's downloader fetches it and -# keeps it under its sha256. So a test builds the upstream crate file and -# each patched one anew, with an id of the run in the library, and writes -# the upstream checksum into the Cargo.lock of its workspace copy. No cache -# holds a crate file of the run, and the test counts the requests for the -# upstream one. +# crate has a name crates.io cannot hold. The flow of the hub without a +# cargo config names the mirror in the cargo home of the test instead, which +# that hub's cargo is run with (`CARGO_BAZEL_ISOLATED=false`). A crate file +# under test passes through the repository cache as well: Bazel's +# downloader fetches it and keeps it under its sha256. So a test builds the +# upstream crate file and each patched one anew, with an id of the run in +# the library, and writes the upstream checksum into the Cargo.lock of its +# workspace copy. No cache holds a crate file of the run, and the test +# counts the requests for the upstream one. # # Not required for merge: no `merge_group` trigger, and no job here is a # `needs` of ci.yml's `ci-ok`. diff --git a/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs b/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs index 7c8c570e9..c2b0292e9 100644 --- a/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs +++ b/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs @@ -114,6 +114,18 @@ //! (static.rust-lang.org), and the crates it builds its own tools from //! (static.crates.io). //! +//! # The hub without a cargo config +//! +//! A third flow ([`hosted_from_cargo_without_config`]) runs hosted mode on +//! the hub as the rules_rust README has it, which names no `cargo_config` +//! ([`open_readme_hub`]). cargo-bazel then hands no cargo config of the +//! checkout to its cargo, so the registry that hosted mode defines in +//! `.cargo/config.toml` and names in `Cargo.toml` is one that cargo has +//! never heard of. The flow holds what the release does there: the scan +//! exits 0 and reports the crate as hosted, and the next `bazel build` +//! fails in the hub with cargo's `registry index was not found in any +//! configuration`. +//! //! # Known gaps //! //! Two assertions of the vendored flow hold what the release does with @@ -193,8 +205,20 @@ const CONFIG: &str = ".cargo/config.toml"; const LOCK: &str = "Cargo.lock"; const MANIFEST: &str = "Cargo.toml"; const BAZEL_LOCK: &str = "MODULE.bazel.lock"; +/// The file that declares the hub, and the line of it that names the cargo +/// config. Every era's hub has that line; without it the hub is the one of +/// the rules_rust README. +const MODULE: &str = "MODULE.bazel"; +const HUB_CONFIG: &str = " cargo_config = \"//:.cargo/config.toml\",\n"; /// How a `Cargo.lock` names crates.io, with or without a mirror. const CRATES_IO: &str = "registry+https://github.com/rust-lang/crates.io-index"; +/// Where a cargo home keeps what cargo has read of the sparse index of +/// crates.io: the directory cargo names so before 1.85 and the one from 1.85 +/// on (the cargo of rules_rust 0.56.0 is 1.83.0, that of 0.75.0 is 1.99.1). +const CRATES_IO_INDEXES: [&str; 2] = [ + "registry/index/index.crates.io-6f17d22bba15001f", + "registry/index/index.crates.io-1949cf8c6b5b557f", +]; /// What the fixture's lock holds as the checksum of the crate: the sum of /// no file. const PLACEHOLDER: &str = "0000000000000000000000000000000000000000000000000000000000000000"; @@ -442,10 +466,11 @@ fn grant(standin: &StandIn, upstream: &Package, patch: &Patched, served: &[u8]) /// `CARGO_HOME` is `home`, a directory of the test: under Bzlmod rules_rust /// 0.56.0 runs its cargo with the cargo home of the environment, and a /// flow says which one that is. rules_rust 0.75.0 gives its cargo a home -/// inside the output base, which `CARGO_BAZEL_ISOLATED` states. -fn cargo_env(home: &Path) -> Vec<(String, String)> { - let home = home.to_string_lossy(); - let pins = [("CARGO_HOME", &*home), ("CARGO_BAZEL_ISOLATED", "true")]; +/// inside the output base when `isolated`, which `CARGO_BAZEL_ISOLATED` +/// states, and runs it with `home` as well when not. +fn cargo_env(home: &Path, isolated: bool) -> Vec<(String, String)> { + let (home, isolated) = (home.to_string_lossy(), isolated.to_string()); + let pins = [("CARGO_HOME", &*home), ("CARGO_BAZEL_ISOLATED", &*isolated)]; let owned = pins.map(|(name, value)| (name.to_string(), value.to_string())); owned.to_vec() } @@ -462,7 +487,7 @@ fn cargo_home() -> tempfile::TempDir { fn open(test: &str) -> Option<(tempfile::TempDir, Cell, StandIn, Package)> { let mut cell = Cell::open(test, &FIXTURE)?; let home = cargo_home(); - cell.repo_env = cargo_env(home.path()); + cell.repo_env = cargo_env(home.path(), true); let upstream = Package::upstream(); let standin = registry(&upstream); cell.write(LOCK, lock_pinning(&upstream)); @@ -470,6 +495,50 @@ fn open(test: &str) -> Option<(tempfile::TempDir, Cell, StandIn, Package)> { Some((home, cell, standin, upstream)) } +/// [`open`] for the hub of the rules_rust README: the `MODULE.bazel` of the +/// era without the line that names a cargo config, in a checkout that has +/// none. +/// +/// The crate still comes from the stand-in, the way the README says a +/// private registry is reached: `CARGO_BAZEL_ISOLATED=false` has the hub run +/// its cargo with the cargo home of the machine, here the one of the test, +/// and that home's `config.toml` names the mirror. cargo-bazel itself reads +/// no cargo config the hub does not name, so for a crate the lock takes +/// from crates.io it goes by what the cargo home keeps of that index: the +/// `config.json`, for the download URL, and with rules_rust 0.56.0 the +/// crate's file of the index as well. A cargo behind a mirror fetches +/// neither. The home holds both, as a home does whose cargo has fetched the +/// crate from crates.io itself, with the stand-in as the download host. +fn open_readme_hub(test: &str) -> Option<(tempfile::TempDir, Cell, StandIn, Package)> { + let mut cell = Cell::open(test, &FIXTURE)?; + let home = cargo_home(); + cell.repo_env = cargo_env(home.path(), false); + let upstream = Package::upstream(); + let standin = registry(&upstream); + let origin = standin.origin(); + cell.write(LOCK, lock_pinning(&upstream)); + let module = std::fs::read_to_string(cell.path(MODULE)).expect(MODULE); + cell.write(MODULE, module.replace(HUB_CONFIG, "")); + let keep = |rel: &str, bytes: &[u8]| { + let path = home.path().join(rel); + std::fs::create_dir_all(path.parent().expect("a file path")).expect("create the parent"); + std::fs::write(&path, bytes).unwrap_or_else(|e| panic!("write {}: {e}", path.display())); + }; + keep("config.toml", mirror_config(&origin).as_bytes()); + let index = json!({ "dl": format!("{origin}/{FILES}/crates"), "api": origin }).to_string(); + // The crate's file of the index as cargo caches it: the version of the + // cache format (3) and of the index schema (2), what the server said of + // the file's age, and per release its version and its line of the file. + let line = upstream.index_line(); + let cached = format!("etag: \"run\"\0{VERSION}\0{}\0", line.trim_end()); + let cached = [&[3][..], &2u32.to_le_bytes(), cached.as_bytes()].concat(); + for directory in CRATES_IO_INDEXES { + keep(&format!("{directory}/config.json"), index.as_bytes()); + keep(&format!("{directory}/.cache/{}", index_path()), &cached); + } + Some((home, cell, standin, upstream)) +} + /// `socket-patch ` in `cell`, with `standin` as the Socket API, as /// the patch server (a hosted pin is recognised by that origin) and as the /// index `rollback` reads the upstream checksum from. @@ -614,7 +683,7 @@ fn second_machine( Some(cell) }; let home = cargo_home(); - twin.repo_env = cargo_env(home.path()); + twin.repo_env = cargo_env(home.path(), true); let other = registry(upstream); twin.write(CONFIG, mirror_config(&other.origin())); grant(&other, upstream, second, served); @@ -806,7 +875,7 @@ fn hosted_from_cargo_with_config() { // file of a crate the lock pins while its home holds one. other.serve(1, &second.entry, tampered.index_line().as_bytes()); let third_home = cargo_home(); - twin.repo_env = cargo_env(third_home.path()); + twin.repo_env = cargo_env(third_home.path(), true); let asked = other.asked(&second.entry); refuses(&twin, "changed between lock files"); assert!( @@ -1033,14 +1102,72 @@ fn vendored_from_cargo_with_config() { ); } +/// The flow of the hub without a cargo config (module docs). +#[test] +#[ignore = "real Bazel; run with --ignored"] +fn hosted_from_cargo_without_config() { + let test = "cargo::hosted_from_cargo_without_config"; + let started = Instant::now(); + let Some((_home, cell, standin, upstream)) = open_readme_hub(test) else { + return; + }; + let release = cell.release().to_string(); + let origin = standin.origin(); + let patch = Patched::new(); + grant(&standin, &upstream, &patch, &patch.package.crate_file); + + // (a) The unpatched build, with the upstream crate from the stand-in. + let before = builds_upstream(&cell, &standin, &upstream); + + // (b) The one command of hosted mode. It reports the crate as hosted + // and rewrites the three files, as on a hub that names the config. The + // config is new: the checkout had none, and it holds the patch registry + // alone. + let scan = socket_patch(&cell, &standin, &["scan", "--mode", "hosted", "--yes"]); + let pinned = cell.pinned_by(&scan, PURL, &patch.uuid, &before, &[CONFIG, LOCK, MANIFEST]); + let (index, registry) = (format!("sparse+{origin}{}/", patch.index), patch.registry()); + let defined = format!("[registries.{registry}]\nindex = \"{index}\"\n"); + assert_eq!(text(&pinned, CONFIG), defined, "{test}: {CONFIG}"); + let named = format!("registry = \"{registry}\""); + assert!( + text(&pinned, MANIFEST).contains(&named), + "{test}: {MANIFEST}" + ); + + // (c) The same build fails: the hub hands its cargo no config, so the + // registry the manifest names is defined nowhere cargo looks. + let build = cell.bazel("build", &[CONSUMER]); + let undefined = format!("registry index was not found in any configuration: `{registry}`"); + assert!( + !build.ok() && build.stderr().contains(&undefined), + "{test}: the build after `scan --mode hosted` did not fail with {undefined:?}:\n{}", + build.dump() + ); + assert_eq!( + standin.asked(&patch.entry), + 0, + "{test}: cargo read the patch index" + ); + + println!( + "{test}: Bazel {release} built {:?} on the hub of the rules_rust README; `scan --mode \ + hosted` exited 0, reported the crate applied and rewrote {:?}; the next `bazel build` \ + failed with {undefined:?}; {:?}", + upstream.marker, + [CONFIG, LOCK, MANIFEST], + started.elapsed() + ); +} + /// Without a Bazel, what stays fixed now that no checksum is committed: the /// fixture's directories and the legs of its workflow are those of its /// table; the fixture's lock holds the one entry a test pins, with the /// placeholder, and pinning changes nothing else; two runs serve two /// upstream crate files; the fixture depends on the crate under the names /// written out above, and commits no cargo config; every era's hub names -/// the config a test writes and has no cargo-bazel lock; and crates.io -/// cannot hold the name. +/// the config a test writes, has no cargo-bazel lock, and is the hub of the +/// rules_rust README but for that one line; and crates.io cannot hold the +/// name. #[test] fn a_test_pins_the_one_crate_of_the_fixture_lock() { FIXTURE.check(); @@ -1068,6 +1195,11 @@ fn a_test_pins_the_one_crate_of_the_fixture_lock() { cargo_lockfile = \"//:{LOCK}\",\n manifests = [\"//:{MANIFEST}\"],\n)\n" ); assert!(module.contains(&hub), "{era}: no\n{hub}in\n{module}"); + // Without that one line it is the hub of the rules_rust README. + let readme = "crate.from_cargo(\n name = \"crates\",\n cargo_lockfile = \ + \"//:Cargo.lock\",\n manifests = [\"//:Cargo.toml\"],\n)\n"; + let without = module.replace(HUB_CONFIG, ""); + assert!(without.contains(readme), "{era}: no\n{readme}in\n{without}"); } assert!(NAME.len() > 64, "crates.io can hold {NAME}"); From e60911aebb4a1e806f184a80daa5a3cd8465b3ac Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Fri, 20 Nov 2026 12:43:01 -0500 Subject: [PATCH 2/3] Write no hosted Cargo pin that a rules_rust hub would fail on A crate_universe hub of rules_rust runs cargo over the Cargo.toml files and the Cargo.lock it names, in a directory of its own, with one cargo config: the file the hub names as `cargo_config`, and none when it names none (the previous commit has the source). Hosted mode names a registry of its own on the dependency in Cargo.toml and defines it in the project's `.cargo/config.toml`. On the hub of the rules_rust README, which names no `cargo_config`, `scan --mode hosted` exited 0 and reported the crate applied, and the next `bazel build` failed: registry index was not found in any configuration: `socket-patch-` A command that exits 0 must not leave a build broken. Hosted mode now writes no pin for a crate while such a hub is in the way. What changes In a Bazel workspace (`bazel::bazel_root`: a workspace file in the project's directory or above it, up to the root of its git checkout), `scan --mode hosted` and `get --mode hosted` read the workspace files for the calls that make a crate_universe hub: - the `from_cargo` tags of the `crate` extension of rules_rust in MODULE.bazel and the files it includes, under either file the ruleset defines the extension in (`//crate_universe:extensions.bzl`, and `extension.bzl`, which hands it on; rules_rust 0.56.0 and 0.75.0); - `crates_repository(...)` calls of the WORKSPACE file, when the module files make no such tag and the checkout names no Bazel 9.0.0 or later, which never reads that file. A hub concerns the project when a label of its `cargo_lockfile` or its `manifests` is the project's Cargo.lock, its Cargo.toml or the manifest of a workspace member. `@//` is the checkout under the root module's name, and in a WORKSPACE file under the name that `workspace(name = ...)` gives the main repository (`WorkspaceGlobals.workspace`, Bazel 7.7.2 and 8.8.2). The hub is in the way unless its `cargo_config` is one string, the label of the config the rewriter writes (`.cargo/config.toml`, or the project's legacy `.cargo/config`). For a crate the rewriter would otherwise pin, nothing is written. The crate is `skipped` (`redirect_unconfirmed`), status and exit code are those of every other crate hosted mode leaves out (0), and one warning per crate, `redirect_cargo_bazel_config_unread`, names the hub by file and line and the two ways on: @: the rules_rust hub at MODULE.bazel:10 reads the Cargo files of this project, and it names no `cargo_config`. [...] dependency skipped (nothing rewritten). Commit .cargo/config.toml (where the project has none, create it with a comment line in it: `rollback` deletes a config it leaves empty, and a hub that names a missing file fails), give the hub `cargo_config = "//:.cargo/config.toml"`, the label of that file as one string, and run this command again; or use `--mode vendored`, which commits the patched crate and needs no registry The file comes first in it. A hub that is given the attribute alone fails on the missing file (`Failed to splice workspace crates`; measured once on Bazel 9.3.2, held by no test), and a config that a scan created is deleted by the `rollback` that empties it; a comment line keeps it. Where the reason is a use of the extension that may hide a hub, the hub that is seen may name the config already, and there may be no other. The warning then names that line and no attribute: @: MODULE.bazel:17: `crate` is used other than to call a tag, so a rules_rust hub that socket-patch cannot read may take the Cargo files of this project. [...] Write that line as the call of a tag (`crate.from_cargo(...)`) or as `use_repo(...)` and run this command again: it then pins the crate, or names the hub that is in the way; or use `--mode vendored`, [...] The rewriter's own refusals answer first and keep their codes. A pin that is in place already stays what it was: applied, with nothing to write and no warning. A vendored crate that the scan would take over stays vendored and patched (`redirect_takeover_kept_vendored`). The model is `redirect_cargo_directory_source`: the same place in `rewrite_cargo_with`, the same exit code and the same shape of text. The check sits where the three plans of a crate are complete, so that it withholds only what would be written. The engine reads the workspace with each read of the candidate files, for a run with a cargo candidate on disk, and hands the reason to the rewriter under a synthetic key among those files, as it does for the hubs of rules_jvm_external. Where the reading is not certain A broken build weighs more than a patch that waits for one line of the user's, so wherever a hub, or a use of the extension through a name bound to it, is seen, a reading that is not certain withholds the pin, and the warning says what could not be read: - a `cargo_lockfile` or `manifests` that is not written out as a string or a list of strings (a constant, a concatenation, an attribute left to `**`): the hub may read the project's files; - a `cargo_config` that is not written out as one string is not taken for the right file, a list of one label included; - a call that is not a statement of its own counts: it may run; - a use of the extension that may hide a call (`Tags::doubts`) counts; - a WORKSPACE call counts as above. Bazel 7 reads that file and Bazel 8 when it is told to, so a WORKSPACE file that a move to Bzlmod left behind can withhold a pin on Bazel 8 where the module files make no hub. The warning names the call. Where `workspace(name = ...)` is not written out as a string, any `@name//` label of that file is taken for the checkout. Where no hub is seen to concern the project, nothing is read further, said or written: in no Bazel workspace, in a workspace without such a hub, for a hub that names the config, for a hub whose labels are all written out and name other files, and for the tag of the same name of another ruleset. A doubt of the reader about the workspace files in general (`Workspace::doubts`) withholds nothing where no hub is seen: it would take hosted mode from every Cargo project beside a Bazel workspace that has nothing to do with rules_rust. Tests - bazel::cargo: eleven unit tests over workspace files: the README hub and its warning, each spelling of the right config, a wrong or unreadable config, which files concern which project, hubs whose files are not written out, every call that counts, tags of other extensions, a hidden call and its own way on, workspaces without a hub, the WORKSPACE file per Bazel release and under the names it gives the checkout, and the read from a checkout on disk. - The rewriter: one test. Each of the three files alone is enough to withhold (no lock, a config that lost the registry, a config that defines it already). The engine: one test, the lock and each manifest handed over. - e2e_bazel_build, without a Bazel: the binary in checkouts of the fixture's Cargo files. On the README hub `scan`, its dry run and `get` write nothing and a person reads why; a line that may hide a hub withholds beside a hub that names the config, and the warning names the line; with the file and the attribute the same scan pins, and `rollback` gives the checkout back. Where no hub reads the project (six shapes) the envelope and the three files are those of a checkout without a Bazel workspace, byte for byte. A vendored crate stays vendored. - e2e_bazel_build on real Bazel: `cargo::hosted_from_cargo_without_config` now holds that the scan withholds, that the same build still prints the unpatched marker and changes no file, and then both ways the warning names, on the hub of the README: `scan --mode vendored` and the build prints the patched marker, and `vendor --revert` gives the checkout back; with the committed config and the one attribute the hosted scan applies and the build prints the patched marker, from the patch host. Bazel 9.3.2 and 8.8.2 (rules_rust 0.75.0) and 7.7.2 (rules_rust 0.56.0), macOS arm64: 124 s, 58 s and 110 s for this flow, 217 s, 174 s and 188 s for the three Cargo flows of a leg, two at a time. Held against a binary of the parent commit, the flow fails where it asks that the scan applied nothing. One held gap: `rollback` deletes a cargo config it leaves empty, and a hub that names a missing file fails. The warning says to create such a file with a comment line in it, and the hermetic test holds the deletion so that the advice goes when that is fixed. Not covered In the first four a hub without the config fails as before. - A hub that another Bazel module declares; a `crates_repository` in a `.bzl` file that the WORKSPACE file loads, or called under another name. - Where no hub is seen: a tag called on `use_extension(...)` itself, with no name bound to it; a MODULE.bazel that cannot be read (a link that leads out of the Bazel root, a file past the reader's bounds); and what else the workspace files hide from every reading. - Flags and rc files: none is read, in the checkout or outside it. `--noenable_bzlmod` has Bazel read the WORKSPACE file in place of module files that make a hub. - A Bazel root above a project that is in no git checkout: only the project's own directory is looked at for a workspace file there. - The in-memory hosted engine, which is handed a project's files and reads no Bazel file. - `crate.from_specs` hubs and `crates_vendor` targets. - A hub with a cargo-bazel lock (`lockfile`) asks for a repin after any change of Cargo.lock, a pin included. That is unchanged. - A pin an earlier release wrote on such a hub stays and stays reported as applied; that build is failing already. docs/ecosystems.md has the section "Bazel workspaces: Cargo", and CLI_CONTRACT.md the code. scripts/loc-budget.json: the bazel group's cap follows the code, 2486 to 2618. Co-Authored-By: Claude Fable 5.2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 3 +- .../tests/e2e_bazel_build/cargo.rs | 420 ++++++++++- crates/socket-patch-core/src/bazel/cargo.rs | 690 ++++++++++++++++++ crates/socket-patch-core/src/bazel/mod.rs | 1 + crates/socket-patch-core/src/hosted/engine.rs | 99 ++- .../src/patch/redirect/mod.rs | 91 +++ docs/ecosystems.md | 124 +++- scripts/loc-budget.json | 2 +- 8 files changed, 1390 insertions(+), 40 deletions(-) create mode 100644 crates/socket-patch-core/src/bazel/cargo.rs diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index cdcae4bc9..8932f8c0d 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -410,7 +410,7 @@ Pinned by `tests/scan/scan_proxy_fallback_e2e.rs`, `tests/cli/api_client_errors_ **Vendored entries and the rest of the CLI.** Because nothing is in the manifest, vendored patches are invisible to `apply` (nothing to apply in place) but fully visible to `list` (listed from the ledger, labeled `Mode: vendored (recorded in .socket/vendor/state.json)` in human mode, exit 0 on a vendored-only project), `vex` (attested from the embedded records while a lockfile still wires the artifact — see "Manifest-less VEX"), `repair` (health-checked and rebuilt from the ledger), and `scan --prune` (lockfile-driven reconcile). They are exempt from standalone `vendor`'s manifest reconcile (`reconcile_dropped` never touches `detached` entries) and exit via `remove ` (which reverts them), `vendor --revert`, or `rollback`, whose vendored leg reverts every in-scope ledger entry (unscoped and identifier-scoped runs; path-scoped runs reach them only when an installed copy matches). -`scan --mode hosted` swaps the in-place apply for the registry-redirect pipeline: discover → resolve hosted-patch references (grant token + integrity + per-dep registry override) → rewrite ONLY the patched dependencies' lockfile / registry-config entries to point at the hosted packages. A dep counts as **redirected** only when its hosted-artifact URL (or per-dep registry index URL) actually landed in a project file — a granted reference whose rewriter found nothing to edit is neither counted nor attested. **No ledger (v5.0)**: hosted mode writes ONLY the lockfile / registry-config edits — `.socket/vendor/redirect-state.json` is never written (on success or failure), and a pre-v5 one on disk is ignored (never read for planning, never quarantined, left byte-identical). The lockfiles are the only record of a hosted patch: `list`, `vex`, `rollback`, `remove`, `vendor` and `repair` all discover the hosted pins from them (a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin), and commit-ready output is just the lockfile / config changes. Cargo and golang are confirmed only by their rewriter's own report (`confirmed_cargo_uuids` / `confirmed_golang_uuids`): a golang dep counts only when its go.mod `replace M V => patch.socket.dev/gopatch/ ` and both go.sum lines are in place (a re-run restores the zip line `go mod tidy` drops while no package of the module is loaded, so a later import still builds), never because the patch-server origin or leftover go.sum lines appear somewhere. Gradle is confirmed the same way (`confirmed_gradle_uuids`): only when the final files hold the owned script, the index row, the live apply line in every build's settings file and the suffixed version in every lock entry of the GA (see [Gradle builds](#gradle-builds-v50)). A golang module that go.mod does not require and go.sum has no zip (`h1:` of the module tree) line for at the patched version is outside the build graph and is refused with `redirect_golang_not_in_module_graph` (nothing written); with a `go.work` in effect the build graph is the workspace's (every `use` member's go.mod, go.sum and `go.work.sum`), so a member raising the module refuses the pin with `redirect_golang_version_mismatch`, and a `go.work` that overrides, ignores or rejects the replace refuses it with `redirect_golang_workspace_conflict` before anything is written. Run in the directory holding the `go.work`, a module the workspace shares (no root `go.mod`, or the root module is a `use` member and another member lists it or `go.work` already holds the Socket replace) is pinned in `go.work` and its two sum lines go into `go.work.sum`, the members' files untouched; a member's own `replace` of it, which the go.work pin would override, refuses it (`redirect_golang_workspace_conflict`); a `/go.mod`-only line doesn't count, since go writes one for every version MVS reads (#509). In a go < 1.17 graph, where `require` holds only lower bounds, the build selects the highest of the required version and go.sum's zip-line versions, and a patch for a lower version warns `redirect_golang_version_mismatch` (nothing written). Only the exact module `patch.socket.dev/gopatch/` is socket-owned; any other module path is refused with `redirect_golang_untrusted_module_path`. A vendored golang module is taken over like cargo and the npm family: its vendor wiring, committed copy and ledger entry are reverted first (`redirect_takeover_reverted_vendored`). A vendored PyPI package (requirements.txt, Poetry, Pipenv, uv, Hatch, PDM, pylock) is taken over the same way: its vendored wiring is restored to the recorded registry entry, its ledger entry and wheel are removed, and it is redirected in the same commit. The Python rewriters treat any non-registry source as user-authored, so without the revert they refused socket-patch's own vendored source and left the project vendored. A vendored Composer package is taken over the same way: its `composer.lock` entry is restored to the recorded pre-vendor entry, its ledger entry and committed copy are removed, and it is redirected in the same commit, so the lock matches a direct hosted scan and `rollback` can restore it from packagist. Without the revert the rewriter edited the vendored entry in place and kept its patch-uuid `dist.reference`, which `rollback` refused. A vendored Composer entry the takeover cannot revert (its vendored ledger entry is missing) is left vendored with `redirect_composer_vendored_entry`, never repointed in place. A takeover revert that leaves vendored wiring in place is refused with `redirect_vendored_revert_failed`. That covers a drift-skipped record (`vendor_lock_entry_drifted`) and a reverted file that still references the artifact (`vendor_revert_residual_reference`). The ledger entry and artifact are kept, and the package stays vendored and skipped. `--dry-run` predicts the same refusal from the same signals instead of previewing `redirect_would_revert_vendored`. The hosted requirements.txt rewriter only rewrites an existing pin in the root `requirements.txt`, so a vendored requirements.txt package whose wiring is a pin in a `-r` include or a `(transitive)` line vendored mode appended is kept vendored, wet and `--dry-run` alike, with `redirect_requirements_takeover_unreachable` (a top-level `warnings[]` entry, and the `errorCode` of the package's `skipped` event). Its wiring, ledger entry and wheel are kept, so it stays vendored and patched (exit 0). The uv and Poetry rewriters are covered the same way: a vendored uv package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (earlier releases pinned such an entry down to the patch's version, where vendored uv now refuses the patch with `pypi_uv_lock_package_missing`; the revert brings the lock's own version back, and hosted mode only pins the version the lock resolves) is kept vendored with `redirect_uv_takeover_version_unreachable`, and a vendored Poetry package on a Poetry 0.x lock (which hosted mode refuses outright) is kept vendored with `redirect_poetry_lock_unsupported`. **Staged takeover (v5.0)**: the takeover is atomic. Each purl's vendored revert is staged in memory (the run's group commit, the same journaled commit vendored mode uses), the hosted rewrite plans against the reverted project, and a staged purl the rewrite does not pin is retracted: every staged revert is undone, that purl keeps its vendored wiring, ledger entry and artifact byte-identical, and the rest are staged and rewritten again. A retracted purl is a `skipped` event whose `errorCode` is the cause: its existing skip reason (unavailable wheel metadata, …), the rewriter warning that names the package (`redirect_yarn_berry_missing_checksum`, `redirect_pypi_platform_wheel`, …), `redirect_requirements_takeover_unreachable`, the rewrite's lock-level refusal (`redirect_yarn_berry_mixed_line_endings`, `redirect_bun_lock_unsupported`, a Gradle planner refusal, …) or `redirect_takeover_not_pinned`; that warning is reported in `warnings[]`, followed by `redirect_takeover_kept_vendored` naming the package. Exit 0: the package stays vendored and patched, never unpatched in both modes. The reverts, the hosted pins and the vendored ledger then reach the disk in one commit, and the reverted artifacts are deleted only after it: a refusal or write failure before the commit writes nothing, a failed commit puts back the files it replaced, and a commit interrupted after its journal was written is finished by the next command that takes the apply lock. `--dry-run` runs the same steps and drops the staged state instead of committing, so its `verified` events and warnings are the wet run's `applied` events and warnings. A hosted run with no takeover stages its writes the same way but commits them without the journal (it writes nothing under `.socket/`): a file that fails to be replaced puts back the ones already replaced, so a failed run leaves no lock half-redirected. Re-runs over already-rewritten output plan from the current lock text and are idempotent (exit 0, lock unchanged). **Lock (v5.0)**: the hosted engine acquires `<.socket>/apply.lock` before its first wet write (the staged takeover reverts) — not on `--dry-run`, and not when the run would write nothing (zero redirects, all skipped) — so previews and no-op runs never create `.socket/`; contention is `lock_held` and a lock-file I/O fault (a read-only project root, a file squatting on `.socket/`) is `lock_io` — both exit 1, refused BEFORE any project file is written, and rendered like every other lock holder: human `Error (): ` on stderr (+ the `--lock-timeout` hint for a live holder); JSON prints the envelope with `status: "error"` and `error: {code: "lock_held" | "lock_io", message}` (no `redirect` payload: nothing was rewritten). **Takeover symlink pre-check (v5.0)**: a vendored→hosted takeover whose recorded wiring file is a symlink is refused up front with `redirect_symlinked_file_unsupported` — wet and `--dry-run` alike, before any revert — so "nothing was written" holds. **Human mode (v5.0)**: hosted `scan` runs discovery and update detection like the other modes, then rewrites without a prompt (scan never prompts) and prints its result line with one row per package it switched (see "Human output conventions"); `--dry-run` previews through the engine, and a detail fetch that leaves nothing to redirect enters the engine as a no-op (`Redirected 0 packages; rewrote 0 files.`, no lock, no `.socket/`). The detail fetch prints the same progress counter and per-package `Warning: could not fetch details for …` lines as the agent arm. An EMPTY hosted discovery prints `No patches available for installed packages.` and exits 0 without entering the engine; a discovery whose every offer is paid-tier for an org without paid access prints `No downloadable patches: every patch found requires a paid Socket plan.`, and exits 0 without entering the engine (the closing lines count what a paid plan would fix and name the plan page) (parity with the agent/vendored arms). Under `--json` (v5.0, MAJOR) the outcome is recorded into the scan envelope: one event per selected patch, sorted by purl, each with `details.mode: "hosted"` — `applied` for a confirmed pin (`verified` under `--dry-run`), `skipped` with the skip reason as `errorCode` (and its detail as `reason`), or `skipped` / `redirect_unconfirmed` for a patch that was granted but no lockfile entry pinning it could be rewritten (the human output's `Not hosted : …` line; more than three such packages share one line that names the first three) — plus a `redirect: {mode: "hosted", rewrittenFiles}` payload (the files rewritten, or that would be). A `skipped` event does not change `status` or the exit code (the hosted exit policy is an open decision, #704). Rewriter warnings carry stable `redirect_*` codes (e.g. `redirect_npm_no_lockfile`, `redirect_gradle_manual_snippet`, `redirect_golang_unsupported`); new codes are additive (MINOR). v5.0 additive codes: `redirect_composer_no_lockfile` / `redirect_gem_no_gemfile` (composer / gem: neither manifest nor lock present — once per run, after the intake gates), `redirect_pypi_no_lockfile` (pypi: a granted patch reached the PyPI rewriters but the project has no root `requirements.txt`, `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock`, PEP 751 / PEP 723 lock or Hatch configuration — once per run, beside each patch's `redirect_unconfirmed` row; #638), `redirect_gem_bundle_gemfile_unsupported` (gem: `BUNDLE_GEMFILE` — `BUNDLE_GEMFILE:` in the bundler app config, which outranks the environment variable as in `Bundler::Settings`, else the environment variable — names a manifest other than the project's `Gemfile` / `gems.rb`, so no gem is redirected or attested; a value naming one of those two selects that pair even when the other spelling is present), `redirect_gem_bundle_lockfile_unsupported` (gem: bundler 4's custom lockfile — the `BUNDLE_LOCKFILE` environment variable, else `BUNDLE_LOCKFILE:` in the bundler app config, else in the global config — names a lock other than the loaded pair's own `Gemfile.lock` / `gems.locked`, so no gem is redirected or attested rather than pinning a lock bundler ignores), `redirect_gem_twin_manifest_ambiguous` (gem: a `Gemfile` + `gems.rb` twin under default discovery; bundler 1.x loads the `Gemfile` and bundler ≥ 2 loads `gems.rb`, and a lock's `BUNDLED WITH` records which bundler wrote it, not which one installs it, so neither pair is wired or attested — remove the unused spelling or set `BUNDLE_GEMFILE` to the one in use), `redirect_gem_mirror_overrides_source` (gem: Bundler's all-source, exact patch-source or patch-hostname mirror can route the gem's Socket `source:` to an unpatched upstream gem. Intake reads the app config (`BUNDLE_APP_CONFIG`, where a set-but-empty value selects `/config`, honoring `BUNDLE_IGNORE_CONFIG`) and all `BUNDLE_MIRROR__...` variables visible to the scan; app config overrides the environment per encoded key, then `mirror.all` takes precedence over exact source, which takes precedence over hostname. URI matching follows Bundler's whole-URI case folding, default-port/trailing-slash normalization and single slash key alias, not URL prefixes. An exact-source fallback-timeout key without a mirror URL shadows the hostname mirror and fetches that source directly; a configured URL is conservatively refused even if a timeout could bypass an unreachable mirror at install time. Like `redirect_gem_bundle_gemfile_unsupported`, the gate leaves the Gemfile pair byte-identical and confirms no gem redirect. On an embedded `scan --vex`, rediscovered older hosted gem pins may attest only from verified installed bytes: a missing tree is not excused by the lockfile, and `--vex-no-verify` omits those hosted gems with `mirror_overrides_source` rather than trusting their intercepted source. Agent/vendored evidence, unrelated ecosystems and standalone VEX behavior are unchanged. Details identify the setting form and its app/environment origin without printing mirror values or source URLs, which may contain credentials. Remove the applicable all/source/hostname setting (including any slash alias) from that origin and reuse its existing mirror URL under `mirror.https://rubygems.org` to clear the refusal; an environment setting must be unset in the scan/install environment. User-global Bundler config and mirrors set only in a later install environment are not inspected; keep those mirrors scoped to the upstream source too), `redirect_maven_no_pom` (no `pom.xml` and no Gradle build), `redirect_nuget_lock_unparseable` (a present-but-corrupt `packages.lock.json` — warned once, nothing mutated; an absent lock still proceeds; a leading UTF-8 BOM, which dotnet reads past, is not corrupt and is kept on rewrite), `redirect_nuget_lock_other_version` (the lock also resolves the patched id at another version in some target framework: the exact-id `packageSourceMapping` would route that framework to a feed that serves only the patched version, so the dep is skipped with nothing written; only lock entries at the patched version are ever re-pinned, and `resolved` is never rewritten), `redirect_nuget_mapping_set_aside` (v5.0 #462: another source's `packageSourceMapping` also named the patched id exactly — Visual Studio's mapping UI writes such lists — which ties with the Socket source so NuGet would take the package from whichever feed answers first; that pattern, or its whole `` when it was the only one, is commented out in a `` comment while the patch is wired, and `remove` / `rollback` put it back byte-exact), `redirect_nuget_mapping_conflict` (such a pattern sits in markup that cannot go in a comment; the dep is skipped, nothing written). v5.0 #354: when the rewriter creates the `packageSourceMapping`, its `*` catch-all also names the sources NuGet merges in from the user config and every parent directory's config (honoring ``), since NuGet drops every source no pattern names; a fresh config only seeds `nuget.org` when those inherited configs have it (a parent that cleared it for a mirror keeps that choice); when an inherited config already has a `packageSourceMapping` (NuGet merges those too), no catch-all is written at all — only the Socket pattern — so a source the parent restricts is never widened; the exclusivity set-aside exempts only the source the run wires, so a `socket-patch-*` lookalike key or a stale uuid naming the id is set aside too, `redirect_cargo_lock_pkg_ambiguous` (several same-name+version `[[package]]` blocks and none carries the index `source` — transactional skip), `redirect_cargo_dep_overridden` (the crate does not resolve to crates.io because the user overrides it — its one `Cargo.lock` block carries a git / other-registry `source`, as a `[patch]` git or registry override in the root manifest or any cargo config leaves it, or has no `source` while the root `Cargo.toml` / project `.cargo/config*` holds a `[patch]` path entry for it; with no `Cargo.lock`, the same `[patch.crates-io]` / `[patch."https://github.com/rust-lang/crates.io-index"]` entry for the crate, by key or `package = "…"`, in the root `Cargo.toml` or the project `.cargo/config*` — transactional skip, nothing rewritten: a Socket pin would silently replace the user's fork and the then-unused `[patch]` entry breaks `cargo --locked`). Also additive: `redirect_cargo_directory_source` (#455: a cargo config replaces crates.io with an on-disk source through its `[source.crates-io] replace-with` chain — a directory source, as a `cargo vendor` project has, or a `cargo local-registry` — so the project builds every crates.io crate offline from it and a Socket registry pin would break `cargo build --frozen --offline`. The configs read are the ones cargo reads for a build in the scanned directory: the project's own `.cargo/config*`, every parent directory's (a monorepo root) and `$CARGO_HOME`'s. Every cargo dependency is skipped with nothing written, one warning each naming the crate and the config that set the replacement, and the detail points at agent mode, which patches the copy cargo builds in place. A replacement with a registry mirror is not an offline source and is still redirected). Also additive: `redirect_gem_version_not_locked` (gem: no `GEM` section of the lock lists the crawled `name (version)`, for example a version another project installed into the shared gem home; the gem is skipped with nothing written, so the user's declared constraint and the locked version are never overwritten), `redirect_gem_no_lockfile` (gem: the project has a `Gemfile` / `gems.rb` but no `Gemfile.lock` / `gems.locked`, so nothing records which version it resolves and the crawled version may be another project's copy in the shared gem home; the gem is skipped with nothing written, never pinned over the user's constraint or appended as a new dependency, and the detail asks for `bundle lock` (or `bundle install`) and a re-run). Also v5.0: a registry override of the wrong kind (or none at all) warns the arm's missing-override code for nuget/gem/golang. Refusals stay fail-closed with a diagnosis that names the actual cause: a yarn-berry lock entry resolving through a non-`npm:` protocol keeps `redirect_yarn_berry_unsupported_protocol` with the entry's ACTUAL protocol in the detail — except socket-patch's OWN vendored wiring (a `file:` range into `.socket/vendor/`), which gets the distinct `redirect_yarn_berry_vendored_entry` code whose detail names the retirement path (`remove ` per package, or `vendor --revert` which unwinds every vendored package, then re-run `scan --mode hosted`). Both leave the entry byte-identical; neither changes exit code or status. **yarn berry line endings (v5.0)**: yarn writes a NEW `yarn.lock` with the OS line ending (`os.EOL` — CRLF on Windows) and keeps an existing lock's majority ending on every later write, and a `core.autocrlf` checkout turns an LF lock CRLF on any OS — so a uniformly CRLF lock is rewritten in its own ending: every untouched byte (a leading BOM included) round-trips (and `rollback`'s upstream restore keeps the lock's own ending). A lock that MIXES CRLF and LF (or holds a bare CR) has no single ending to keep — yarn's own `--immutable` check rejects it too (YN0028) — so it is refused untouched with `redirect_yarn_berry_mixed_line_endings` (the detail names `yarn install`, which normalizes it). The root `package.json`, which the rewrite re-renders to add `resolutions`, gets the same gate: a mixed one is refused untouched with the same code — the decision vendored mode takes with `vendor_yarn_berry_mixed_line_endings`, from the same shared berry gate set. This replaces v4's `redirect_yarn_berry_crlf_unsupported`, which refused every CRLF lock and is no longer emitted. A vendored→hosted takeover of a yarn-berry entry is checked against these project gates (mixed `yarn.lock` / `package.json` line endings, unsupported `cacheKey`, a non-zero `.yarnrc.yml` `compressionLevel`) on the project as it is before any revert, because the revert re-renders `package.json` in its majority ending — wet and `--dry-run` alike. A refused purl keeps its vendored wiring, ledger entry and artifact byte-identical, is skipped with the gate's code (reported in `warnings[]`, followed by `redirect_takeover_kept_vendored`) and is never announced as `redirect_takeover_reverted_vendored`. +`scan --mode hosted` swaps the in-place apply for the registry-redirect pipeline: discover → resolve hosted-patch references (grant token + integrity + per-dep registry override) → rewrite ONLY the patched dependencies' lockfile / registry-config entries to point at the hosted packages. A dep counts as **redirected** only when its hosted-artifact URL (or per-dep registry index URL) actually landed in a project file — a granted reference whose rewriter found nothing to edit is neither counted nor attested. **No ledger (v5.0)**: hosted mode writes ONLY the lockfile / registry-config edits — `.socket/vendor/redirect-state.json` is never written (on success or failure), and a pre-v5 one on disk is ignored (never read for planning, never quarantined, left byte-identical). The lockfiles are the only record of a hosted patch: `list`, `vex`, `rollback`, `remove`, `vendor` and `repair` all discover the hosted pins from them (a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin), and commit-ready output is just the lockfile / config changes. Cargo and golang are confirmed only by their rewriter's own report (`confirmed_cargo_uuids` / `confirmed_golang_uuids`): a golang dep counts only when its go.mod `replace M V => patch.socket.dev/gopatch/ ` and both go.sum lines are in place (a re-run restores the zip line `go mod tidy` drops while no package of the module is loaded, so a later import still builds), never because the patch-server origin or leftover go.sum lines appear somewhere. Gradle is confirmed the same way (`confirmed_gradle_uuids`): only when the final files hold the owned script, the index row, the live apply line in every build's settings file and the suffixed version in every lock entry of the GA (see [Gradle builds](#gradle-builds-v50)). A golang module that go.mod does not require and go.sum has no zip (`h1:` of the module tree) line for at the patched version is outside the build graph and is refused with `redirect_golang_not_in_module_graph` (nothing written); with a `go.work` in effect the build graph is the workspace's (every `use` member's go.mod, go.sum and `go.work.sum`), so a member raising the module refuses the pin with `redirect_golang_version_mismatch`, and a `go.work` that overrides, ignores or rejects the replace refuses it with `redirect_golang_workspace_conflict` before anything is written. Run in the directory holding the `go.work`, a module the workspace shares (no root `go.mod`, or the root module is a `use` member and another member lists it or `go.work` already holds the Socket replace) is pinned in `go.work` and its two sum lines go into `go.work.sum`, the members' files untouched; a member's own `replace` of it, which the go.work pin would override, refuses it (`redirect_golang_workspace_conflict`); a `/go.mod`-only line doesn't count, since go writes one for every version MVS reads (#509). In a go < 1.17 graph, where `require` holds only lower bounds, the build selects the highest of the required version and go.sum's zip-line versions, and a patch for a lower version warns `redirect_golang_version_mismatch` (nothing written). Only the exact module `patch.socket.dev/gopatch/` is socket-owned; any other module path is refused with `redirect_golang_untrusted_module_path`. A vendored golang module is taken over like cargo and the npm family: its vendor wiring, committed copy and ledger entry are reverted first (`redirect_takeover_reverted_vendored`). A vendored PyPI package (requirements.txt, Poetry, Pipenv, uv, Hatch, PDM, pylock) is taken over the same way: its vendored wiring is restored to the recorded registry entry, its ledger entry and wheel are removed, and it is redirected in the same commit. The Python rewriters treat any non-registry source as user-authored, so without the revert they refused socket-patch's own vendored source and left the project vendored. A vendored Composer package is taken over the same way: its `composer.lock` entry is restored to the recorded pre-vendor entry, its ledger entry and committed copy are removed, and it is redirected in the same commit, so the lock matches a direct hosted scan and `rollback` can restore it from packagist. Without the revert the rewriter edited the vendored entry in place and kept its patch-uuid `dist.reference`, which `rollback` refused. A vendored Composer entry the takeover cannot revert (its vendored ledger entry is missing) is left vendored with `redirect_composer_vendored_entry`, never repointed in place. A takeover revert that leaves vendored wiring in place is refused with `redirect_vendored_revert_failed`. That covers a drift-skipped record (`vendor_lock_entry_drifted`) and a reverted file that still references the artifact (`vendor_revert_residual_reference`). The ledger entry and artifact are kept, and the package stays vendored and skipped. `--dry-run` predicts the same refusal from the same signals instead of previewing `redirect_would_revert_vendored`. The hosted requirements.txt rewriter only rewrites an existing pin in the root `requirements.txt`, so a vendored requirements.txt package whose wiring is a pin in a `-r` include or a `(transitive)` line vendored mode appended is kept vendored, wet and `--dry-run` alike, with `redirect_requirements_takeover_unreachable` (a top-level `warnings[]` entry, and the `errorCode` of the package's `skipped` event). Its wiring, ledger entry and wheel are kept, so it stays vendored and patched (exit 0). The uv and Poetry rewriters are covered the same way: a vendored uv package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (earlier releases pinned such an entry down to the patch's version, where vendored uv now refuses the patch with `pypi_uv_lock_package_missing`; the revert brings the lock's own version back, and hosted mode only pins the version the lock resolves) is kept vendored with `redirect_uv_takeover_version_unreachable`, and a vendored Poetry package on a Poetry 0.x lock (which hosted mode refuses outright) is kept vendored with `redirect_poetry_lock_unsupported`. **Staged takeover (v5.0)**: the takeover is atomic. Each purl's vendored revert is staged in memory (the run's group commit, the same journaled commit vendored mode uses), the hosted rewrite plans against the reverted project, and a staged purl the rewrite does not pin is retracted: every staged revert is undone, that purl keeps its vendored wiring, ledger entry and artifact byte-identical, and the rest are staged and rewritten again. A retracted purl is a `skipped` event whose `errorCode` is the cause: its existing skip reason (unavailable wheel metadata, …), the rewriter warning that names the package (`redirect_yarn_berry_missing_checksum`, `redirect_pypi_platform_wheel`, …), `redirect_requirements_takeover_unreachable`, the rewrite's lock-level refusal (`redirect_yarn_berry_mixed_line_endings`, `redirect_bun_lock_unsupported`, a Gradle planner refusal, …) or `redirect_takeover_not_pinned`; that warning is reported in `warnings[]`, followed by `redirect_takeover_kept_vendored` naming the package. Exit 0: the package stays vendored and patched, never unpatched in both modes. The reverts, the hosted pins and the vendored ledger then reach the disk in one commit, and the reverted artifacts are deleted only after it: a refusal or write failure before the commit writes nothing, a failed commit puts back the files it replaced, and a commit interrupted after its journal was written is finished by the next command that takes the apply lock. `--dry-run` runs the same steps and drops the staged state instead of committing, so its `verified` events and warnings are the wet run's `applied` events and warnings. A hosted run with no takeover stages its writes the same way but commits them without the journal (it writes nothing under `.socket/`): a file that fails to be replaced puts back the ones already replaced, so a failed run leaves no lock half-redirected. Re-runs over already-rewritten output plan from the current lock text and are idempotent (exit 0, lock unchanged). **Lock (v5.0)**: the hosted engine acquires `<.socket>/apply.lock` before its first wet write (the staged takeover reverts) — not on `--dry-run`, and not when the run would write nothing (zero redirects, all skipped) — so previews and no-op runs never create `.socket/`; contention is `lock_held` and a lock-file I/O fault (a read-only project root, a file squatting on `.socket/`) is `lock_io` — both exit 1, refused BEFORE any project file is written, and rendered like every other lock holder: human `Error (): ` on stderr (+ the `--lock-timeout` hint for a live holder); JSON prints the envelope with `status: "error"` and `error: {code: "lock_held" | "lock_io", message}` (no `redirect` payload: nothing was rewritten). **Takeover symlink pre-check (v5.0)**: a vendored→hosted takeover whose recorded wiring file is a symlink is refused up front with `redirect_symlinked_file_unsupported` — wet and `--dry-run` alike, before any revert — so "nothing was written" holds. **Human mode (v5.0)**: hosted `scan` runs discovery and update detection like the other modes, then rewrites without a prompt (scan never prompts) and prints its result line with one row per package it switched (see "Human output conventions"); `--dry-run` previews through the engine, and a detail fetch that leaves nothing to redirect enters the engine as a no-op (`Redirected 0 packages; rewrote 0 files.`, no lock, no `.socket/`). The detail fetch prints the same progress counter and per-package `Warning: could not fetch details for …` lines as the agent arm. An EMPTY hosted discovery prints `No patches available for installed packages.` and exits 0 without entering the engine; a discovery whose every offer is paid-tier for an org without paid access prints `No downloadable patches: every patch found requires a paid Socket plan.`, and exits 0 without entering the engine (the closing lines count what a paid plan would fix and name the plan page) (parity with the agent/vendored arms). Under `--json` (v5.0, MAJOR) the outcome is recorded into the scan envelope: one event per selected patch, sorted by purl, each with `details.mode: "hosted"` — `applied` for a confirmed pin (`verified` under `--dry-run`), `skipped` with the skip reason as `errorCode` (and its detail as `reason`), or `skipped` / `redirect_unconfirmed` for a patch that was granted but no lockfile entry pinning it could be rewritten (the human output's `Not hosted : …` line; more than three such packages share one line that names the first three) — plus a `redirect: {mode: "hosted", rewrittenFiles}` payload (the files rewritten, or that would be). A `skipped` event does not change `status` or the exit code (the hosted exit policy is an open decision, #704). Rewriter warnings carry stable `redirect_*` codes (e.g. `redirect_npm_no_lockfile`, `redirect_gradle_manual_snippet`, `redirect_golang_unsupported`); new codes are additive (MINOR). v5.0 additive codes: `redirect_composer_no_lockfile` / `redirect_gem_no_gemfile` (composer / gem: neither manifest nor lock present — once per run, after the intake gates), `redirect_pypi_no_lockfile` (pypi: a granted patch reached the PyPI rewriters but the project has no root `requirements.txt`, `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock`, PEP 751 / PEP 723 lock or Hatch configuration — once per run, beside each patch's `redirect_unconfirmed` row; #638), `redirect_gem_bundle_gemfile_unsupported` (gem: `BUNDLE_GEMFILE` — `BUNDLE_GEMFILE:` in the bundler app config, which outranks the environment variable as in `Bundler::Settings`, else the environment variable — names a manifest other than the project's `Gemfile` / `gems.rb`, so no gem is redirected or attested; a value naming one of those two selects that pair even when the other spelling is present), `redirect_gem_bundle_lockfile_unsupported` (gem: bundler 4's custom lockfile — the `BUNDLE_LOCKFILE` environment variable, else `BUNDLE_LOCKFILE:` in the bundler app config, else in the global config — names a lock other than the loaded pair's own `Gemfile.lock` / `gems.locked`, so no gem is redirected or attested rather than pinning a lock bundler ignores), `redirect_gem_twin_manifest_ambiguous` (gem: a `Gemfile` + `gems.rb` twin under default discovery; bundler 1.x loads the `Gemfile` and bundler ≥ 2 loads `gems.rb`, and a lock's `BUNDLED WITH` records which bundler wrote it, not which one installs it, so neither pair is wired or attested — remove the unused spelling or set `BUNDLE_GEMFILE` to the one in use), `redirect_gem_mirror_overrides_source` (gem: Bundler's all-source, exact patch-source or patch-hostname mirror can route the gem's Socket `source:` to an unpatched upstream gem. Intake reads the app config (`BUNDLE_APP_CONFIG`, where a set-but-empty value selects `/config`, honoring `BUNDLE_IGNORE_CONFIG`) and all `BUNDLE_MIRROR__...` variables visible to the scan; app config overrides the environment per encoded key, then `mirror.all` takes precedence over exact source, which takes precedence over hostname. URI matching follows Bundler's whole-URI case folding, default-port/trailing-slash normalization and single slash key alias, not URL prefixes. An exact-source fallback-timeout key without a mirror URL shadows the hostname mirror and fetches that source directly; a configured URL is conservatively refused even if a timeout could bypass an unreachable mirror at install time. Like `redirect_gem_bundle_gemfile_unsupported`, the gate leaves the Gemfile pair byte-identical and confirms no gem redirect. On an embedded `scan --vex`, rediscovered older hosted gem pins may attest only from verified installed bytes: a missing tree is not excused by the lockfile, and `--vex-no-verify` omits those hosted gems with `mirror_overrides_source` rather than trusting their intercepted source. Agent/vendored evidence, unrelated ecosystems and standalone VEX behavior are unchanged. Details identify the setting form and its app/environment origin without printing mirror values or source URLs, which may contain credentials. Remove the applicable all/source/hostname setting (including any slash alias) from that origin and reuse its existing mirror URL under `mirror.https://rubygems.org` to clear the refusal; an environment setting must be unset in the scan/install environment. User-global Bundler config and mirrors set only in a later install environment are not inspected; keep those mirrors scoped to the upstream source too), `redirect_maven_no_pom` (no `pom.xml` and no Gradle build), `redirect_nuget_lock_unparseable` (a present-but-corrupt `packages.lock.json` — warned once, nothing mutated; an absent lock still proceeds; a leading UTF-8 BOM, which dotnet reads past, is not corrupt and is kept on rewrite), `redirect_nuget_lock_other_version` (the lock also resolves the patched id at another version in some target framework: the exact-id `packageSourceMapping` would route that framework to a feed that serves only the patched version, so the dep is skipped with nothing written; only lock entries at the patched version are ever re-pinned, and `resolved` is never rewritten), `redirect_nuget_mapping_set_aside` (v5.0 #462: another source's `packageSourceMapping` also named the patched id exactly — Visual Studio's mapping UI writes such lists — which ties with the Socket source so NuGet would take the package from whichever feed answers first; that pattern, or its whole `` when it was the only one, is commented out in a `` comment while the patch is wired, and `remove` / `rollback` put it back byte-exact), `redirect_nuget_mapping_conflict` (such a pattern sits in markup that cannot go in a comment; the dep is skipped, nothing written). v5.0 #354: when the rewriter creates the `packageSourceMapping`, its `*` catch-all also names the sources NuGet merges in from the user config and every parent directory's config (honoring ``), since NuGet drops every source no pattern names; a fresh config only seeds `nuget.org` when those inherited configs have it (a parent that cleared it for a mirror keeps that choice); when an inherited config already has a `packageSourceMapping` (NuGet merges those too), no catch-all is written at all — only the Socket pattern — so a source the parent restricts is never widened; the exclusivity set-aside exempts only the source the run wires, so a `socket-patch-*` lookalike key or a stale uuid naming the id is set aside too, `redirect_cargo_lock_pkg_ambiguous` (several same-name+version `[[package]]` blocks and none carries the index `source` — transactional skip), `redirect_cargo_dep_overridden` (the crate does not resolve to crates.io because the user overrides it — its one `Cargo.lock` block carries a git / other-registry `source`, as a `[patch]` git or registry override in the root manifest or any cargo config leaves it, or has no `source` while the root `Cargo.toml` / project `.cargo/config*` holds a `[patch]` path entry for it; with no `Cargo.lock`, the same `[patch.crates-io]` / `[patch."https://github.com/rust-lang/crates.io-index"]` entry for the crate, by key or `package = "…"`, in the root `Cargo.toml` or the project `.cargo/config*` — transactional skip, nothing rewritten: a Socket pin would silently replace the user's fork and the then-unused `[patch]` entry breaks `cargo --locked`). Also additive: `redirect_cargo_directory_source` (#455: a cargo config replaces crates.io with an on-disk source through its `[source.crates-io] replace-with` chain — a directory source, as a `cargo vendor` project has, or a `cargo local-registry` — so the project builds every crates.io crate offline from it and a Socket registry pin would break `cargo build --frozen --offline`. The configs read are the ones cargo reads for a build in the scanned directory: the project's own `.cargo/config*`, every parent directory's (a monorepo root) and `$CARGO_HOME`'s. Every cargo dependency is skipped with nothing written, one warning each naming the crate and the config that set the replacement, and the detail points at agent mode, which patches the copy cargo builds in place. A replacement with a registry mirror is not an offline source and is still redirected). Also additive: `redirect_cargo_bazel_config_unread` (Cargo in a Bazel workspace, see the table below). Also additive: `redirect_gem_version_not_locked` (gem: no `GEM` section of the lock lists the crawled `name (version)`, for example a version another project installed into the shared gem home; the gem is skipped with nothing written, so the user's declared constraint and the locked version are never overwritten), `redirect_gem_no_lockfile` (gem: the project has a `Gemfile` / `gems.rb` but no `Gemfile.lock` / `gems.locked`, so nothing records which version it resolves and the crawled version may be another project's copy in the shared gem home; the gem is skipped with nothing written, never pinned over the user's constraint or appended as a new dependency, and the detail asks for `bundle lock` (or `bundle install`) and a re-run). Also v5.0: a registry override of the wrong kind (or none at all) warns the arm's missing-override code for nuget/gem/golang. Refusals stay fail-closed with a diagnosis that names the actual cause: a yarn-berry lock entry resolving through a non-`npm:` protocol keeps `redirect_yarn_berry_unsupported_protocol` with the entry's ACTUAL protocol in the detail — except socket-patch's OWN vendored wiring (a `file:` range into `.socket/vendor/`), which gets the distinct `redirect_yarn_berry_vendored_entry` code whose detail names the retirement path (`remove ` per package, or `vendor --revert` which unwinds every vendored package, then re-run `scan --mode hosted`). Both leave the entry byte-identical; neither changes exit code or status. **yarn berry line endings (v5.0)**: yarn writes a NEW `yarn.lock` with the OS line ending (`os.EOL` — CRLF on Windows) and keeps an existing lock's majority ending on every later write, and a `core.autocrlf` checkout turns an LF lock CRLF on any OS — so a uniformly CRLF lock is rewritten in its own ending: every untouched byte (a leading BOM included) round-trips (and `rollback`'s upstream restore keeps the lock's own ending). A lock that MIXES CRLF and LF (or holds a bare CR) has no single ending to keep — yarn's own `--immutable` check rejects it too (YN0028) — so it is refused untouched with `redirect_yarn_berry_mixed_line_endings` (the detail names `yarn install`, which normalizes it). The root `package.json`, which the rewrite re-renders to add `resolutions`, gets the same gate: a mixed one is refused untouched with the same code — the decision vendored mode takes with `vendor_yarn_berry_mixed_line_endings`, from the same shared berry gate set. This replaces v4's `redirect_yarn_berry_crlf_unsupported`, which refused every CRLF lock and is no longer emitted. A vendored→hosted takeover of a yarn-berry entry is checked against these project gates (mixed `yarn.lock` / `package.json` line endings, unsupported `cacheKey`, a non-zero `.yarnrc.yml` `compressionLevel`) on the project as it is before any revert, because the revert re-renders `package.json` in its majority ending — wet and `--dry-run` alike. A refused purl keeps its vendored wiring, ledger entry and artifact byte-identical, is skipped with the gate's code (reported in `warnings[]`, followed by `redirect_takeover_kept_vendored`) and is never announced as `redirect_takeover_reverted_vendored`. **Attribution gate (v5.0).** A hosted run never leaves wiring that lockfile discovery calls contested: before the rewrite writes any lockfile, the same discovery `vex`, `list`, `rollback`, `remove` and `vendor` read runs over the project as the rewrite would leave it. A candidate whose pin discovery reads but cannot attribute to one package version (a requirements `-r` include resolving the same version from the registry beside a rewired `Pipfile.lock`, #567; a Maven pin in a ``, #260) is left out of the rewrite and reported as a `skipped` event with `errorCode` `redirect_unattributable` (nothing written for it; exit code unchanged). Unchanged: a pin in a file discovery does not read (a pre-2.6 bundler `Gemfile`, locked by the next `bundle install`) keeps the rewriter's verdict, and so does a deliberate partial redirect the run already reports (a bundled or `bun patch`-ed copy left on the registry, a yarn `npm:` alias entry left on the registry with `redirect_yarn_classic_alias_skipped` / `redirect_yarn_berry_alias_skipped` while the package's direct entry is pinned, a dep withheld from the vlt rewrite while a sibling lock takes it), and so does a vendored→hosted takeover: its vendored wiring is reverted in the run's staged overlay before the rewrite plans, and it is redirected when the rewriters pin it (otherwise it is retracted and stays vendored), in a `--dry-run` preview the same. A pin whose own pnpm or yarn lock also installs an unpatched copy of the same `name@version` (a pnpm `file:` directory or tarball dependency; a yarn `file:` / URL / berry `portal:` / `link:` dependency declared under another name) is written too, with a `redirect_copy_unpatched` warning naming that entry, and is not attested. A lockless NuGet / Cargo pin (an exclusive Socket source mapping without `packages.lock.json`, a Cargo registry pin without `Cargo.lock`) is still written, with a `redirect_pin_lockless` warning: no lockfile records its version, so `vex` cannot attest it and `rollback` / `remove` / `vendor` refuse it until the lockfile exists (whether such pins should be written at all is an open decision). The rollout's recorded view uses the same discovery: a uuid a file merely mentions (a stale `package.json` field, an inactive `pdm.lock`, a comment) is not a pin and does not count as already patched. @@ -2559,6 +2559,7 @@ command's integration tests check its output. | `patch_withdrawn` | `rolledBack` / `verified` (hosted event) | scan `--mode hosted`: the lockfile pinned a patch the server reports `withdrawn` or `not_found`; the pin was restored (dry run: would be) to its upstream registry entry. `reason` names the uuid and status. See "Withdrawn pins". | | `vendor_prebuilt_stub_invalid` | `failed` | RubyGems: the server stub lacks required attributes or is otherwise invalid; no local stub fallback is permitted. | | `bazelignore_not_updated` | run-level `warnings[]` | Go vendored and agent mode in a Bazel workspace (additive): the `.bazelignore` at the Bazel root, which keeps Bazel and gazelle out of the module copies under `.socket/vendor/golang` and `.socket/go-patches`, was not updated. It could not be read or written (a symbolic link or a directory in its place, a read-only directory, a read-only file). The detail names the line to add, or says that the lines of a copy that is gone can be deleted; the command's patch or undo stands and its exit code is unchanged. See docs/ecosystems.md "Bazel workspaces: `.bazelignore`". | +| `redirect_cargo_bazel_config_unread` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (additive): a rules_rust crate_universe hub (`crate.from_cargo` in `MODULE.bazel` or a file it includes, or `crates_repository` in the `WORKSPACE` file) reads the project's `Cargo.lock` or one of its `Cargo.toml` files, or may read them (its `cargo_lockfile` or `manifests` is not written out as strings, or a use of the extension may hide a call), and its `cargo_config` is not one string that is the label of the cargo config hosted mode writes (`.cargo/config.toml`, or the project's `.cargo/config`). cargo-bazel runs cargo with the config its hub names and no other, so the registry of a pin would be unknown to it and `bazel build` would fail (`registry index was not found in any configuration`). The crate is not pinned: nothing is written for it, its event is `skipped` / `redirect_unconfirmed` (a vendored crate that the scan would take over stays vendored: `skipped` with this code, and `redirect_takeover_kept_vendored`), and status and exit code are unchanged. One warning per crate; the detail names the hub's file and line, the config to commit (created with a comment line in it where the project has none), the attribute to give the hub (`cargo_config = "//:.cargo/config.toml"`), after which the same command pins the crate, and `--mode vendored`. For a use of the extension that may hide a call the detail names that line to rewrite and no attribute: the next run pins the crate or names the hub that is in the way. A `WORKSPACE` call counts only when the module files make no hub and the checkout names no Bazel 9 or later. The rewriter's own refusals answer first and keep their codes. A pin that is in place already stays `applied`, and no warning is given for it. Nothing is read or changed in no Bazel workspace, or where no such hub is seen to read the project's Cargo files. See docs/ecosystems.md "Bazel workspaces: Cargo". | | `maven_bazel_hub_unpatched` | run-level `warnings[]` (human: `Warning: …`) | Maven in a Bazel workspace, vendored and agent mode (additive): a pinned rules_jvm_external hub of the root `MODULE.bazel` fetches the jar of a Maven patch that this run vendored or applied, or tried to (`vendor`, `apply`, and `get` / `scan` in those modes; in agent mode only for a package it found installed; never under `--global`), and the block that hosted mode keeps in `MODULE.bazel` holds no entry for that jar. The hub builds the jar its own lock (`maven_install.json`) names, so the patch does not reach the Bazel build: these modes patch no hub. One warning per purl; the detail names the lock, the hub, the file and line of its `maven.install`, and the hosted command that patches a pinned hub. The events, the exit code and every written file are those of the same run without the workspace. A `vendor` whose `--ecosystems` leaves Maven out gives none. A vendored run that takes a hosted pin over (`vendor`, `scan` / `get --mode vendored`, `vendor --revert`) leaves the block's entry for it as it is. A hosted run never gives it: see the `maven_bazel_*` rows below and docs/ecosystems.md "Bazel workspaces: Maven". | | `maven_bazel_unreadable` | vendored / agent: run-level `warnings[]` (human: `Warning: …`); scan/get `--mode hosted`: `warnings[]` (warning) | Maven in a Bazel workspace (additive): whether a rules_jvm_external hub builds the jar of a Maven patch could not be told, because something could not be read with certainty: a hub without `lock_file` in vendored and agent mode (hosted mode gives `maven_bazel_not_pinned`), a `maven.artifact` or `maven.from_toml` tag that cannot be read, a lock that is missing, empty, not format 2 or 3, larger than 16 MiB or no JSON, a `WORKSPACE` file that calls `maven_install` or cannot be read while `.bazeliskrc` or `.bazelversion` at the Bazel root names no release of Bazel 9 or later, a `maven.install` whose `name` or `lock_file` is not a plain string, a construct of `MODULE.bazel` the reader does not follow, and, in vendored and agent mode, a `maven.override` of the artifact (hosted mode gives `maven_bazel_no_effect`). Hosted mode also gives it for a hub on an isolated `use_extension`, an rc file that cannot be read (also one that an import takes from outside the workspace by a relative path, and one with a line whose reading Bazel's own source calls an oversight: a quote that is never closed, a `\` with nothing after it), a `MODULE.bazel` that is a symbolic link, and a block of `MODULE.bazel` that was changed by hand (also in its first line alone), and then writes no entry for the hub. One warning per purl; the detail names the file, the line and the reason. | | `maven_bazel_not_at_root` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (additive): a rules_jvm_external hub builds the jar of a Maven patch, and the project is below the root of its Bazel workspace, in whose `MODULE.bazel` the patch of a hub is written. Like the five codes below and `maven_bazel_unreadable` in hosted mode, it means that the run wrote no entry for the jar in the block of the root `MODULE.bazel`, because it could not show from the checkout that the entry works. One warning per purl, in the place of `redirect_maven_no_pom` for that purl; the detail says why and ends with what the block of the root holds for the jar, which is read for a project below the root too (the detail then names the file as "the MODULE.bazel at the root"). Nothing: "socket-patch wrote nothing to MODULE.bazel for it, so the Bazel build keeps the upstream jar". The event is the one of the same run without the hub: `skipped` / `redirect_unconfirmed` in a tree only Bazel builds, `applied` beside a `pom.xml` or Gradle pin that landed. The entry of the patch, from an earlier run: that entry stays, the event is `applied`, and the detail ends "socket-patch left the entry that MODULE.bazel holds for it as it is". The entry of another patch of the same jar: it stays too, the event is the one without the hub, and the detail ends "socket-patch wrote nothing to MODULE.bazel for it: the entry of patch remains there, and where the block is in effect the Bazel build takes that patch's jar". Exit code unchanged. With none of these codes the run writes the block: `redirect.rewrittenFiles` holds `MODULE.bazel`, the event is `applied`, and `rollback` / `remove` take the entry out. See docs/ecosystems.md "Bazel workspaces: Maven". | diff --git a/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs b/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs index c2b0292e9..df1269daa 100644 --- a/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs +++ b/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs @@ -120,11 +120,31 @@ //! the hub as the rules_rust README has it, which names no `cargo_config` //! ([`open_readme_hub`]). cargo-bazel then hands no cargo config of the //! checkout to its cargo, so the registry that hosted mode defines in -//! `.cargo/config.toml` and names in `Cargo.toml` is one that cargo has -//! never heard of. The flow holds what the release does there: the scan -//! exits 0 and reports the crate as hosted, and the next `bazel build` -//! fails in the hub with cargo's `registry index was not found in any -//! configuration`. +//! `.cargo/config.toml` and names in `Cargo.toml` would be one that cargo +//! has never heard of, and the hub would fail on the manifest (`registry +//! index was not found in any configuration`). The flow holds that +//! `socket-patch` writes no such pin: +//! +//! - (a) The build prints the unpatched marker. +//! - (b) `scan --mode hosted --yes` exits 0, reports the crate as skipped +//! and warns `redirect_cargo_bazel_config_unread`: the warning names the +//! hub by its file and line, the config to commit, the attribute the hub +//! lacks and vendored mode. No file of the checkout changes, and a +//! person is told the same. +//! - (c) The same build still prints the unpatched marker and changes no +//! file. +//! - (d) The second way the warning names, on the hub as it is: `scan +//! --mode vendored --yes`, and the build prints the patched marker, from +//! the committed copy. `vendor --revert` gives the checkout back. +//! - (e) The first way: the checkout gets `.cargo/config.toml`, with a +//! comment line in it, and the hub `cargo_config = +//! "//:.cargo/config.toml"`. The same scan then applies the patch and +//! rewrites the three files, and the build prints the patched marker, +//! from the patch host. +//! +//! The tests without a Bazel run the same command in checkouts of the +//! fixture's Cargo files ([`checkout`]): what it writes where no hub reads +//! them is what it writes in no Bazel workspace, byte for byte. //! //! # Known gaps //! @@ -160,10 +180,10 @@ use serde_json::json; use sha2::{Digest, Sha256, Sha512}; use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; -use crate::envelope::{event_for, has_event}; +use crate::envelope::{codes_in, event_for, has_event}; use crate::harness::{ - changed, fixture_path, Cell, Fixture, Ran, Release, StandIn, Tree, CONSUMER, NOTHING, - REPO_CACHE, + cell_without_bazel, changed, fixture_path, Cell, Fixture, Ran, Release, StandIn, Tree, + CONSUMER, NOTHING, REPO_CACHE, }; use crate::hosted_memory_common::{Patch, ORG}; @@ -543,6 +563,18 @@ fn open_readme_hub(test: &str) -> Option<(tempfile::TempDir, Cell, StandIn, Pack /// the patch server (a hosted pin is recognised by that origin) and as the /// index `rollback` reads the upstream checksum from. fn socket_patch(cell: &Cell, standin: &StandIn, args: &[&str]) -> Ran { + with_hosts(standin, |api, env| cell.socket_patch(api, args, env)) +} + +/// [`socket_patch`] as a person runs it, without `--json`: what it printed +/// on stdout and on stderr. +fn socket_patch_person(cell: &Cell, standin: &StandIn, args: &[&str]) -> [String; 2] { + with_hosts(standin, |api, env| cell.socket_patch_person(api, args, env)) +} + +/// `run` with the API origin and the variables of a `socket-patch` child +/// that talks to `standin`. +fn with_hosts(standin: &StandIn, run: impl FnOnce(&str, &[(&str, &str)]) -> T) -> T { let origin = standin.origin(); let index = format!("{origin}/{INDEX}"); let env = [ @@ -550,7 +582,7 @@ fn socket_patch(cell: &Cell, standin: &StandIn, args: &[&str]) -> Ran { ("SOCKET_CRATES_INDEX", index.as_str()), ("CARGO_HOME", NO_CARGO_HOME), ]; - cell.socket_patch(&origin, args, &env) + run(&origin, &env) } /// A text file of a tree. @@ -1102,6 +1134,72 @@ fn vendored_from_cargo_with_config() { ); } +/// The code of the warning for a crate that hosted mode does not pin +/// because a rules_rust hub would not read the cargo config of the pin. +const UNREAD: &str = "redirect_cargo_bazel_config_unread"; + +/// The attribute a hub names the project's cargo config with, and how the +/// warning says to write it. +const NAMES_CONFIG: &str = "cargo_config = \"//:.cargo/config.toml\""; +const GIVE_THE_HUB: &str = "give the hub `cargo_config = \"//:.cargo/config.toml\"`"; + +/// What a finished hosted command says when it pins nothing for the crate +/// because of a hub: it exits 0 with the status `success`, the crate is +/// skipped as one no file pins, no file is named as rewritten, and its one +/// warning of [`UNREAD`] names the crate, says that nothing was written +/// and ends with vendored mode. Gives the detail of that warning. +fn unread(test: &str, scan: &Ran) -> String { + let said = &scan.dump; + assert_eq!(scan.code, 0, "{test}: {said}"); + assert_eq!(scan.envelope["status"], "success", "{test}: {said}"); + assert_eq!(scan.envelope["summary"]["applied"], 0, "{test}: {said}"); + let skipped = has_event(&scan.envelope, "skipped", "redirect_unconfirmed", PURL); + assert!(skipped, "{test}: {said}"); + let rewritten = &scan.envelope["redirect"]["rewrittenFiles"]; + assert_eq!(*rewritten, json!([]), "{test}: {said}"); + let warnings = scan.envelope["warnings"].as_array().expect("warnings"); + let ours: Vec<&str> = warnings + .iter() + .filter(|warning| warning["code"] == UNREAD) + .filter_map(|warning| warning["detail"].as_str()) + .collect(); + let [detail] = ours[..] else { + panic!("{test}: not one {UNREAD} warning: {said}"); + }; + let vendored = + "; or use `--mode vendored`, which commits the patched crate and needs no registry"; + let about = detail.starts_with(&format!("{NAME}@{VERSION}: ")) + && detail.contains("; dependency skipped (nothing rewritten). ") + && detail.ends_with(vendored); + assert!(about, "{test}: {detail}"); + detail.to_string() +} + +/// [`unread`] for the hub at `at` (`MODULE.bazel:10`), which names no +/// config: the warning names the hub, the file to commit first, and the +/// attribute to give the hub. +fn withheld(test: &str, scan: &Ran, at: &str) { + let detail = unread(test, scan); + let starts = format!( + "{NAME}@{VERSION}: the rules_rust hub at {at} reads the Cargo files of this project, and \ + it names no `cargo_config`. " + ); + assert!(detail.starts_with(&starts), "{test}: {detail}"); + let way = format!( + "Commit {CONFIG} (where the project has none, create it with a comment line in it: \ + `rollback` deletes a config it leaves empty, and a hub that names a missing file fails), \ + {GIVE_THE_HUB}, the label of that file as one string, and run this command again; or \ + use `--mode vendored`" + ); + assert!(detail.contains(&way), "{test}: no {way:?} in {detail}"); +} + +/// The line of `module` on which its hub is called, as a warning names it. +fn hub_at(module: &str) -> String { + let call = module.lines().position(|line| line == "crate.from_cargo("); + format!("{MODULE}:{}", call.expect("a hub") + 1) +} + /// The flow of the hub without a cargo config (module docs). #[test] #[ignore = "real Bazel; run with --ignored"] @@ -1119,46 +1217,298 @@ fn hosted_from_cargo_without_config() { // (a) The unpatched build, with the upstream crate from the stand-in. let before = builds_upstream(&cell, &standin, &upstream); - // (b) The one command of hosted mode. It reports the crate as hosted - // and rewrites the three files, as on a hub that names the config. The - // config is new: the checkout had none, and it holds the patch registry - // alone. - let scan = socket_patch(&cell, &standin, &["scan", "--mode", "hosted", "--yes"]); - let pinned = cell.pinned_by(&scan, PURL, &patch.uuid, &before, &[CONFIG, LOCK, MANIFEST]); - let (index, registry) = (format!("sparse+{origin}{}/", patch.index), patch.registry()); - let defined = format!("[registries.{registry}]\nindex = \"{index}\"\n"); - assert_eq!(text(&pinned, CONFIG), defined, "{test}: {CONFIG}"); - let named = format!("registry = \"{registry}\""); - assert!( - text(&pinned, MANIFEST).contains(&named), - "{test}: {MANIFEST}" - ); + // (b) The one command of hosted mode pins nothing: the hub would fail + // on the pin. It says so, names the hub and the two ways on, and + // changes no file. A person reads the same warning. + let command = ["scan", "--mode", "hosted", "--yes"]; + let readme = text(&before, MODULE); + let scan = socket_patch(&cell, &standin, &command); + withheld(test, &scan, &hub_at(&readme)); + assert_eq!(changed(&before, &cell.tree()), NOTHING, "{test}"); + let [_, stderr] = socket_patch_person(&cell, &standin, &command); + let told = stderr + .lines() + .any(|line| line.starts_with("Warning: ") && line.contains(GIVE_THE_HUB)); + assert!(told, "{test}: a person is not told:\n{stderr}"); + assert_eq!(changed(&before, &cell.tree()), NOTHING, "{test}"); + + // (c) The same build: still the upstream crate, no file changed, and + // Bazel's lock stands. + cell.builds_patched(&before, &upstream.marker); + + // (d) The second way the warning names, on the hub as it is: vendored + // mode needs no registry, and the build is the patched one, from the + // committed copy. Its undo gives the checkout back. + let vendored = socket_patch(&cell, &standin, &["scan", "--mode", "vendored", "--yes"]); + let said = &vendored.dump; + assert_eq!(vendored.code, 0, "{test}: {said}"); + assert_eq!(vendored.envelope["summary"]["applied"], 1, "{test}: {said}"); + cell.builds(&patch.package.marker); + let undo = socket_patch(&cell, &standin, &["vendor", "--revert"]); + undone(&cell, &undo, "removed", &before, &upstream.marker); + let fetched = standin.asked(&patch.route); - // (c) The same build fails: the hub hands its cargo no config, so the - // registry the manifest names is defined nowhere cargo looks. - let build = cell.bazel("build", &[CONSUMER]); - let undefined = format!("registry index was not found in any configuration: `{registry}`"); + // (e) The first way: that file in the checkout, with a comment line in + // it, and the one attribute on the hub. Then the same command applies + // the patch, and the build is the patched one, from the patch host, + // which is asked for its index file of the crate and, at least twice, + // for the crate file. + let lockfile = " cargo_lockfile = "; + let named = readme.replacen(lockfile, &format!("{HUB_CONFIG}{lockfile}"), 1); + assert!(named.contains(NAMES_CONFIG), "{test}:\n{named}"); + cell.write(MODULE, named); + let comment = "# The cargo settings of this project.\n"; + cell.write(CONFIG, comment); + let edited = cell.tree(); + let scan = socket_patch(&cell, &standin, &command); + let pinned = cell.pinned_by(&scan, PURL, &patch.uuid, &edited, &[CONFIG, LOCK, MANIFEST]); + let warned = codes_in(&scan.envelope["warnings"]); assert!( - !build.ok() && build.stderr().contains(&undefined), - "{test}: the build after `scan --mode hosted` did not fail with {undefined:?}:\n{}", - build.dump() + !warned.iter().any(|code| code == UNREAD), + "{test}: {}", + scan.dump ); - assert_eq!( + let (index, registry) = (format!("sparse+{origin}{}/", patch.index), patch.registry()); + let defined = format!("{comment}\n[registries.{registry}]\nindex = \"{index}\"\n"); + assert_eq!(text(&pinned, CONFIG), defined, "{test}: {CONFIG}"); + cell.builds(&patch.package.marker); + let (asked, fetches) = ( standin.asked(&patch.entry), - 0, - "{test}: cargo read the patch index" + standin.asked(&patch.route) - fetched, + ); + assert!( + asked > 0 && fetches >= 2, + "{test}: requests to the patch host by the hosted build: {asked} for its index file of \ + the crate and {fetches} for the crate file" ); println!( "{test}: Bazel {release} built {:?} on the hub of the rules_rust README; `scan --mode \ - hosted` exited 0, reported the crate applied and rewrote {:?}; the next `bazel build` \ - failed with {undefined:?}; {:?}", + hosted` exited 0, warned {UNREAD} and changed no file, and the same build printed the \ + same line; after `scan --mode vendored` on that hub the build printed {:?}, and \ + `vendor --revert` restored the checkout; with {CONFIG} in the checkout and \ + `{NAMES_CONFIG}` on the hub the hosted scan rewrote {:?}, and the build printed the \ + same line from the patch host (requests for the crate file: {fetches}); {:?}", upstream.marker, + patch.package.marker, [CONFIG, LOCK, MANIFEST], started.elapsed() ); } +/// Without a Bazel: the Cargo project of the fixture in a checkout of its +/// own, in the directory `project` of it (`""`: the root, else with a `/` +/// at its end), its lock pinning `upstream`, with `files` beside it, each +/// from the root of the checkout. +fn checkout(project: &str, upstream: &Package, files: &[(&str, &str)]) -> Cell { + let cell = cell_without_bazel("9.3.2"); + let manifest = std::fs::read(fixture_path(FIXTURE.dir).join("common").join(MANIFEST)); + cell.write(&format!("{project}{MANIFEST}"), manifest.expect(MANIFEST)); + cell.write(&format!("{project}{LOCK}"), lock_pinning(upstream)); + cell.write(".git/HEAD", "ref: refs/heads/main\n"); + for (rel, text) in files { + cell.write(rel, text); + } + cell +} + +/// The `MODULE.bazel` of the newer era, whose hub names the cargo config. +fn module_naming_config() -> String { + let path = fixture_path(FIXTURE.dir).join("e9").join(MODULE); + std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{}: {e}", path.display())) +} + +/// An upstream crate, a stand-in that holds it, and a patch of it that the +/// stand-in grants. +fn granted() -> (Package, StandIn, Patched) { + let (upstream, patch) = (Package::upstream(), Patched::new()); + let standin = registry(&upstream); + grant(&standin, &upstream, &patch, &patch.package.crate_file); + (upstream, standin, patch) +} + +/// Without a Bazel: on the hub of the README every hosted command pins +/// nothing and changes no file, a dry run and `get` included, and a person +/// reads which package was left out and why. With the config in the +/// checkout and the attribute on the hub a line that may hide a hub still +/// withholds the pin, and the warning names that line and no attribute. +/// Without the line the same scan pins the crate, a scan after it leaves +/// the pin as it is, and `rollback` gives the checkout back, the config +/// with its comment line. +#[test] +fn a_hub_without_the_config_gets_no_pin_until_it_names_the_config() { + let (upstream, standin, patch) = granted(); + let named = module_naming_config(); + let readme = named.replace(HUB_CONFIG, ""); + let cell = checkout("", &upstream, &[(MODULE, &readme)]); + let before = cell.tree(); + let scan = ["scan", "--mode", "hosted", "--yes"]; + let commands: [&[&str]; 3] = [ + &scan, + &["scan", "--mode", "hosted", "--yes", "--dry-run"], + &["get", PURL, "--mode", "hosted", "--yes"], + ]; + for command in commands { + let ran = socket_patch(&cell, &standin, command); + withheld(&format!("{command:?}"), &ran, &hub_at(&readme)); + assert_eq!(changed(&before, &cell.tree()), NOTHING, "{command:?}"); + } + let [stdout, stderr] = socket_patch_person(&cell, &standin, &scan); + let left_out = format!( + " {PURL}: no lockfile entry pinning it could be rewritten (see the warning below)" + ); + let lines: Vec<&str> = stderr.lines().collect(); + let warning = format!( + "Warning: {NAME}@{VERSION}: the rules_rust hub at {}", + hub_at(&readme) + ); + let told = lines.contains(&"No patches could be switched to hosted:") + && lines.contains(&left_out.as_str()) + && lines.iter().any(|line| line.starts_with(&warning)); + assert!(told, "--- stdout\n{stdout}\n--- stderr\n{stderr}"); + assert_eq!(changed(&before, &cell.tree()), NOTHING); + + let comment = "# The cargo settings of this project.\n"; + cell.write(MODULE, &named); + cell.write(CONFIG, comment); + let edited = cell.tree(); + // The hub that is seen names the config, and another may be hidden. + let hidden = format!("{named}make = crate.from_cargo\n"); + cell.write(MODULE, &hidden); + let detail = unread("a hidden call", &socket_patch(&cell, &standin, &scan)); + let line = format!( + "{NAME}@{VERSION}: {MODULE}:{}: `crate` is used other than to call a tag, so a rules_rust \ + hub that socket-patch cannot read may take the Cargo files of this project. ", + hidden.lines().count() + ); + let way = "Write that line as the call of a tag (`crate.from_cargo(...)`) or as \ + `use_repo(...)` and run this command again: it then pins the crate, or names the \ + hub that is in the way; or use `--mode vendored`"; + let named_line = detail.starts_with(&line) && detail.contains(way); + assert!(named_line && !detail.contains("ive the hub"), "{detail}"); + cell.write(MODULE, &named); + assert_eq!(changed(&edited, &cell.tree()), NOTHING); + let ran = socket_patch(&cell, &standin, &scan); + let pinned = cell.pinned_by(&ran, PURL, &patch.uuid, &edited, &[CONFIG, LOCK, MANIFEST]); + // A pin that is in place stays what it was, whatever the hub names by + // then: applied, with no file to rewrite and no word about the hub. + cell.write(MODULE, &readme); + let again = socket_patch(&cell, &standin, &scan); + assert_eq!(again.envelope["summary"]["applied"], 1, "{}", again.dump); + assert_eq!( + codes_in(&again.envelope["warnings"]), + NOTHING, + "{}", + again.dump + ); + cell.write(MODULE, &named); + assert_eq!(changed(&pinned, &cell.tree()), NOTHING, "{}", again.dump); + let undo = socket_patch(&cell, &standin, &["rollback", "--yes"]); + assert_eq!(undo.code, 0, "{}", undo.dump); + assert_eq!(undo.envelope["summary"]["rolledBack"], 1, "{}", undo.dump); + assert_eq!(changed(&edited, &cell.tree()), NOTHING, "{}", undo.dump); + + // KNOWN GAP, and what the comment line is for: a config that holds + // nothing else is deleted by the `rollback` that empties it, and a hub + // that names a missing file fails. Once `rollback` keeps a file it did + // not create, the warning needs no such advice and this goes. + cell.write(CONFIG, ""); + let ran = socket_patch(&cell, &standin, &scan); + assert_eq!(ran.envelope["summary"]["applied"], 1, "{}", ran.dump); + let undo = socket_patch(&cell, &standin, &["rollback", "--yes"]); + assert_eq!(changed(&edited, &cell.tree()), [CONFIG], "{}", undo.dump); + assert!(!cell.tree().contains_key(CONFIG), "{}", undo.dump); +} + +/// Without a Bazel: where no hub is seen to read the project's Cargo files +/// the scan is the one of a project in no Bazel workspace: the same +/// envelope, and the same three files, byte for byte. That holds for a +/// workspace without a hub, for a hub that names the config, for one over +/// the Cargo files of another directory, for a project beside the one a +/// hub reads, for a `WORKSPACE` file that Bazel 9 never reads, and for the +/// tag of another ruleset. +#[test] +fn a_project_that_no_hub_is_seen_to_read_is_pinned_as_in_no_bazel_workspace() { + let (upstream, standin, patch) = granted(); + let scan = ["scan", "--mode", "hosted", "--yes"]; + let written = [CONFIG, LOCK, MANIFEST]; + let plain = checkout("", &upstream, &[]); + let before = plain.tree(); + let unmarked = socket_patch(&plain, &standin, &scan); + let pinned = plain.pinned_by(&unmarked, PURL, &patch.uuid, &before, &written); + + let named = module_naming_config(); + let readme = named.replace(HUB_CONFIG, ""); + let hub = readme.find("crate = use_extension(").expect("a hub"); + let (no_hub, elsewhere) = (&readme[..hub], readme.replace("\"//:", "\"//third_party:")); + let rules_rs = "bazel_dep(name = \"rules_rs\", version = \"0.2.2\")\n\ + crate = use_extension(\"@rules_rs//rs:extensions.bzl\", \"crate\")\n\ + crate.from_cargo(\n name = \"crates\",\n cargo_lock = \"//:Cargo.lock\",\n \ + cargo_toml = \"//:Cargo.toml\",\n)\nuse_repo(crate, \"crates\")\n"; + let stale = "load(\"@rules_rust//crate_universe:defs.bzl\", \"crates_repository\")\n\ + crates_repository(\n name = \"crates\",\n cargo_lockfile = \ + \"//:Cargo.lock\",\n manifests = [\"//:Cargo.toml\"],\n)\n"; + let cases: [(&str, &[(&str, &str)]); 6] = [ + ("", &[(MODULE, no_hub)]), + ("", &[(MODULE, &named)]), + ("", &[(MODULE, &elsewhere)]), + ("tool/", &[(MODULE, &readme)]), + ("", &[("WORKSPACE", stale), (".bazelversion", "9.3.2\n")]), + ("", &[(MODULE, rules_rs)]), + ]; + for (project, files) in cases { + let cell = checkout(project, &upstream, files); + let before = cell.tree(); + let below = ["--cwd", project.trim_end_matches('/')]; + let command = [&scan[..], &below[..2 * usize::from(!project.is_empty())]].concat(); + let ran = socket_patch(&cell, &standin, &command); + let said = format!("{project:?} {files:?}: {}", ran.dump); + assert_eq!(ran.code, 0, "{said}"); + assert_eq!(ran.envelope, unmarked.envelope, "{said}"); + let after = cell.tree(); + let files = written.map(|file| format!("{project}{file}")); + assert_eq!(changed(&before, &after), files, "{said}"); + for (file, written) in files.iter().zip(written) { + assert_eq!(after[file], pinned[written], "{file}: {said}"); + } + } +} + +/// Without a Bazel: a crate that vendored mode patched stays vendored when +/// a hosted scan would take it over on the hub of the README: the scan +/// names the hub, keeps every file of the vendored copy and its wiring, +/// and the crate stays patched. +#[test] +fn a_vendored_crate_stays_vendored_on_a_hub_without_the_config() { + let (upstream, standin, patch) = granted(); + let readme = module_naming_config().replace(HUB_CONFIG, ""); + let cell = checkout("", &upstream, &[(MODULE, &readme)]); + let vendored = socket_patch(&cell, &standin, &["scan", "--mode", "vendored", "--yes"]); + assert_eq!( + vendored.envelope["summary"]["applied"], 1, + "{}", + vendored.dump + ); + let kept = cell.tree(); + assert!(kept.contains_key(&format!("{}/src/lib.rs", patch.vendored()))); + + let hosted = socket_patch(&cell, &standin, &["scan", "--mode", "hosted", "--yes"]); + let said = &hosted.dump; + assert_eq!(hosted.code, 0, "{said}"); + assert!( + has_event(&hosted.envelope, "skipped", UNREAD, PURL), + "{said}" + ); + let warned = codes_in(&hosted.envelope["warnings"]); + for code in [UNREAD, "redirect_takeover_kept_vendored"] { + assert!( + warned.iter().any(|warned| warned == code), + "no {code}: {said}" + ); + } + assert_eq!(changed(&kept, &cell.tree()), NOTHING, "{said}"); +} + /// Without a Bazel, what stays fixed now that no checksum is committed: the /// fixture's directories and the legs of its workflow are those of its /// table; the fixture's lock holds the one entry a test pins, with the diff --git a/crates/socket-patch-core/src/bazel/cargo.rs b/crates/socket-patch-core/src/bazel/cargo.rs new file mode 100644 index 000000000..0e5a83c92 --- /dev/null +++ b/crates/socket-patch-core/src/bazel/cargo.rs @@ -0,0 +1,690 @@ +//! Whether a rules_rust hub of a Bazel workspace would fail on the registry +//! that hosted mode pins a crate to. +//! +//! A crate_universe hub is a `crate.from_cargo` tag under Bzlmod, or a +//! `crates_repository` call in a `WORKSPACE` file. Its tool, cargo-bazel, +//! links what stands beside each `Cargo.toml` the hub names into a +//! directory of its own, writes a root manifest of its own there and runs +//! cargo there, with the `Cargo.lock` the hub names (`symlink_roots` and +//! `write_root_manifest` in `crate_universe/src/splicing/splicer.rs`). Of +//! the cargo configs of the checkout that cargo is given one, the file the +//! hub names as `cargo_config`, and none when it names none +//! (`setup_cargo_config` in the same file; all of it the same at rules_rust +//! 0.56.0 and 0.75.0). Hosted mode names a registry of its own on the +//! dependency in `Cargo.toml` and defines it in the project's cargo config. +//! A hub that reads the manifest and not that config fails on it +//! (``registry index was not found in any configuration: +//! `socket-patch-` ``), and with it every Bazel build that needs the +//! hub: measured on Bazel 9.3.2 (rules_rust 0.75.0) and 7.7.2 (0.56.0) for +//! the hub as the rules_rust README has it, which names no `cargo_config`. +//! +//! [`unread`] finds such a hub for a project, so that the cargo rewriter +//! writes no pin there. It answers from the calls it can see, through +//! [`super::workspace`]: +//! +//! - A hub concerns the project when a label of its `cargo_lockfile` or its +//! `manifests` is a Cargo file of the project, and also when one of the +//! two is not written out as a string or a list of strings, or is left +//! to a `**`: it may read them. +//! - Such a hub is in the way unless its `cargo_config` is one string, the +//! label of the config the rewriter writes. A config that cannot be read +//! is not taken for the right one. +//! - A call counts whether or not it is a statement of its own +//! ([`Call::whole`]): one that may run is one that may fail. +//! - When the workspace uses a name bound to the extension in a way that +//! may hide a call ([`super::workspace::Tags::doubts`]), a hub may read +//! the project's files, and that is in the way too. The way on is then +//! that line: there may be no hub to give an attribute. +//! - The calls of a `WORKSPACE` file count when the module files make no +//! such tag and the checkout does not name a Bazel from 9.0.0 on, which +//! never reads that file. Bazel 7 reads it, and Bazel 8 when it is told +//! to. Beside module files that make one, the file is taken for what a +//! move to Bzlmod left behind. +//! +//! So a reading that is not certain withholds the pin wherever a hub, or a +//! use of the extension through a name bound to it, is seen: the cost is a +//! patch that waits for one line of the user's, against a build that fails. +//! Where nothing of the kind is seen the answer is `None` and a run is the +//! one it was: a project in no Bazel workspace, a workspace without a +//! crate_universe hub, and a hub whose labels are all written out and name +//! other files. +//! +//! Not seen, and there a hub without the config fails as before: +//! +//! - a hub that another Bazel module declares, and a `crates_repository` in +//! a `.bzl` file that the `WORKSPACE` file loads or under another name +//! than its own; +//! - a tag called on `use_extension(...)` itself, with no name bound to it, +//! a `MODULE.bazel` that cannot be read (a link that leads out of the +//! Bazel root, a file past the reader's bounds), and whatever else the +//! files of the workspace hide from every answer ([`Workspace::doubts`]), +//! where no hub is seen; +//! - what Bazel is told by flags and rc files, none of which is read: +//! `--noenable_bzlmod` has it read the `WORKSPACE` file in place of +//! module files that make a hub; +//! - a Bazel root above a project that is in no git checkout +//! ([`super::bazel_root`] looks at the project's own directory there). +//! +//! What the hub's `cargo_config` holds is not looked at either: the +//! registry is defined in the file the rewriter writes, so the hub has to +//! name that file. + +use std::path::Path; + +use super::rc::declared_release; +use super::starlark::{Call, Value}; +use super::workspace::{disk, Entry, Extension, Repo, Workspace}; + +/// The key under which a hosted run hands what [`unread`] says to the +/// cargo rewriter, among its candidate files: no path of a file. +pub const KEY: &str = ""; + +/// The warning for a crate that hosted mode does not pin because of it. +pub const UNREAD: &str = "redirect_cargo_bazel_config_unread"; + +/// The module extension whose `from_cargo` tag makes a hub, by the two +/// files that define it at rules_rust 0.56.0 and 0.75.0: `extension.bzl` +/// hands on the `crate` of `extensions.bzl`. +const EXTENSIONS: [Extension<'static>; 2] = [ + Extension { + module: "rules_rust", + file: "//crate_universe:extensions.bzl", + name: "crate", + }, + Extension { + module: "rules_rust", + file: "//crate_universe:extension.bzl", + name: "crate", + }, +]; + +/// Why hosted mode must not pin a crate of the project at `project`, and +/// what to do about it: a rules_rust hub of the project's Bazel workspace +/// reads, or may read, one of `files` (the `Cargo.lock` and the manifests +/// the rewriter writes, each from the project) and is not seen to name +/// `config`, the cargo config the rewriter writes, from the project too. +/// `None` when no such hub is seen (the module docs say what is), and for a +/// project in no Bazel workspace ([`super::bazel_root`], whose bounds these +/// are), for which nothing is read. +pub fn unread(project: &Path, files: &[String], config: &str) -> Option { + let root = super::bazel_root(project)?; + // Bazel has `/` between the parts of a path on every system. + let parts = root.project.components(); + let parts: Vec<_> = parts + .map(|part| part.as_os_str().to_string_lossy()) + .collect(); + let view = disk(&root.dir); + judge(&view, &parts.join("/"), files, config) +} + +/// The files of the main repository that `call` names under the attribute +/// `name`, each from the Bazel root: none for an attribute it does not +/// pass. `None` when they are not written out: the attribute is no string +/// and no list of strings, or it is not written and the call passes `**`. +/// +/// `own` is the name a `WORKSPACE` file gives the main repository, for a +/// call of that file: `workspace(name = "shop")` makes `@shop//` the main +/// repository there (`WorkspaceGlobals.workspace`, Bazel 7.7.2 and 8.8.2). +/// Under a name that is not written out, any `@name//` may be it. +fn named(ws: &Workspace, own: Option<&Value>, call: &Call, name: &str) -> Option> { + let labels: Vec<&String> = match call.arg(name) { + None if call.unpacks() => return None, + None => Vec::new(), + Some(Value::Str(label)) => vec![label], + Some(Value::List(labels)) => labels.iter().collect(), + Some(_) => return None, + }; + let main = |label: &&String| { + let label = ws.label(label)?; + let main = match (&label.repo, own) { + (Repo::Main, _) => true, + (Repo::Apparent(repo), Some(Value::Str(own))) => repo == own, + (Repo::Apparent(_), Some(_)) => true, + _ => false, + }; + main.then(|| label.path()) + }; + Some(labels.iter().filter_map(main).collect()) +} + +/// [`unread`] for the workspace that `view` shows and the project in its +/// directory `project` (empty: the root itself). +fn judge( + view: &dyn Fn(&str) -> Entry, + project: &str, + files: &[String], + config: &str, +) -> Option { + let below = |file: &str| match project { + "" => file.to_string(), + project => format!("{project}/{file}"), + }; + let (files, written) = (files.iter().map(|file| below(file)), below(config)); + let files: Vec = files.collect(); + // What stands after the reason, whatever it is, around the first way + // on, which is the reason's own. + let withheld = |why: String, way: String| { + format!( + "{why}. cargo-bazel hands the cargo it runs no cargo config but the one its hub \ + names, and without {written}, where hosted mode defines the patch registry, that \ + cargo fails on the pinned Cargo.toml (`registry index was not found in any \ + configuration`) and `bazel build` with it; dependency skipped (nothing rewritten). \ + {way}; or use `--mode vendored`, which commits the patched crate and needs no \ + registry" + ) + }; + let workspace = Workspace::read(view); + // The calls that make a hub, each with its file, the first doubt about + // a call that may be hidden, and the name a `WORKSPACE` file whose + // calls count gives the main repository. + let (mut hubs, mut hidden, mut own): (Vec<(&str, &Call)>, _, _) = (Vec::new(), None, None); + for extension in &EXTENSIONS { + let tags = workspace.tags(extension, "from_cargo"); + hubs.extend(tags.calls.iter().map(|tag| (tag.file, tag.call))); + hidden = hidden.or(tags.doubts.into_iter().next()); + } + if let (true, None, Some(Ok((file, legacy)))) = (hubs.is_empty(), &hidden, &workspace.legacy) { + let read = declared_release(view).is_none_or(|release| release < (9, 0, 0)); + let hub = |call: &&Call| read && call.callee == "crates_repository"; + let calls = legacy.calls.iter().filter(hub); + hubs.extend(calls.map(|call| (file.as_str(), call))); + let naming = legacy.calls.iter().find(|call| call.callee == "workspace"); + own = naming.and_then(|call| call.arg("name")); + } + for (file, call) in hubs { + let read = ["cargo_lockfile", "manifests"].map(|name| named(&workspace, own, call, name)); + let ours = |path: &String| files.contains(path); + let (reads, how) = match read.iter().flatten().flatten().any(ours) { + true => ("reads", ""), + false if read.iter().all(Option::is_some) => continue, + false => ( + "may read", + " (its `cargo_lockfile` or its `manifests` is not written out)", + ), + }; + let one = matches!(call.arg("cargo_config"), Some(Value::Str(_))); + if one && named(&workspace, own, call, "cargo_config") == Some(vec![written.clone()]) { + continue; + } + let names = match call.arg("cargo_config") { + None if !call.unpacks() => "it names no `cargo_config`".to_string(), + _ => format!( + "its `cargo_config` is not the label of {written} as one string (the settings \ + of another file it names belong in {written})" + ), + }; + let why = format!( + "the rules_rust hub at {file}:{} {reads} the Cargo files of this project{how}, and \ + {names}", + call.line + ); + // The file first: a hub that names a missing one fails. + let way = format!( + "Commit {written} (where the project has none, create it with a comment line in it: \ + `rollback` deletes a config it leaves empty, and a hub that names a missing file \ + fails), give the hub `cargo_config = \"//{project}:{config}\"`, the label of that \ + file as one string, and run this command again" + ); + return Some(withheld(why, way)); + } + hidden.map(|doubt| { + let why = format!( + "{}:{}: {}, so a rules_rust hub that socket-patch cannot read may take the Cargo \ + files of this project", + doubt.file, doubt.line, doubt.why + ); + // No hub to give an attribute: one that is seen may name the config. + let way = "Write that line as the call of a tag (`crate.from_cargo(...)`) or as \ + `use_repo(...)` and run this command again: it then pins the crate, or names \ + the hub that is in the way"; + withheld(why, way.to_string()) + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + const CONFIG: &str = ".cargo/config.toml"; + + /// The files the rewriter writes for a project with one member. + fn project_files() -> Vec { + ["Cargo.lock", "Cargo.toml", "member/Cargo.toml"] + .map(String::from) + .to_vec() + } + + /// A root module that asks for the extension, and `hub` after it. + fn module(hub: &str) -> String { + format!( + "module(name = \"shop\", version = \"0.0.0\")\n\ + bazel_dep(name = \"rules_rust\", version = \"0.75.0\")\n\ + crate = use_extension(\"@rules_rust//crate_universe:extensions.bzl\", \"crate\")\n\ + {hub}\nuse_repo(crate, \"crates\")\n" + ) + } + + /// A `crate.from_cargo` with `attributes`, each on a line of its own. + fn hub(attributes: &[&str]) -> String { + format!("crate.from_cargo(\n {},\n)", attributes.join(",\n ")) + } + + /// The attributes of the hub of the rules_rust README. + const README: [&str; 3] = [ + "name = \"crates\"", + "cargo_lockfile = \"//:Cargo.lock\"", + "manifests = [\"//:Cargo.toml\"]", + ]; + + /// What [`judge`] says of the project in `project` of a workspace with + /// the files `files`. + fn said(files: &[(&str, &str)], project: &str) -> Option { + let view = |path: &str| { + let file = files.iter().find(|(name, _)| *name == path); + file.map_or(Entry::Absent, |(_, text)| Entry::Text(text.to_string())) + }; + judge(&view, project, &project_files(), CONFIG) + } + + /// [`said`] for the root project of a workspace whose `MODULE.bazel` + /// has the one hub of `attributes`. + fn of_hub(attributes: &[&str]) -> Option { + said(&[("MODULE.bazel", &module(&hub(attributes)))], "") + } + + /// The README hub with `more` attributes. + fn readme_with<'a>(more: &[&'a str]) -> Vec<&'a str> { + README.iter().chain(more).copied().collect() + } + + #[test] + fn the_hub_of_the_readme_is_in_the_way_and_the_warning_names_both_ways_on() { + let why = of_hub(&README).expect("the README hub"); + // The call is on line 4 of the file; the attribute, the file to + // commit and the other mode are named. + for part in [ + "the rules_rust hub at MODULE.bazel:4 reads the Cargo files of this project, and it \ + names no `cargo_config`. ", + "where hosted mode defines the patch registry", + "`registry index was not found in any configuration`", + "dependency skipped (nothing rewritten)", + "Commit .cargo/config.toml (where the project has none, create it with a comment \ + line in it: `rollback` deletes a config it leaves empty", + "give the hub `cargo_config = \"//:.cargo/config.toml\"`, the label of that file as \ + one string, and run this command again; or use `--mode vendored`", + ] { + assert!(why.contains(part), "no {part:?} in {why}"); + } + } + + #[test] + fn a_hub_that_names_the_config_as_one_string_is_not_in_the_way() { + for label in [ + "//:.cargo/config.toml", + ":.cargo/config.toml", + ".cargo/config.toml", + "@//:.cargo/config.toml", + "@@//:.cargo/config.toml", + // The name the root module has for itself. + "@shop//:.cargo/config.toml", + ] { + let named = format!("cargo_config = \"{label}\""); + assert_eq!(of_hub(&readme_with(&[&named])), None, "{label}"); + } + // The legacy file, when that is the one the rewriter writes. + let files = [( + "MODULE.bazel", + &*module(&hub(&readme_with(&["cargo_config = \"//:.cargo/config\""]))), + )]; + let view = |path: &str| match files.iter().find(|(name, _)| *name == path) { + Some((_, text)) => Entry::Text(text.to_string()), + None => Entry::Absent, + }; + assert_eq!(judge(&view, "", &project_files(), ".cargo/config"), None); + let other = judge(&view, "", &project_files(), CONFIG).expect("another file"); + assert!(other.contains("its `cargo_config` is not the label of .cargo/config.toml")); + } + + #[test] + fn a_config_that_is_another_file_or_cannot_be_read_is_not_taken_for_the_right_one() { + for config in [ + // Another file, another repository, and no label at all. + "cargo_config = \"//bazel:cargo.toml\"", + "cargo_config = \"@other//:.cargo/config.toml\"", + "cargo_config = \"//:.cargo/../config.toml\"", + // Not one string. + "cargo_config = CONFIG", + "cargo_config = \"//:.cargo/\" + \"config.toml\"", + "cargo_config = [\"//:.cargo/config.toml\"]", + "**settings", + ] { + let why = of_hub(&readme_with(&[config])).unwrap_or_else(|| panic!("{config}")); + let said = "reads the Cargo files of this project, and its `cargo_config` is not the \ + label of .cargo/config.toml as one string (the settings of another file \ + it names belong in .cargo/config.toml). "; + assert!(why.contains(said), "{config}: {why}"); + } + } + + #[test] + fn a_hub_concerns_the_project_whose_files_it_names_and_no_other() { + let lock = "cargo_lockfile = \"//:Cargo.lock\""; + let elsewhere = "cargo_lockfile = \"//third_party:Cargo.lock\""; + // Each file the rewriter writes, named alone: the lock, the root + // manifest, a member's, and the lock by the root module's own name. + for attributes in [ + vec![lock, "manifests = []"], + vec![elsewhere, "manifests = [\"//:Cargo.toml\"]"], + vec![ + elsewhere, + "manifests = [\"//x:Cargo.toml\", \"//member:Cargo.toml\"]", + ], + vec![elsewhere, "manifests = [\"//:member/Cargo.toml\"]"], + vec!["cargo_lockfile = \"@shop//:Cargo.lock\""], + vec!["manifests = [\"Cargo.toml\"]"], + ] { + let why = of_hub(&attributes).unwrap_or_else(|| panic!("{attributes:?}")); + assert!( + why.contains(":4 reads the Cargo files"), + "{attributes:?}: {why}" + ); + } + // Other files, the same names in another repository, and a hub + // that names no file. + for attributes in [ + vec![elsewhere, "manifests = [\"//third_party:Cargo.toml\"]"], + vec![ + elsewhere, + "manifests = [\"//other:Cargo.toml\", \"//:Cargo.Bazel.toml\"]", + ], + vec![ + "cargo_lockfile = \"@dep//:Cargo.lock\"", + "manifests = [\"@dep//:Cargo.toml\"]", + ], + vec!["cargo_lockfile = \"@@dep+//:Cargo.lock\""], + vec!["name = \"crates\""], + ] { + assert_eq!(of_hub(&attributes), None, "{attributes:?}"); + } + // The same hub, seen from each project of the workspace: the one + // in the directory it names, and its neighbours. + let nested = hub(&[ + "cargo_lockfile = \"//svc/app:Cargo.lock\"", + "manifests = [\"//:svc/app/Cargo.toml\"]", + ]); + let files = [("MODULE.bazel", &*module(&nested))]; + let why = said(&files, "svc/app").expect("the project of the hub"); + assert!(why.contains("give the hub `cargo_config = \"//svc/app:.cargo/config.toml\"`")); + assert!( + why.contains("Commit svc/app/.cargo/config.toml (where"), + "{why}" + ); + assert!( + why.contains("without svc/app/.cargo/config.toml, where"), + "{why}" + ); + for project in ["", "svc", "svc/app/tool", "svc/application"] { + assert_eq!(said(&files, project), None, "{project:?}"); + } + let right = nested.replace( + ")", + " cargo_config = \"//svc/app:.cargo/config.toml\",\n)", + ); + assert_eq!(said(&[("MODULE.bazel", &*module(&right))], "svc/app"), None); + let roots = nested.replace(")", " cargo_config = \"//:.cargo/config.toml\",\n)"); + assert!(said(&[("MODULE.bazel", &*module(&roots))], "svc/app").is_some()); + } + + #[test] + fn a_hub_whose_files_are_not_written_out_may_read_the_project() { + for attributes in [ + vec!["cargo_lockfile = LOCK", "manifests = [\"//x:Cargo.toml\"]"], + vec![ + "cargo_lockfile = \"//x:Cargo.lock\"", + "manifests = MANIFESTS", + ], + vec!["manifests = [\"//x:Cargo.toml\"] + MORE"], + vec!["manifests = [\"//x:Cargo.toml\", MEMBER]"], + vec!["**hub"], + ] { + let why = of_hub(&attributes).unwrap_or_else(|| panic!("{attributes:?}")); + let said = ":4 may read the Cargo files of this project (its `cargo_lockfile` or \ + its `manifests` is not written out), and "; + assert!(why.contains(said), "{attributes:?}: {why}"); + // With the config named as one string it is not in the way. + let mut named = attributes.clone(); + named.push("cargo_config = \"//:.cargo/config.toml\""); + assert_eq!(of_hub(&named), None, "{named:?}"); + } + } + + #[test] + fn every_call_that_makes_a_hub_counts() { + let readme = hub(&README); + let right = hub(&readme_with(&["cargo_config = \"//:.cargo/config.toml\""])); + let in_the_way = |module: &str, at: &str| { + let why = said(&[("MODULE.bazel", module)], ""); + let why = why.unwrap_or_else(|| panic!("{module}")); + assert!( + why.starts_with(&format!("the rules_rust hub at {at} reads")), + "{why}" + ); + }; + // The second of two hubs, one in an included file, one that may + // not run, and one on a proxy of a dev dependency. + in_the_way(&module(&format!("{right}\n{readme}")), "MODULE.bazel:10"); + let included = format!( + "{}include(\"//rust:crates.MODULE.bazel\")\n", + module(&right) + ); + let part = module(&readme).replace("module(name = \"shop\", version = \"0.0.0\")\n", ""); + let files = [ + ("MODULE.bazel", &*included), + ("rust/crates.MODULE.bazel", &*part), + ]; + let why = said(&files, "").expect("the included hub"); + assert!( + why.starts_with("the rules_rust hub at rust/crates.MODULE.bazel:3 reads"), + "{why}" + ); + in_the_way(&module(&format!("ON and {readme}")), "MODULE.bazel:4"); + let dev = module(&readme).replace("\"crate\")", "\"crate\", dev_dependency = True)"); + in_the_way(&dev, "MODULE.bazel:4"); + // The file that hands the extension on, and another name for the + // ruleset's repository. + in_the_way( + &module(&readme).replace("extensions.bzl", "extension.bzl"), + "MODULE.bazel:4", + ); + let renamed = module(&readme) + .replace( + "version = \"0.75.0\")", + "version = \"0.75.0\", repo_name = \"rr\")", + ) + .replace("@rules_rust//", "@rr//"); + in_the_way(&renamed, "MODULE.bazel:4"); + // Both hubs name the config: none is in the way. + assert_eq!( + said( + &[("MODULE.bazel", &module(&format!("{right}\n{right}")))], + "" + ), + None + ); + } + + #[test] + fn a_tag_of_another_extension_is_no_hub() { + // rules_rs has a tag of the same name, which reads the config of + // the checkout itself; so may any other module's. + let rules_rs = "bazel_dep(name = \"rules_rs\", version = \"0.2.2\")\n\ + crate = use_extension(\"@rules_rs//rs:extensions.bzl\", \"crate\")\n\ + crate.from_cargo(\n name = \"crates\",\n cargo_lock = \ + \"//:Cargo.lock\",\n cargo_toml = \"//:Cargo.toml\",\n)\n"; + assert_eq!(said(&[("MODULE.bazel", rules_rs)], ""), None); + for other in [ + module(&hub(&README)).replace("@rules_rust//", "@rules_other//"), + module(&hub(&README)).replace("\"crate\")", "\"other\")"), + module(&hub(&README)) + .replace("//crate_universe:extensions.bzl", "//rust:extensions.bzl"), + module(&hub(&README)).replace("crate.from_cargo", "crate.from_specs"), + module("crate.spec(package = \"cfg-if\", version = \"1.0.0\")"), + ] { + assert_eq!(said(&[("MODULE.bazel", &other)], ""), None, "{other}"); + } + } + + #[test] + fn a_use_of_the_extension_that_may_hide_a_call_is_in_the_way() { + let right = hub(&readme_with(&["cargo_config = \"//:.cargo/config.toml\""])); + for (hidden, doubt) in [ + ( + "make = crate.from_cargo", + "`crate` is used other than to call a tag", + ), + ( + "crate.from_cargo.again()", + "`crate.from_cargo.again` is no call of a tag", + ), + ] { + let why = said( + &[("MODULE.bazel", &module(&format!("{right}\n{hidden}")))], + "", + ); + let why = why.unwrap_or_else(|| panic!("{hidden}")); + let said = format!( + "MODULE.bazel:10: {doubt}, so a rules_rust hub that socket-patch cannot read may \ + take the Cargo files of this project. " + ); + assert!(why.starts_with(&said), "{hidden}: {why}"); + // The hub that is seen names the config already, and there may + // be no other: the way on is that line, not an attribute. + let way = "Write that line as the call of a tag (`crate.from_cargo(...)`) or as \ + `use_repo(...)` and run this command again: it then pins the crate, or \ + names the hub that is in the way; or use `--mode vendored`"; + assert!(why.contains(way), "{hidden}: {why}"); + assert!(!why.contains("ive the hub"), "{hidden}: {why}"); + } + } + + #[test] + fn nothing_is_said_where_no_hub_is_seen() { + // No file, a workspace that asks for the extension and makes no + // hub, and files that cannot be read where no hub is seen. + assert_eq!(said(&[], ""), None); + assert_eq!(said(&[("MODULE.bazel", &module(""))], ""), None); + assert_eq!(said(&[("REPO.bazel", "")], ""), None); + let doubtful = "module(name = \"shop\")\ninclude(\"//rust:missing.MODULE.bazel\")\n\ + use_extension(NAME, \"crate\")\n"; + assert_eq!(said(&[("MODULE.bazel", doubtful)], ""), None); + } + + #[test] + fn a_workspace_file_counts_where_bazel_may_read_it() { + let call = |more: &str| { + format!( + "workspace(name = \"shop\")\nload(\"@rules_rust//crate_universe:defs.bzl\", \ + \"crates_repository\")\ncrates_repository(\n name = \"crates\",\n \ + cargo_lockfile = \"//:Cargo.lock\",\n manifests = [\"//:Cargo.toml\"],\n{more})\n" + ) + }; + let (readme, right) = ( + call(""), + call(" cargo_config = \"//:.cargo/config.toml\",\n"), + ); + for name in ["WORKSPACE", "WORKSPACE.bazel", "WORKSPACE.bzlmod"] { + let why = said(&[(name, &readme)], "").unwrap_or_else(|| panic!("{name}")); + let said_ = format!("the rules_rust hub at {name}:3 reads the Cargo files"); + assert!(why.starts_with(&said_), "{why}"); + assert_eq!(said(&[(name, &right)], ""), None, "{name}"); + } + // Bazel 7 and 8 may read the file, an unnamed release may be one of + // them, and Bazel 9 never reads it. + for (release, read) in [ + ("7.7.2", true), + ("8.8.2", true), + ("9.3.2", false), + ("9.x", true), + ] { + let files = [("WORKSPACE", &*readme), (".bazelversion", release)]; + assert_eq!(said(&files, "").is_some(), read, "{release}"); + } + // Beside module files that make a hub the file is what a move to + // Bzlmod left behind; beside ones that make none it counts. + let hub_ = hub(&readme_with(&["cargo_config = \"//:.cargo/config.toml\""])); + let files = [("MODULE.bazel", &*module(&hub_)), ("WORKSPACE", &*readme)]; + assert_eq!(said(&files, ""), None); + let files = [("MODULE.bazel", &*module("")), ("WORKSPACE", &*readme)]; + assert!(said(&files, "").is_some()); + // `@shop//` is the main repository there, by the name the file + // gives it; under a name that is not written out any `@name//` may + // be. Another repository's files are not the project's. + let own = readme.replace("\"//:", "\"@shop//:"); + let why = said(&[("WORKSPACE", &own)], "").expect("the workspace's own name"); + assert!( + why.starts_with("the rules_rust hub at WORKSPACE:3 reads"), + "{why}" + ); + let own_config = call(" cargo_config = \"@shop//:.cargo/config.toml\",\n"); + assert_eq!(said(&[("WORKSPACE", &own_config)], ""), None); + let other = readme.replace("\"//:", "\"@other//:"); + assert_eq!(said(&[("WORKSPACE", &other)], ""), None); + let unnamed = other.replace("name = \"shop\"", "name = NAME"); + assert!(said(&[("WORKSPACE", &unnamed)], "").is_some()); + // Not seen: the rule under another name, and a file that only + // loads the `.bzl` file that calls it. + let renamed = readme.replace("crates_repository(", "crates("); + assert_eq!(said(&[("WORKSPACE", &renamed)], ""), None); + let loads = "load(\"//:deps.bzl\", \"rust_deps\")\nrust_deps()\n"; + assert_eq!( + said(&[("WORKSPACE", loads), ("deps.bzl", &readme)], ""), + None + ); + } + + /// On disk: the project's own directory, one below the Bazel root of + /// its checkout, and a project in no Bazel workspace, for which + /// nothing is read. + #[test] + fn the_workspace_of_a_project_is_read_from_its_checkout() { + let tmp = tempfile::tempdir().unwrap(); + let base = std::fs::canonicalize(tmp.path()).unwrap(); + let write = |rel: &str, text: &str| { + let path = base.join(rel); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(path, text).unwrap(); + }; + write("repo/.git/HEAD", "ref: refs/heads/main\n"); + write("repo/svc/app/Cargo.toml", ""); + let nested = hub(&[ + "cargo_lockfile = \"//svc/app:Cargo.lock\"", + "manifests = [\"//svc/app:Cargo.toml\"]", + ]); + let (repo, app) = (base.join("repo"), base.join("repo/svc/app")); + assert_eq!( + unread(&app, &project_files(), CONFIG), + None, + "no workspace file" + ); + write("repo/MODULE.bazel", &module(&nested)); + let why = unread(&app, &project_files(), CONFIG).expect("the project of the hub"); + assert!( + why.starts_with("the rules_rust hub at MODULE.bazel:4 reads"), + "{why}" + ); + assert_eq!( + unread(&repo, &project_files(), CONFIG), + None, + "the root is another project" + ); + assert_eq!(unread(&repo.join("svc"), &project_files(), CONFIG), None); + // A workspace file above the checkout is not this project's. + write("MODULE.bazel", &module(&hub(&README))); + std::fs::remove_file(repo.join("MODULE.bazel")).unwrap(); + assert_eq!( + unread(&repo, &project_files(), CONFIG), + None, + "above the checkout" + ); + } +} diff --git a/crates/socket-patch-core/src/bazel/mod.rs b/crates/socket-patch-core/src/bazel/mod.rs index f2d19a28d..665a07a83 100644 --- a/crates/socket-patch-core/src/bazel/mod.rs +++ b/crates/socket-patch-core/src/bazel/mod.rs @@ -9,6 +9,7 @@ pub const WORKSPACE_FILES: [&str; 4] = ["MODULE.bazel", "REPO.bazel", "WORKSPACE", "WORKSPACE.bazel"]; pub mod built; +pub mod cargo; pub mod ignore; pub mod maven; pub mod rc; diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 09b507b2e..cc2302570 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -1198,12 +1198,30 @@ pub async fn read_candidate_files_in( // dependencies of their own: a member's direct `cfg-if = "1"` must be // pinned alongside the root's, or the redirected lock entry is // unsatisfiable. Keyed `/Cargo.toml` for the cargo rewriter. - if out.files.contains_key("Cargo.toml") && candidates.iter().any(|c| c.dep.ecosystem == "cargo") - { + let cargo = candidates.iter().any(|c| c.dep.ecosystem == "cargo"); + if out.files.contains_key("Cargo.toml") && cargo { for rel in crate::utils::cargo_workspace::member_manifests_in(view).await { out.read(view, unreadable, &rel).await; } } + // In a Bazel workspace on disk a rules_rust hub may run cargo over + // these files without the cargo config that defines the registry of a + // pin. What stands against a pin is read from the checkout here and + // rides a synthetic key to the cargo rewriter. A project in no Bazel + // workspace, or one that no such hub is seen to read, gets none, and + // its run is the one it was. + if let Some(root) = view.disk_root().filter(|_| cargo) { + use crate::bazel::cargo::{unread, KEY}; + let member = |rel: &&String| crate::patch::redirect::is_cargo_member_manifest_key(rel); + let mut files = vec!["Cargo.lock".to_string(), "Cargo.toml".to_string()]; + files.extend(out.files.keys().filter(member).cloned()); + let legacy = out.files.contains_key(".cargo/config"); + let config = [".cargo/config.toml", ".cargo/config"][usize::from(legacy)]; + let root = root.to_path_buf(); + let read = crate::utils::fs::run_blocking(move || unread(&root, &files, config)); + out.files + .extend(read.await.map(|why| (KEY.to_string(), why))); + } // Go: the workspace in effect decides which version a replace must pin, // whether it takes effect at all, and whether it belongs in go.work. @@ -6186,6 +6204,83 @@ snapshots: } } + /// In a Bazel workspace on disk the read of a run with a cargo + /// candidate hands the rewriter what stands against a pin: the hub + /// that reads the project's manifests, a member's among them, and not + /// the config the rewriter writes, which is the legacy file where the + /// project has one. Nothing is handed over in no Bazel workspace, for a + /// hub that names that config, or to a run without a cargo candidate. + #[tokio::test] + async fn a_bazel_hub_without_the_cargo_config_rides_to_the_cargo_rewriter() { + use crate::bazel::cargo::KEY; + let tmp = tempfile::tempdir().unwrap(); + let write = |rel: &str, text: &str| { + let path = tmp.path().join(rel); + std::fs::create_dir_all(path.parent().unwrap()).unwrap(); + std::fs::write(path, text).unwrap(); + }; + let hub = |attributes: &str| { + format!( + "bazel_dep(name = \"rules_rust\", version = \"0.75.0\")\ncrate = use_extension(\ + \"@rules_rust//crate_universe:extensions.bzl\", \"crate\")\n\ + crate.from_cargo({attributes})\n" + ) + }; + write("Cargo.toml", "[workspace]\nmembers = [\"member\"]\n"); + write( + "member/Cargo.toml", + "[package]\nname = \"member\"\nversion = \"0.1.0\"\n", + ); + let cargo = Candidate { + purl: "pkg:cargo/serde@1.0.190".into(), + dep: grant_dep("cargo", "https://patch.test/serde-1.0.190.crate".into()), + }; + let view = ProjectView::Disk(tmp.path()); + let handed = |candidate: &Candidate| { + let candidates = [candidate.clone()]; + async move { + let read = read_candidate_files(&view, &BTreeSet::new(), &candidates).await; + read.files.get(KEY).cloned() + } + }; + assert_eq!(handed(&cargo).await, None, "no Bazel workspace"); + + write( + "MODULE.bazel", + &hub("manifests = [\"//member:Cargo.toml\"]"), + ); + let why = handed(&cargo).await.expect("the hub reads a member"); + assert!( + why.starts_with("the rules_rust hub at MODULE.bazel:3 reads"), + "{why}" + ); + assert!(why.contains("without .cargo/config.toml, where"), "{why}"); + + let root = "manifests = [\"//:Cargo.toml\"]"; + write("MODULE.bazel", &hub(root)); + assert!(handed(&cargo).await.is_some(), "the hub reads the root"); + let lock = "cargo_lockfile = \"//:Cargo.lock\", manifests = [\"//x:Cargo.toml\"]"; + write("MODULE.bazel", &hub(lock)); + assert!(handed(&cargo).await.is_some(), "the hub reads the lock"); + write("MODULE.bazel", &hub(root)); + assert_eq!( + handed(&left_pad_candidate()).await, + None, + "no cargo candidate" + ); + let named = format!("{root}, cargo_config = \"//:.cargo/config.toml\""); + write("MODULE.bazel", &hub(&named)); + assert_eq!(handed(&cargo).await, None, "the hub names the config"); + write(".cargo/config", ""); + let why = handed(&cargo) + .await + .expect("the rewriter writes the legacy file"); + assert!( + why.contains("is not the label of .cargo/config as one string"), + "{why}" + ); + } + fn gem_candidate() -> Candidate { use crate::patch::redirect::{Integrity, RegistryOverride, RegistryOverrideIdentifiers}; Candidate { diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 6bcc0ed48..dbd80d36e 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -2434,6 +2434,21 @@ fn rewrite_cargo_with( // when a healthy block is already wired in). let config_plan = plan_cargo_config(&cargo_config, cargo_config_key, ®, index_url); + // In a Bazel workspace, a rules_rust hub that reads these files + // and not the config would fail on the pin, and `bazel build` with + // it ([`crate::bazel::cargo`]): nothing is written for the crate. + // A pin that is in place already stays what it was. + let writes = config_plan.is_some() + || toml_plans.iter().any(|(_, plan)| plan.changed) + || matches!(lock_commit, LockCommit::Write(_)); + if let Some(why) = files.get(crate::bazel::cargo::KEY).filter(|_| writes) { + result.warnings.push(RewriteWarning { + code: crate::bazel::cargo::UNREAD.into(), + detail: format!("{}@{}: {why}", dep.name, dep.version), + }); + continue; + } + // COMMIT — everything planned, nothing can fail past this point, so // the three files change together or not at all. Edit order matches // the historical ledger order: config, manifest, lock. @@ -19719,6 +19734,82 @@ mod tests { assert!(r.confirmed_cargo_uuids.contains(CARGO_UUID)); } + /// In a Bazel workspace whose rules_rust hub would not read the config + /// that defines the pin's registry, the engine hands the reason in + /// under [`crate::bazel::cargo::KEY`]: a crate the rewriter would pin + /// is skipped with it and nothing is written. A refusal of the + /// rewriter's own answers first, under its own code, and a pin that an + /// earlier run left in place stays confirmed, with no warning and no + /// write, exactly as without the key. + #[test] + fn cargo_pin_is_withheld_for_a_bazel_hub_that_does_not_read_the_config() { + use crate::bazel::cargo::{KEY, UNREAD}; + let toml = "[package]\nname = \"app\"\nversion = \"0.1.0\"\n\n\ + [dependencies]\nserde = \"1.0.190\"\n"; + let plain = cargo_files(toml); + let pinned = rewrite_registry_redirect(&plain, &[cargo_sparse_override()]); + assert_eq!(pinned.files.len(), 3, "{:?}", pinned.files.keys()); + let codes = |r: &RewriteResult| -> Vec { + r.warnings.iter().map(|w| w.code.clone()).collect() + }; + + let mut hub = plain.clone(); + hub.insert( + KEY.to_string(), + "the hub at MODULE.bazel:4 is in the way".into(), + ); + let r = rewrite_registry_redirect(&hub, &[cargo_sparse_override()]); + assert!(r.files.is_empty() && r.edits.is_empty(), "{:?}", r.files); + assert!(r.confirmed_cargo_uuids.is_empty()); + assert_eq!(codes(&r), [UNREAD]); + let detail = &r.warnings[0].detail; + assert_eq!( + detail, + "serde@1.0.190: the hub at MODULE.bazel:4 is in the way" + ); + + // Each file the rewriter would write is enough to withhold: a + // project without a lock (config and manifest), a pin whose + // registry the config no longer defines (config), and a config + // that defines it already (manifest and lock). Without the key + // each is written. + let config = ".cargo/config.toml"; + let (mut lockless, mut undefined, mut defined) = (hub.clone(), hub.clone(), hub.clone()); + lockless.remove("Cargo.lock"); + undefined.extend(pinned.files.clone()); + undefined.remove(config); + defined.insert(config.to_string(), pinned.files[config].clone()); + for mut files in [lockless, undefined, defined] { + let r = rewrite_registry_redirect(&files, &[cargo_sparse_override()]); + assert!(r.files.is_empty() && r.edits.is_empty(), "{:?}", r.files); + assert!(r.confirmed_cargo_uuids.is_empty()); + assert_eq!(codes(&r), [UNREAD]); + files.remove(KEY); + let r = rewrite_registry_redirect(&files, &[cargo_sparse_override()]); + assert!(!r.files.is_empty() && r.warnings.is_empty(), "{files:?}"); + } + + // The rewriter's own refusal: the crate is in the lock and in no + // manifest. + let mut undeclared = hub.clone(); + let other = toml.replace("serde = \"1.0.190\"", "itoa = \"1\""); + undeclared.insert("Cargo.toml".to_string(), other); + let r = rewrite_registry_redirect(&undeclared, &[cargo_sparse_override()]); + assert_eq!(codes(&r), ["redirect_cargo_toml_dep_not_found"]); + + // The three files as the earlier run left them. + let mut again = hub.clone(); + again.extend(pinned.files.clone()); + let with = rewrite_registry_redirect(&again, &[cargo_sparse_override()]); + again.remove(KEY); + let without = rewrite_registry_redirect(&again, &[cargo_sparse_override()]); + for r in [&with, &without] { + assert!(r.files.is_empty() && r.edits.is_empty(), "{:?}", r.files); + assert!(r.warnings.is_empty(), "{:?}", r.warnings); + assert!(r.confirmed_cargo_uuids.contains(CARGO_UUID)); + } + } + /// #455 review: the offline source may come from outside the project /// files (a monorepo root's `.cargo/config.toml`, `$CARGO_HOME`'s), or /// be a `cargo local-registry`; both are refused like a project diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 43dcb2fd9..9452b369e 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -16,7 +16,7 @@ The backticked slug in each row is the value `-e`/`--ecosystems` accepts (e.g. |-----------|------------------------|------------------------------|--------------------------| | npm (`npm`) — pnpm / yarn / berry / bun / vlt | ✅ any install layout, vlt's `node_modules/.vlt` store included (every store copy, copy-on-write) | ✅ seven lockfile flavors: package-lock, yarn classic, yarn berry (node-modules linker; PnP refused), pnpm v9, pnpm legacy v5.4/v6.0 (`pnpm 7/8` — frozen installs are path-bound because those majors absolutize `file:` override specifiers; moved checkouts run one `pnpm install --offline --no-frozen-lockfile`, surfaced as `vendor_pnpm_legacy_absolute_specifier`), bun text `bun.lock` lockfileVersion 0/1/2/3 and native binary `bun.lockb` revisions 1/2/3 (binary locks stay binary; text workspace vendoring requires lockfileVersion 2 — see [Bun compatibility](testing/bun-compatibility.md)), vlt `vlt-lock.json` lockfileVersion 0/1 (patched package directories for direct dependencies of the root or a workspace member; transitive targets refused — see [vlt notes](#npm-vlt-notes)). Rush monorepos on pnpm 9 and newer are wired through `pnpm-config.json` and each directly-depending project's `package.json` — see [Rush notes](#npm-rush-monorepos) | ✅ package-lock / npm-shrinkwrap, pnpm-lock.yaml and legacy shrinkwrap.yaml (pnpm majors 1–12; block and flow resolutions), yarn classic, yarn berry, bun, vlt (`vlt-lock.json` without `lockfileVersion`, 0 or 1) — pnpm, berry, bun and vlt carry constraints, see [npm hosted-mode notes](#npm-hosted-mode-notes) and [vlt notes](#npm-vlt-notes) | | PyPI (`pypi`) — uv / poetry / pdm / pipenv / pip | ✅ in place | ✅ uv project/script locks, PEP 751 `pylock.toml` / `pylock..toml`, poetry, pdm, pipenv (Pipenv 2018 or later — every `Pipfile.lock` category is rewired, lock-only checkouts included; Pipenv 2023+ does not hash-check local wheels — `vendor_integrity_unverified`; Pipenv 2023.7.9+ exports the vendored wheel without a hash, so `pipenv requirements --hash` output fails pip's hash-checking mode — `pypi_pipenv_hash_export_unsupported`; a venv still holding the upstream release is reported as `pypi_pipenv_stale_install`; see [Pipenv compatibility](testing/pipenv-compatibility.md)), and requirements.txt. Native uv vendoring requires uv ≥ 0.2.35 (the `[[package]]` lock grammar); hosted mode covers native `uv.lock` from uv 0.1.45 (the first release whose `uv lock` writes one) and requirements from uv 0.0.5; see [uv compatibility](testing/uv-compatibility.md). | ✅ requirements.txt including hash continuations, uv project/script locks, and PEP 751 locks. Version/source ambiguity is refused; see [uv compatibility](testing/uv-compatibility.md). Poetry 1.x and 2.x locks are supported; Poetry 0.x ignores URL sources and is refused. See [Poetry compatibility](testing/poetry-compatibility.md). Pipenv `Pipfile.lock` (pipfile-spec 6 — Pipenv 7 and later; `path` references for 7–11, `file` from 2018; lock-only checkouts and Pipenv's out-of-tree venv are discovered; a warm venv that Pipenv will not reinstall over warns `redirect_pypi_stale_install`; see [Pipenv compatibility](testing/pipenv-compatibility.md)). `pdm.lock` is supported for the lock formats PDM 0.12–1.4 and 2.8.1+ write (`lock_version` 2 / 4.3–4.5.1); the identity-losing 3.1 / 4.0–4.2 formats (PDM 1.8–2.7) are refused. PDM 2.8.0 writes an indistinguishable `4.3` lock but shares that identity-loss bug, so a rewritten 2.8.0 lock crashes `pdm sync` — upgrade to ≥ 2.8.1. See [PDM compatibility](testing/pdm-compatibility.md). | -| Cargo (`cargo`) | ✅ in-place + `.cargo-checksum.json` rewrite (shared registry-cache caveat — see [Cargo: shared registry cache](#cargo-shared-registry-cache)) | ✅ `[patch.crates-io]` path entry in the root `Cargo.toml` (v5; per-version Socket keys; pre-v5 `.cargo/config*` wiring migrates on re-run) | ✅ per-patch sparse registry (`[registries.socket-patch-]` + Cargo.lock source/checksum); needs cargo 1.68+ (sparse registries) — a project whose `rust-version` or `rust-toolchain[.toml]` declares an older cargo is refused (`redirect_cargo_old_cargo`), use `--mode vendored`; direct dependencies only — a crate another dependency also pulls in is refused, use `--mode vendored`; a crate the user overrides with `[patch]` (path, git or another registry) is refused too, the override is left alone; a `cargo vendor` or `cargo local-registry` project (crates.io replaced by an on-disk source in the project's, a parent directory's or `$CARGO_HOME`'s `.cargo/config*`) is refused, use agent mode; with no `Cargo.lock` the graph is unknown, so only a project whose sole dependency is the patched crate is redirected | +| Cargo (`cargo`) | ✅ in-place + `.cargo-checksum.json` rewrite (shared registry-cache caveat — see [Cargo: shared registry cache](#cargo-shared-registry-cache)) | ✅ `[patch.crates-io]` path entry in the root `Cargo.toml` (v5; per-version Socket keys; pre-v5 `.cargo/config*` wiring migrates on re-run) | ✅ per-patch sparse registry (`[registries.socket-patch-]` + Cargo.lock source/checksum); needs cargo 1.68+ (sparse registries) — a project whose `rust-version` or `rust-toolchain[.toml]` declares an older cargo is refused (`redirect_cargo_old_cargo`), use `--mode vendored`; direct dependencies only — a crate another dependency also pulls in is refused, use `--mode vendored`; a crate the user overrides with `[patch]` (path, git or another registry) is refused too, the override is left alone; a `cargo vendor` or `cargo local-registry` project (crates.io replaced by an on-disk source in the project's, a parent directory's or `$CARGO_HOME`'s `.cargo/config*`) is refused, use agent mode; with no `Cargo.lock` the graph is unknown, so only a project whose sole dependency is the patched crate is redirected; in a Bazel workspace a crate is not pinned while a rules_rust hub reads the project's Cargo files without its cargo config (`redirect_cargo_bazel_config_unread`), see [Bazel workspaces: Cargo](#bazel-workspaces-cargo) | | RubyGems (`gem`) | ✅ in place | ✅ Gemfile + Gemfile.lock path pair (a CRLF `Gemfile.lock`, as a `core.autocrlf` checkout leaves it, is edited and reverted in its own line ending, and a lock mixing CRLF and LF fails closed; a gem locked as precompiled platform gems is vendored from prebuilt patched platform gems, see [Ruby gems: native extensions](#ruby-gems-native-extensions); `Gemfile` spelling only — a `gems.rb` twin, which bundler ≥ 2 loads instead, or a `BUNDLE_GEMFILE`-configured manifest makes vendoring refuse with `gemfile_not_loaded` before any write) | ✅ a `source:` option on the gem's own declaration, the lock converged to match (see [Ruby gems: hosted mode](#ruby-gems-hosted-mode)) — edits `gems.rb` + `gems.locked` or `Gemfile` + `Gemfile.lock`, whichever the project holds (a `Gemfile` + `gems.rb` twin is refused with `redirect_gem_twin_manifest_ambiguous`: bundler 1.x loads the `Gemfile`, bundler ≥ 2 loads `gems.rb`, and nothing in the project says which bundler installs it; bundler 4's `BUNDLE_LOCKFILE` (environment, app config or global config) naming any other lock is refused with `redirect_gem_bundle_lockfile_unsupported`, as is vendoring with `gemfile_not_loaded`; spellings that diverge beyond Socket's own edits fail closed with `redirect_gem_gemfile_spellings_diverge`; `BUNDLE_GEMFILE` from `.bundle/config` (which outranks the environment, as in bundler), the environment, or the global `~/.bundle/config` / `$BUNDLE_USER_CONFIG` (lowest, as in bundler) is followed when it names the project's `Gemfile` / `gems.rb`, and any other configured manifest is refused with `redirect_gem_bundle_gemfile_unsupported`); a Bundler all-source, exact-source or hostname mirror in app config or the scan environment can capture the patch registry, so the redirect is refused with `redirect_gem_mirror_overrides_source` without printing mirror URLs (scope mirrors to `mirror.https://rubygems.org`; user-global config and mirrors set only in a later install environment are not inspected); the `CHECKSUMS` pin needs bundler ≥ 2.6 (an older lock is still pinned by source and installs frozen; the run says once, with `redirect_gem_no_checksums_section`, that it cannot record the checksums); a stale pre-redirect materialization that `bundle install` would reuse instead of refetching is replaced with the verified patched gem when it is the project's own, else flagged `redirect_gem_stale_install` with the remedy (see "Ruby gems: gems already installed" below and CLI_CONTRACT.md's "Gem stale-install guard") | | Go (`golang`) | ✅ `go.mod` `replace` → `.socket/go-patches/` — see [Go: directory replaces and go.sum](#go-directory-replaces-and-gosum) | ✅ `replace` → the committed vendor tree | ✅ (free tier) fork-style `replace` → `patch.socket.dev/gopatch/` + committed `go.sum` pin; see [Go notes](#go-directory-replaces-and-gosum). Paid hosted patches are unsupported; `redirect_golang_unsupported` names the vendored remedy | | Maven (`maven`) — Maven and Gradle | ✅ in place in every copy the build consumes: each `~/.m2` copy it reads and each Gradle `files-2.1` copy; `~/.m2` `.sha1`/`.md5` sidecars are rewritten, Gradle copies get advisories; jar-member records swap in the patch service's whole jar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) and [Gradle](#gradle) | ✅ suffixed Maven repository (`-socket.` pin + `.mvn/maven.config` + fallback file repository) for every pom root, single-module or reactor, or Gradle 6.8+ same-GAV repository with settings wiring and SHA-256 checks (a root with both `pom.xml` and a Gradle build wires both); see [JVM vendoring](design/maven-vendoring.md) and [Gradle](#gradle) | ✅ fail-closed by a Socket-only `-socket.` suffix: pom projects get a pinned `` (`${property}` versions are refused); Gradle 6.8+ builds get an owned settings script, lock-entry rewrites and a resolution tripwire — see [Maven & NuGet caveats](#maven--nuget-caveats) and [Gradle](#gradle) | @@ -1838,6 +1838,128 @@ Limits: - `socket-patch vex` looks at the hub only for the block's entries: a patch that only the `pom.xml` pin or a patched jar attests is attested, although the Bazel build does not have it. +## Bazel workspaces: Cargo + +A crate_universe hub of rules_rust (`crate.from_cargo` in `MODULE.bazel`, `crates_repository` +in a `WORKSPACE` file) builds its crates from the `Cargo.toml` files and the `Cargo.lock` it +names. Its tool, cargo-bazel, links what stands beside each of those `Cargo.toml` files into a +directory of its own, writes a root manifest of its own there and runs cargo there. Of the +cargo configs of the checkout that cargo gets one: the file the hub names as `cargo_config`, +and none when the hub names none (rules_rust 0.56.0 and 0.75.0). The hub of the rules_rust +README names none. + +Hosted mode pins a crate in three files: it defines a registry in the project's +`.cargo/config.toml`, names it on the dependency in `Cargo.toml` +(`registry = "socket-patch-"`) and repoints the entry of `Cargo.lock`. On a hub with +`cargo_config = "//:.cargo/config.toml"` the next `bazel build` builds the patched crate, and +vendored mode builds patched there too (both tested on real Bazel 9.3.2 and 8.8.2 with +rules_rust 0.75.0 and on 7.7.2 with 0.56.0, for a hub without a cargo-bazel lock). + +A hub that reads the manifest and not that config fails on the pin, and every `bazel build` +that needs the hub fails with it: + +``` +error: failed to parse manifest at .../Cargo.toml +Caused by: + registry index was not found in any configuration: `socket-patch-` +``` + +So in a Bazel workspace `scan --mode hosted` and `get --mode hosted` write no pin for a crate +while such a hub is in the way. Nothing is written for the crate, it is `skipped` +(`redirect_unconfirmed`), the exit code is 0 as for the other crates hosted mode leaves out, +and one warning per crate says why (`redirect_cargo_bazel_config_unread`): + +``` +Warning: cfg-if@1.0.4: the rules_rust hub at MODULE.bazel:10 reads the Cargo files of this project, and it names no `cargo_config`. cargo-bazel hands the cargo it runs no cargo config but the one its hub names, and without .cargo/config.toml, where hosted mode defines the patch registry, that cargo fails on the pinned Cargo.toml (`registry index was not found in any configuration`) and `bazel build` with it; dependency skipped (nothing rewritten). Commit .cargo/config.toml (where the project has none, create it with a comment line in it: `rollback` deletes a config it leaves empty, and a hub that names a missing file fails), give the hub `cargo_config = "//:.cargo/config.toml"`, the label of that file as one string, and run this command again; or use `--mode vendored`, which commits the patched crate and needs no registry +``` + +The two ways on, both tested on the hub of the README on the three releases, from the refused +scan to the patched build: + +- Commit `.cargo/config.toml` and give the hub the attribute, then run the same command: it + pins the crate, and the build is the patched one. The file comes first. A project that has + none creates it with a comment line in it: a hub that names a missing file fails, and + `rollback` deletes a cargo config it leaves empty, which a comment line prevents. +- Use `--mode vendored`. It commits the patched crate under `.socket/vendor/cargo`, points a + `[patch.crates-io]` entry at it and needs no registry. + +Where the reason is a line that may hide a hub, the hub that is seen may name the config +already, and there may be no other. The warning then names that line and no attribute: + +``` +Warning: cfg-if@1.0.4: MODULE.bazel:17: `crate` is used other than to call a tag, so a rules_rust hub that socket-patch cannot read may take the Cargo files of this project. [...] dependency skipped (nothing rewritten). Write that line as the call of a tag (`crate.from_cargo(...)`) or as `use_repo(...)` and run this command again: it then pins the crate, or names the hub that is in the way; or use `--mode vendored`, which commits the patched crate and needs no registry +``` + +A vendored crate that a hosted scan would take over stays vendored and patched +(`redirect_takeover_kept_vendored`). A pin that an earlier run wrote stays as it is: the +command reports it applied, as before, and writes nothing. On such a hub that build is +failing already: give the hub the attribute, or `socket-patch rollback`. + +Everything is read from the checkout, and nothing is evaluated: `MODULE.bazel` and the files +it includes, for the `from_cargo` tags of the `crate` extension of rules_rust +(`//crate_universe:extensions.bzl`, and `extension.bzl`, which hands it on) under whatever +name the file gives the ruleset and the extension; the `WORKSPACE` file, for calls of +`crates_repository`; `.bazeliskrc` and `.bazelversion`, for the Bazel release. + +- A hub concerns the project when a label of its `cargo_lockfile` or its `manifests` is the + project's `Cargo.lock`, its `Cargo.toml` or the `Cargo.toml` of a workspace member below + it. `@//` is the checkout, and so is `@//` in a `WORKSPACE` + file whose `workspace(name = "")` says so (any `@name//`, where that name is not + written out as a string). A label of another repository is no file of the project. +- A hub that concerns the project is in the way unless its `cargo_config` is one string, the + label of the config hosted mode writes: the project's `.cargo/config.toml`, or its + `.cargo/config` where the project has that file. Another config is not enough, whatever it + holds: the registry is defined in the project's own, and the settings of the other file + belong there. A list of one label is not one string. +- A reading that is not certain withholds the pin where a hub, or a use of the extension + through a name bound to it, is seen: a patch that waits for one line of yours weighs less + than a build that fails. + - A `cargo_lockfile` or `manifests` that is not written out as a string or a list of strings + (a constant, a concatenation, an attribute left to `**`): the hub may read the project's + files. Its warning says "may read". + - A `cargo_config` that is not written out as one string is not taken for the right file. + - A call that is not a statement of its own (in a conditional, in a comprehension) counts: + it may run. + - A use of the extension that may hide a call (the name bound to it used for anything but + a tag or `use_repo`): a hub that cannot be read may take the project's files. The warning + names the line to rewrite. + - A `crates_repository` of the `WORKSPACE` file counts when the module files make no + `crate.from_cargo` hub, unless `.bazeliskrc` or `.bazelversion` names Bazel 9.0.0 or later, + which never reads that file. Bazel 7 reads it, and Bazel 8 when it is told to. So a + `WORKSPACE` file that a move to Bzlmod left behind can withhold a pin on Bazel 8 where the + module files make no such hub: give its call the attribute, or delete the call. +- Nothing changes where no hub is seen to concern the project: in no Bazel workspace (the rule + is under [Bazel workspaces: `.bazelignore`](#bazel-workspaces-bazelignore)), in a workspace + without a crate_universe hub, for a hub that names the config, for a hub whose labels are all + written out and name other files (another project of the checkout, another repository), and + for a tag of the same name of another ruleset. The run is then the one it was: the same + output, and the same files byte for byte. + +Limits: + +- Not seen, so that the run is the one it was and a hub without the config fails as before: + - a hub that another Bazel module declares; + - a `crates_repository` inside a `.bzl` file that the `WORKSPACE` file loads, or called + under another name; + - where no hub is seen: a tag called on `use_extension(...)` itself, with no name bound to + it; a `MODULE.bazel` that cannot be read (a link that leads out of the Bazel root, a file + past the reader's bounds); and what else the module files hide from every reading (a file + they include that cannot be read, a `use_extension` that is not written out); + - what Bazel is told by flags and rc files. None is read, in the checkout or outside it: + `--noenable_bzlmod` has Bazel read the `WORKSPACE` file in place of module files that make + a hub; + - a Bazel root above a project that is in no git checkout: there only the project's own + directory is looked at for a workspace file. +- Not read: a project the in-memory hosted engine rewrites. It is handed the project's files + and reads no Bazel file, so its run is the one it was. +- Not looked at: `crate.from_specs` hubs, which take no `Cargo.toml`, and `crates_vendor` + targets. +- A hub with a cargo-bazel lock (`lockfile`) asks for a repin after every change of + `Cargo.lock`, a pin of either mode included (`CARGO_BAZEL_REPIN=true bazel build ...`). That + is the ruleset's own rule, and this check does not look at it. +- `socket-patch vex` attests a hosted pin from the three files alone. It does not look at the + hub. + ## Cargo: shared registry cache Agent mode patches the crate in place wherever the crawler finds it. For a non-vendored diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index 6729c125c..a00e2b99a 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -4,7 +4,7 @@ "paths": [ "crates/socket-patch-core/src/bazel/" ], - "max": 2486 + "max": 2618 }, "lock-readers": { "paths": [ From 64c75e9e0a5eb3d313bda5beb6ffc80c7bbcc76b Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Fri, 20 Nov 2026 14:30:46 -0500 Subject: [PATCH 3/3] Close five gaps of the hosted Cargo withhold under Bazel A review of the withhold found five cases where a pin could still be written that a rules_rust hub fails on, or where the warning sent the user the wrong way. Each has a unit row that fails on the code as it was. - A `crates_repository` of the `WORKSPACE` file was looked at only when the module files made no `crate.from_cargo` hub at all. Bazel 7 reads both, so a workspace whose module hub is over another project's files and whose `WORKSPACE` hub reads this one got the pin. The file now counts when no hub of the module files concerns the project. - The rule was known by its own name alone. A `load` that gives it another (`load(..., crates = "crates_repository")`) is now followed. - A hub that names the right config was passed although the file held nothing. The pin is written into it, a later `rollback` deletes the file it leaves empty, and the hub then fails on the missing file: a build that passed before the scan. Such a hub is now in the way, and the warning asks for a comment line. A file that is not there yet is still created by the pin, as before. - The warning gave the label of the config in the project's directory (`//svc/app:.cargo/config.toml`). Where the hub names the project's files from another package (`//:svc/app/Cargo.toml`), that directory is no package and the label names nothing. The label is now written in the package of the hub's own labels. - Under a `workspace(name = ...)` that is not written out, any `@name//` label was taken for the checkout, also as `cargo_config`. That widens what a hub may read, which is safe, and what names the project's config, which is not: another repository's config passed. It now applies to the lock and the manifests only. The engine also had a third copy of the rule for which cargo config the rewriter writes. It now calls the rewriter's own, `cargo_config_key`. The hermetic test that held the `rollback` gap now holds both halves: a config that holds nothing gets no pin, and a config that the pin creates is still deleted by `rollback`. The `bazel` line budget follows the code, 2618 to 2665. Co-Authored-By: Claude Fable 5.2 --- crates/socket-patch-cli/CLI_CONTRACT.md | 2 +- .../tests/e2e_bazel_build/cargo.rs | 30 ++- crates/socket-patch-core/src/bazel/cargo.rs | 242 ++++++++++++++---- crates/socket-patch-core/src/hosted/engine.rs | 3 +- .../src/patch/redirect/mod.rs | 29 ++- docs/ecosystems.md | 38 ++- scripts/loc-budget.json | 2 +- 7 files changed, 266 insertions(+), 80 deletions(-) diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 1246aef20..42ae48296 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -2632,7 +2632,7 @@ command's integration tests check its output. | `patch_withdrawn` | `rolledBack` / `verified` (hosted event) | scan `--mode hosted`: the lockfile pinned a patch the server reports `withdrawn` or `not_found`; the pin was restored (dry run: would be) to its upstream registry entry. `reason` names the uuid and status. See "Withdrawn pins". | | `vendor_prebuilt_stub_invalid` | `failed` | RubyGems: the server stub lacks required attributes or is otherwise invalid; no local stub fallback is permitted. | | `bazelignore_not_updated` | run-level `warnings[]` | Go vendored and agent mode in a Bazel workspace (additive): the `.bazelignore` at the Bazel root, which keeps Bazel and gazelle out of the module copies under `.socket/vendor/golang` and `.socket/go-patches`, was not updated. It could not be read or written (a symbolic link or a directory in its place, a read-only directory, a read-only file). The detail names the line to add, or says that the lines of a copy that is gone can be deleted; the command's patch or undo stands and its exit code is unchanged. See docs/ecosystems.md "Bazel workspaces: `.bazelignore`". | -| `redirect_cargo_bazel_config_unread` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (additive): a rules_rust crate_universe hub (`crate.from_cargo` in `MODULE.bazel` or a file it includes, or `crates_repository` in the `WORKSPACE` file) reads the project's `Cargo.lock` or one of its `Cargo.toml` files, or may read them (its `cargo_lockfile` or `manifests` is not written out as strings, or a use of the extension may hide a call), and its `cargo_config` is not one string that is the label of the cargo config hosted mode writes (`.cargo/config.toml`, or the project's `.cargo/config`). cargo-bazel runs cargo with the config its hub names and no other, so the registry of a pin would be unknown to it and `bazel build` would fail (`registry index was not found in any configuration`). The crate is not pinned: nothing is written for it, its event is `skipped` / `redirect_unconfirmed` (a vendored crate that the scan would take over stays vendored: `skipped` with this code, and `redirect_takeover_kept_vendored`), and status and exit code are unchanged. One warning per crate; the detail names the hub's file and line, the config to commit (created with a comment line in it where the project has none), the attribute to give the hub (`cargo_config = "//:.cargo/config.toml"`), after which the same command pins the crate, and `--mode vendored`. For a use of the extension that may hide a call the detail names that line to rewrite and no attribute: the next run pins the crate or names the hub that is in the way. A `WORKSPACE` call counts only when the module files make no hub and the checkout names no Bazel 9 or later. The rewriter's own refusals answer first and keep their codes. A pin that is in place already stays `applied`, and no warning is given for it. Nothing is read or changed in no Bazel workspace, or where no such hub is seen to read the project's Cargo files. See docs/ecosystems.md "Bazel workspaces: Cargo". | +| `redirect_cargo_bazel_config_unread` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (additive): a rules_rust crate_universe hub (`crate.from_cargo` in `MODULE.bazel` or a file it includes, or `crates_repository` in the `WORKSPACE` file) reads the project's `Cargo.lock` or one of its `Cargo.toml` files, or may read them (its `cargo_lockfile` or `manifests` is not written out as strings, or a use of the extension may hide a call), and its `cargo_config` is not one string that is the label of the cargo config hosted mode writes (`.cargo/config.toml`, or the project's `.cargo/config`). cargo-bazel runs cargo with the config its hub names and no other, so the registry of a pin would be unknown to it and `bazel build` would fail (`registry index was not found in any configuration`). The crate is not pinned: nothing is written for it, its event is `skipped` / `redirect_unconfirmed` (a vendored crate that the scan would take over stays vendored: `skipped` with this code, and `redirect_takeover_kept_vendored`), and status and exit code are unchanged. One warning per crate; the detail names the hub's file and line, the config to commit (created with a comment line in it where the project has none), the attribute to give the hub (`cargo_config = "//:.cargo/config.toml"`; the label is written in the package of the hub's own labels), after which the same command pins the crate, and `--mode vendored`. A hub that names that config is still in the way while the file is there and holds nothing: `rollback` deletes a cargo config it leaves empty, and the hub fails on the missing file; the detail asks for a comment line in it. For a use of the extension that may hide a call the detail names that line to rewrite and no attribute: the next run pins the crate or names the hub that is in the way. A `WORKSPACE` call (`crates_repository`, also under the name a `load` of that file gives it) counts only when no hub of the module files reads the project's Cargo files and the checkout names no Bazel 9 or later. The rewriter's own refusals answer first and keep their codes. A pin that is in place already stays `applied`, and no warning is given for it. Nothing is read or changed in no Bazel workspace, or where no such hub is seen to read the project's Cargo files. See docs/ecosystems.md "Bazel workspaces: Cargo". | | `maven_bazel_hub_unpatched` | run-level `warnings[]` (human: `Warning: …`) | Maven in a Bazel workspace, vendored and agent mode (additive): a pinned rules_jvm_external hub of the root `MODULE.bazel` fetches the jar of a Maven patch that this run vendored or applied, or tried to (`vendor`, `apply`, and `get` / `scan` in those modes; in agent mode only for a package it found installed; never under `--global`), and the block that hosted mode keeps in `MODULE.bazel` holds no entry for that jar. The hub builds the jar its own lock (`maven_install.json`) names, so the patch does not reach the Bazel build: these modes patch no hub. One warning per purl; the detail names the lock, the hub, the file and line of its `maven.install`, and the hosted command that patches a pinned hub. The events, the exit code and every written file are those of the same run without the workspace. A `vendor` whose `--ecosystems` leaves Maven out gives none. A vendored run that takes a hosted pin over (`vendor`, `scan` / `get --mode vendored`, `vendor --revert`) leaves the block's entry for it as it is. A hosted run never gives it: see the `maven_bazel_*` rows below and docs/ecosystems.md "Bazel workspaces: Maven". | | `maven_bazel_unreadable` | vendored / agent: run-level `warnings[]` (human: `Warning: …`); scan/get `--mode hosted`: `warnings[]` (warning) | Maven in a Bazel workspace (additive): whether a rules_jvm_external hub builds the jar of a Maven patch could not be told, because something could not be read with certainty: a hub without `lock_file` in vendored and agent mode (hosted mode gives `maven_bazel_not_pinned`), a `maven.artifact` or `maven.from_toml` tag that cannot be read, a lock that is missing, empty, not format 2 or 3, larger than 16 MiB or no JSON, a `WORKSPACE` file that calls `maven_install` or cannot be read while `.bazeliskrc` or `.bazelversion` at the Bazel root names no release of Bazel 9 or later, a `maven.install` whose `name` or `lock_file` is not a plain string, a construct of `MODULE.bazel` the reader does not follow, and, in vendored and agent mode, a `maven.override` of the artifact (hosted mode gives `maven_bazel_no_effect`). Hosted mode also gives it for a hub on an isolated `use_extension`, an rc file that cannot be read (also one that an import takes from outside the workspace by a relative path, and one with a line whose reading Bazel's own source calls an oversight: a quote that is never closed, a `\` with nothing after it), a `MODULE.bazel` that is a symbolic link, and a block of `MODULE.bazel` that was changed by hand (also in its first line alone), and then writes no entry for the hub. One warning per purl; the detail names the file, the line and the reason. | | `maven_bazel_not_at_root` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (additive): a rules_jvm_external hub builds the jar of a Maven patch, and the project is below the root of its Bazel workspace, in whose `MODULE.bazel` the patch of a hub is written. Like the five codes below and `maven_bazel_unreadable` in hosted mode, it means that the run wrote no entry for the jar in the block of the root `MODULE.bazel`, because it could not show from the checkout that the entry works. One warning per purl, in the place of `redirect_maven_no_pom` for that purl; the detail says why and ends with what the block of the root holds for the jar, which is read for a project below the root too (the detail then names the file as "the MODULE.bazel at the root"). Nothing: "socket-patch wrote nothing to MODULE.bazel for it, so the Bazel build keeps the upstream jar". The event is the one of the same run without the hub: `skipped` / `redirect_unconfirmed` in a tree only Bazel builds, `applied` beside a `pom.xml` or Gradle pin that landed. The entry of the patch, from an earlier run: that entry stays, the event is `applied`, and the detail ends "socket-patch left the entry that MODULE.bazel holds for it as it is". The entry of another patch of the same jar: it stays too, the event is the one without the hub, and the detail ends "socket-patch wrote nothing to MODULE.bazel for it: the entry of patch remains there, and where the block is in effect the Bazel build takes that patch's jar". Exit code unchanged. With none of these codes the run writes the block: `redirect.rewrittenFiles` holds `MODULE.bazel`, the event is `applied`, and `rollback` / `remove` take the entry out. See docs/ecosystems.md "Bazel workspaces: Maven". | diff --git a/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs b/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs index df1269daa..8e57d229c 100644 --- a/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs +++ b/crates/socket-patch-cli/tests/e2e_bazel_build/cargo.rs @@ -1334,7 +1334,8 @@ fn granted() -> (Package, StandIn, Patched) { /// withholds the pin, and the warning names that line and no attribute. /// Without the line the same scan pins the crate, a scan after it leaves /// the pin as it is, and `rollback` gives the checkout back, the config -/// with its comment line. +/// with its comment line. A config that holds nothing gets no pin, and the +/// warning asks for the comment line. #[test] fn a_hub_without_the_config_gets_no_pin_until_it_names_the_config() { let (upstream, standin, patch) = granted(); @@ -1408,11 +1409,30 @@ fn a_hub_without_the_config_gets_no_pin_until_it_names_the_config() { assert_eq!(undo.envelope["summary"]["rolledBack"], 1, "{}", undo.dump); assert_eq!(changed(&edited, &cell.tree()), NOTHING, "{}", undo.dump); - // KNOWN GAP, and what the comment line is for: a config that holds - // nothing else is deleted by the `rollback` that empties it, and a hub - // that names a missing file fails. Once `rollback` keeps a file it did - // not create, the warning needs no such advice and this goes. + // A config that holds nothing gets no pin: the `rollback` of the pin + // would delete the file, and a hub that names a missing file fails. cell.write(CONFIG, ""); + let blank = cell.tree(); + let detail = unread("a blank config", &socket_patch(&cell, &standin, &scan)); + let starts = format!( + "{NAME}@{VERSION}: the rules_rust hub at {} reads the Cargo files of this project and \ + names {CONFIG}, which holds nothing: ", + hub_at(&named) + ); + let way = format!( + "Put a comment line in {CONFIG} and run this command again; or use `--mode vendored`" + ); + assert!( + detail.starts_with(&starts) && detail.contains(&way), + "{detail}" + ); + assert_eq!(changed(&blank, &cell.tree()), NOTHING); + + // KNOWN GAP, and what the comment line is for: a config that the pin + // creates is deleted by the `rollback` that empties it, and a hub that + // names a missing file fails. Once `rollback` keeps the hub's file, + // the warning needs no such advice and this goes. + std::fs::remove_file(cell.path(CONFIG)).expect("the blank config"); let ran = socket_patch(&cell, &standin, &scan); assert_eq!(ran.envelope["summary"]["applied"], 1, "{}", ran.dump); let undo = socket_patch(&cell, &standin, &["rollback", "--yes"]); diff --git a/crates/socket-patch-core/src/bazel/cargo.rs b/crates/socket-patch-core/src/bazel/cargo.rs index 0e5a83c92..dea7b128e 100644 --- a/crates/socket-patch-core/src/bazel/cargo.rs +++ b/crates/socket-patch-core/src/bazel/cargo.rs @@ -29,17 +29,22 @@ //! - Such a hub is in the way unless its `cargo_config` is one string, the //! label of the config the rewriter writes. A config that cannot be read //! is not taken for the right one. +//! - A hub that names that config is in the way while the file is there +//! and holds nothing: `rollback` deletes a config it leaves empty, and +//! the hub then names a missing file, on which it fails. A comment line +//! keeps the file. A file that is not there yet is created by the pin. //! - A call counts whether or not it is a statement of its own //! ([`Call::whole`]): one that may run is one that may fail. //! - When the workspace uses a name bound to the extension in a way that //! may hide a call ([`super::workspace::Tags::doubts`]), a hub may read //! the project's files, and that is in the way too. The way on is then //! that line: there may be no hub to give an attribute. -//! - The calls of a `WORKSPACE` file count when the module files make no -//! such tag and the checkout does not name a Bazel from 9.0.0 on, which -//! never reads that file. Bazel 7 reads it, and Bazel 8 when it is told -//! to. Beside module files that make one, the file is taken for what a -//! move to Bzlmod left behind. +//! - The calls of a `WORKSPACE` file count when no tag of the module files +//! concerns the project and the checkout does not name a Bazel from +//! 9.0.0 on, which never reads that file. Bazel 7 reads it beside the +//! module files, and Bazel 8 when it is told to. Beside a tag that +//! concerns the project, the file is taken for what a move to Bzlmod +//! left behind. //! //! So a reading that is not certain withholds the pin wherever a hub, or a //! use of the extension through a name bound to it, is seen: the cost is a @@ -52,8 +57,8 @@ //! Not seen, and there a hub without the config fails as before: //! //! - a hub that another Bazel module declares, and a `crates_repository` in -//! a `.bzl` file that the `WORKSPACE` file loads or under another name -//! than its own; +//! a `.bzl` file that the `WORKSPACE` file loads or under a name that no +//! `load` of that file gives it; //! - a tag called on `use_extension(...)` itself, with no name bound to it, //! a `MODULE.bazel` that cannot be read (a link that leads out of the //! Bazel root, a file past the reader's bounds), and whatever else the @@ -72,8 +77,8 @@ use std::path::Path; use super::rc::declared_release; -use super::starlark::{Call, Value}; -use super::workspace::{disk, Entry, Extension, Repo, Workspace}; +use super::starlark::{Call, How, Value}; +use super::workspace::{disk, Entry, Extension, Label, Repo, Workspace}; /// The key under which a hosted run hands what [`unread`] says to the /// cargo rewriter, among its candidate files: no path of a file. @@ -117,16 +122,16 @@ pub fn unread(project: &Path, files: &[String], config: &str) -> Option judge(&view, &parts.join("/"), files, config) } -/// The files of the main repository that `call` names under the attribute -/// `name`, each from the Bazel root: none for an attribute it does not -/// pass. `None` when they are not written out: the attribute is no string -/// and no list of strings, or it is not written and the call passes `**`. +/// The labels of files of the main repository that `call` gives the +/// attribute `name`: none for an attribute it does not pass. `None` when +/// they are not written out: the attribute is no string and no list of +/// strings, or it is not written and the call passes `**`. /// /// `own` is the name a `WORKSPACE` file gives the main repository, for a /// call of that file: `workspace(name = "shop")` makes `@shop//` the main /// repository there (`WorkspaceGlobals.workspace`, Bazel 7.7.2 and 8.8.2). /// Under a name that is not written out, any `@name//` may be it. -fn named(ws: &Workspace, own: Option<&Value>, call: &Call, name: &str) -> Option> { +fn named(ws: &Workspace, own: Option<&Value>, call: &Call, name: &str) -> Option> { let labels: Vec<&String> = match call.arg(name) { None if call.unpacks() => return None, None => Vec::new(), @@ -142,11 +147,21 @@ fn named(ws: &Workspace, own: Option<&Value>, call: &Call, name: &str) -> Option (Repo::Apparent(_), Some(_)) => true, _ => false, }; - main.then(|| label.path()) + main.then_some(label) }; Some(labels.iter().filter_map(main).collect()) } +/// What a call that makes a hub says of a project. +enum Hub { + /// It reads none of the project's Cargo files. + Other, + /// It reads them, or may, and names the config the rewriter writes. + Named, + /// It is in the way, and this is the warning's text. + InTheWay(String), +} + /// [`unread`] for the workspace that `view` shows and the project in its /// directory `project` (empty: the root itself). fn judge( @@ -174,37 +189,44 @@ fn judge( ) }; let workspace = Workspace::read(view); - // The calls that make a hub, each with its file, the first doubt about - // a call that may be hidden, and the name a `WORKSPACE` file whose - // calls count gives the main repository. - let (mut hubs, mut hidden, mut own): (Vec<(&str, &Call)>, _, _) = (Vec::new(), None, None); - for extension in &EXTENSIONS { - let tags = workspace.tags(extension, "from_cargo"); - hubs.extend(tags.calls.iter().map(|tag| (tag.file, tag.call))); - hidden = hidden.or(tags.doubts.into_iter().next()); - } - if let (true, None, Some(Ok((file, legacy)))) = (hubs.is_empty(), &hidden, &workspace.legacy) { - let read = declared_release(view).is_none_or(|release| release < (9, 0, 0)); - let hub = |call: &&Call| read && call.callee == "crates_repository"; - let calls = legacy.calls.iter().filter(hub); - hubs.extend(calls.map(|call| (file.as_str(), call))); - let naming = legacy.calls.iter().find(|call| call.callee == "workspace"); - own = naming.and_then(|call| call.arg("name")); - } - for (file, call) in hubs { + // What the hub that `call` of `file` makes says of the project. `own` + // is the name a `WORKSPACE` file gives the main repository, for a call + // of that file. + let says = |file: &str, call: &Call, own: Option<&Value>| { let read = ["cargo_lockfile", "manifests"].map(|name| named(&workspace, own, call, name)); - let ours = |path: &String| files.contains(path); - let (reads, how) = match read.iter().flatten().flatten().any(ours) { - true => ("reads", ""), - false if read.iter().all(Option::is_some) => continue, - false => ( + let ours = |label: &&Label| files.contains(&label.path()); + let first = read.iter().flatten().flatten().find(ours); + let (reads, how) = match first { + Some(_) => ("reads", ""), + None if read.iter().all(Option::is_some) => return Hub::Other, + None => ( "may read", " (its `cargo_lockfile` or its `manifests` is not written out)", ), }; + let hub = format!( + "the rules_rust hub at {file}:{} {reads} the Cargo files of this project{how}", + call.line + ); + // A name that is not written out widens what a hub may read, never + // what names the project's config. + let own = own.filter(|own| matches!(own, Value::Str(_))); let one = matches!(call.arg("cargo_config"), Some(Value::Str(_))); - if one && named(&workspace, own, call, "cargo_config") == Some(vec![written.clone()]) { - continue; + let given = named(&workspace, own, call, "cargo_config").unwrap_or_default(); + if one && given.iter().map(Label::path).eq([written.clone()]) { + // `rollback` deletes a config it leaves empty, and a hub that + // names a missing file fails. + return match view(&written) { + Entry::Text(text) if text.trim().is_empty() => Hub::InTheWay(format!( + "{hub} and names {written}, which holds nothing: the `rollback` of a pin \ + deletes a cargo config it leaves empty, and the hub then fails on the \ + missing file, and `bazel build` with it; dependency skipped (nothing \ + rewritten). Put a comment line in {written} and run this command again; \ + or use `--mode vendored`, which commits the patched crate and needs no \ + registry" + )), + _ => Hub::Named, + }; } let names = match call.arg("cargo_config") { None if !call.unpacks() => "it names no `cargo_config`".to_string(), @@ -213,19 +235,65 @@ fn judge( of another file it names belong in {written})" ), }; - let why = format!( - "the rules_rust hub at {file}:{} {reads} the Cargo files of this project{how}, and \ - {names}", - call.line - ); + // The label as the hub writes its own: the package of the first of + // the project's files it names may not be the project's directory. + let within = |package: &str| match package { + "" => Some(written.as_str()), + package => written.strip_prefix(package)?.strip_prefix('/'), + }; + let label = first.and_then(|label| { + let name = within(&label.package)?; + Some(format!("//{}:{name}", label.package)) + }); + let label = label.unwrap_or_else(|| format!("//{project}:{config}")); // The file first: a hub that names a missing one fails. let way = format!( "Commit {written} (where the project has none, create it with a comment line in it: \ `rollback` deletes a config it leaves empty, and a hub that names a missing file \ - fails), give the hub `cargo_config = \"//{project}:{config}\"`, the label of that \ - file as one string, and run this command again" + fails), give the hub `cargo_config = \"{label}\"`, the label of that file as one \ + string, and run this command again" ); - return Some(withheld(why, way)); + Hub::InTheWay(withheld(format!("{hub}, and {names}"), way)) + }; + // The tags that make a hub, and the first doubt about one that may be + // hidden. + let (mut concerned, mut hidden) = (false, None); + for extension in &EXTENSIONS { + let tags = workspace.tags(extension, "from_cargo"); + for tag in &tags.calls { + match says(tag.file, tag.call, None) { + Hub::InTheWay(why) => return Some(why), + Hub::Named => concerned = true, + Hub::Other => {} + } + } + hidden = hidden.or(tags.doubts.into_iter().next()); + } + // The calls of a `WORKSPACE` file, where no tag concerns the project + // and none may be hidden: by the rule's own name, or by one that a + // `load` gives it. + if let (false, None, Some(Ok((file, legacy)))) = (concerned, &hidden, &workspace.legacy) { + let read = declared_release(view).is_none_or(|release| release < (9, 0, 0)); + let rule = Value::Str("crates_repository".to_string()); + let loads = legacy.calls.iter().filter(|call| call.callee == "load"); + let aliases = loads + .flat_map(|load| &load.args) + .filter_map(|arg| match &arg.how { + How::Keyword(alias) if arg.value == rule => Some(alias.as_str()), + _ => None, + }); + let names: Vec<&str> = aliases.chain(["crates_repository"]).collect(); + let naming = legacy.calls.iter().find(|call| call.callee == "workspace"); + let own = naming.and_then(|call| call.arg("name")); + let hubs = legacy + .calls + .iter() + .filter(|call| read && names.contains(&&*call.callee)); + for call in hubs { + if let Hub::InTheWay(why) = says(file, call, own) { + return Some(why); + } + } } hidden.map(|doubt| { let why = format!( @@ -433,6 +501,49 @@ mod tests { assert_eq!(said(&[("MODULE.bazel", &*module(&right))], "svc/app"), None); let roots = nested.replace(")", " cargo_config = \"//:.cargo/config.toml\",\n)"); assert!(said(&[("MODULE.bazel", &*module(&roots))], "svc/app").is_some()); + // The label the warning gives is written as the hub writes its own: + // where the hub names the project's files from the root package, + // the project's directory is no package of its own. + let flat = hub(&[ + "cargo_lockfile = \"//:svc/app/Cargo.lock\"", + "manifests = [\"//:svc/app/Cargo.toml\"]", + ]); + let why = said(&[("MODULE.bazel", &*module(&flat))], "svc/app").expect("the flat hub"); + let label = "give the hub `cargo_config = \"//:svc/app/.cargo/config.toml\"`"; + assert!(why.contains(label), "{why}"); + let named = flat.replace( + ")", + " cargo_config = \"//:svc/app/.cargo/config.toml\",\n)", + ); + assert_eq!(said(&[("MODULE.bazel", &*module(&named))], "svc/app"), None); + } + + /// A hub that names the config is still in the way while that file + /// holds nothing: the `rollback` of a pin deletes a config it leaves + /// empty, and the hub then names a missing file. A file with a comment + /// line stays, and a file that is not there yet is created by the pin, + /// as before. + #[test] + fn a_named_config_that_holds_nothing_is_in_the_way() { + let right = module(&hub(&readme_with(&[ + "cargo_config = \"//:.cargo/config.toml\"", + ]))); + for blank in ["", "\n", " \n\t\n"] { + let why = said(&[("MODULE.bazel", &right), (CONFIG, blank)], ""); + let why = why.unwrap_or_else(|| panic!("{blank:?}")); + let said_ = "the rules_rust hub at MODULE.bazel:4 reads the Cargo files of this \ + project and names .cargo/config.toml, which holds nothing: the \ + `rollback` of a pin deletes a cargo config it leaves empty, and the hub \ + then fails on the missing file, and `bazel build` with it; dependency \ + skipped (nothing rewritten). Put a comment line in .cargo/config.toml \ + and run this command again; or use `--mode vendored`"; + assert!(why.starts_with(said_), "{blank:?}: {why}"); + } + for kept in ["# The cargo settings of this project.\n", "[net]\n"] { + let files = [("MODULE.bazel", &*right), (CONFIG, kept)]; + assert_eq!(said(&files, ""), None, "{kept:?}"); + } + assert_eq!(said(&[("MODULE.bazel", &right)], ""), None, "no file yet"); } #[test] @@ -616,6 +727,18 @@ mod tests { assert_eq!(said(&files, ""), None); let files = [("MODULE.bazel", &*module("")), ("WORKSPACE", &*readme)]; assert!(said(&files, "").is_some()); + // A module hub over another project's files leaves the question to + // the `WORKSPACE` file too: Bazel 7 reads both. + let tools = hub(&[ + "cargo_lockfile = \"//tools:Cargo.lock\"", + "manifests = [\"//tools:Cargo.toml\"]", + ]); + let files = [("MODULE.bazel", &*module(&tools)), ("WORKSPACE", &*readme)]; + let why = said(&files, "").expect("the WORKSPACE hub beside another project's"); + assert!( + why.starts_with("the rules_rust hub at WORKSPACE:3 reads"), + "{why}" + ); // `@shop//` is the main repository there, by the name the file // gives it; under a name that is not written out any `@name//` may // be. Another repository's files are not the project's. @@ -631,8 +754,27 @@ mod tests { assert_eq!(said(&[("WORKSPACE", &other)], ""), None); let unnamed = other.replace("name = \"shop\"", "name = NAME"); assert!(said(&[("WORKSPACE", &unnamed)], "").is_some()); - // Not seen: the rule under another name, and a file that only - // loads the `.bzl` file that calls it. + // That doubt widens what a hub may read, never what names the + // project's config: there only the checkout by a name that is + // written out counts. + let shared = " cargo_config = \"@shared//:.cargo/config.toml\",\n"; + let unnamed = call(shared).replace("name = \"shop\"", "name = NAME"); + let why = said(&[("WORKSPACE", &unnamed)], "").expect("another repository's config"); + assert!( + why.contains("its `cargo_config` is not the label of .cargo/config.toml"), + "{why}" + ); + // The rule under the name a `load` gives it is the rule. + let alias = readme + .replace("\"crates_repository\")", "crates = \"crates_repository\")") + .replace("crates_repository(", "crates("); + let why = said(&[("WORKSPACE", &alias)], "").expect("the rule by its loaded name"); + assert!( + why.starts_with("the rules_rust hub at WORKSPACE:3 reads"), + "{why}" + ); + // Not seen: the rule under a name no `load` gives it, and a file + // that only loads the `.bzl` file that calls it. let renamed = readme.replace("crates_repository(", "crates("); assert_eq!(said(&[("WORKSPACE", &renamed)], ""), None); let loads = "load(\"//:deps.bzl\", \"rust_deps\")\nrust_deps()\n"; diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index ee35f4278..a63f0fd70 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -1466,8 +1466,7 @@ pub async fn read_candidate_files_in( let member = |rel: &&String| crate::patch::redirect::is_cargo_member_manifest_key(rel); let mut files = vec!["Cargo.lock".to_string(), "Cargo.toml".to_string()]; files.extend(out.files.keys().filter(member).cloned()); - let legacy = out.files.contains_key(".cargo/config"); - let config = [".cargo/config.toml", ".cargo/config"][usize::from(legacy)]; + let config = crate::patch::redirect::cargo_config_key(&out.files); let root = root.to_path_buf(); let read = crate::utils::fs::run_blocking(move || unread(&root, &files, config)); out.files diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 242216842..46f2fc8d6 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -2049,6 +2049,23 @@ fn rewrite_cargo( rewrite_cargo_with(files, overrides, &CratesIoSource::Registry, result); } +/// The cargo config the cargo rewriter writes for a project with `files`, +/// from the project: the one the project's cargo reads. +/// +/// Cargo reads the LEGACY extensionless `.cargo/config` in preference to +/// `config.toml` when both exist (it warns about the duplicate), so a +/// managed `[registries.…]` block written to `config.toml` there is +/// silently inert and the `registry = "socket-patch-…"` the rewriter puts +/// in Cargo.toml then names an undefined registry. Same preference +/// `vendor::cargo_config::config_path` applies on the vendor path. +pub(crate) fn cargo_config_key(files: &BTreeMap) -> &'static str { + if files.contains_key(".cargo/config") { + ".cargo/config" + } else { + ".cargo/config.toml" + } +} + /// [`rewrite_cargo`] with `cargo_outer`: where the cargo configs read for a /// build in the project (its own, every parent directory's and /// `$CARGO_HOME`'s) take crates.io from. The project's own config is also @@ -2114,17 +2131,7 @@ fn rewrite_cargo_with( .get("Cargo.lock") .cloned() .map(|t| CargoHostedLock::new(to_lf("Cargo.lock", t))); - // Cargo reads the LEGACY extensionless `.cargo/config` in preference to - // `config.toml` when both exist (it warns about the duplicate), so a - // managed `[registries.…]` block written to `config.toml` there is - // silently inert and the `registry = "socket-patch-…"` this rewriter puts - // in Cargo.toml then names an undefined registry. Same preference - // `vendor::cargo_config::config_path` applies on the vendor path. - let cargo_config_key = if files.contains_key(".cargo/config") { - ".cargo/config" - } else { - ".cargo/config.toml" - }; + let cargo_config_key = cargo_config_key(files); let mut cargo_config = files .get(cargo_config_key) .cloned() diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 270372d4e..c70e0d282 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -2069,7 +2069,9 @@ scan to the patched build: - Commit `.cargo/config.toml` and give the hub the attribute, then run the same command: it pins the crate, and the build is the patched one. The file comes first. A project that has none creates it with a comment line in it: a hub that names a missing file fails, and - `rollback` deletes a cargo config it leaves empty, which a comment line prevents. + `rollback` deletes a cargo config it leaves empty, which a comment line prevents. The + warning writes the label in the package of the hub's own labels: for a hub that names + `//:svc/app/Cargo.toml` it is `//:svc/app/.cargo/config.toml`. - Use `--mode vendored`. It commits the patched crate under `.socket/vendor/cargo`, points a `[patch.crates-io]` entry at it and needs no registry. @@ -2080,6 +2082,14 @@ already, and there may be no other. The warning then names that line and no attr Warning: cfg-if@1.0.4: MODULE.bazel:17: `crate` is used other than to call a tag, so a rules_rust hub that socket-patch cannot read may take the Cargo files of this project. [...] dependency skipped (nothing rewritten). Write that line as the call of a tag (`crate.from_cargo(...)`) or as `use_repo(...)` and run this command again: it then pins the crate, or names the hub that is in the way; or use `--mode vendored`, which commits the patched crate and needs no registry ``` +A hub that names the config is still in the way while that file is there and holds nothing +but white space. The pin would be written into it, a later `rollback` would delete the file it +leaves empty, and the hub would fail on the missing file. The warning asks for a comment line: + +``` +Warning: cfg-if@1.0.4: the rules_rust hub at MODULE.bazel:10 reads the Cargo files of this project and names .cargo/config.toml, which holds nothing: the `rollback` of a pin deletes a cargo config it leaves empty, and the hub then fails on the missing file, and `bazel build` with it; dependency skipped (nothing rewritten). Put a comment line in .cargo/config.toml and run this command again; or use `--mode vendored`, which commits the patched crate and needs no registry +``` + A vendored crate that a hosted scan would take over stays vendored and patched (`redirect_takeover_kept_vendored`). A pin that an earlier run wrote stays as it is: the command reports it applied, as before, and writes nothing. On such a hub that build is @@ -2089,13 +2099,17 @@ Everything is read from the checkout, and nothing is evaluated: `MODULE.bazel` a it includes, for the `from_cargo` tags of the `crate` extension of rules_rust (`//crate_universe:extensions.bzl`, and `extension.bzl`, which hands it on) under whatever name the file gives the ruleset and the extension; the `WORKSPACE` file, for calls of -`crates_repository`; `.bazeliskrc` and `.bazelversion`, for the Bazel release. +`crates_repository`, also under the name a `load` of that file gives the rule; `.bazeliskrc` +and `.bazelversion`, for the Bazel release; and the cargo config a hub names, for whether it +holds anything. - A hub concerns the project when a label of its `cargo_lockfile` or its `manifests` is the project's `Cargo.lock`, its `Cargo.toml` or the `Cargo.toml` of a workspace member below it. `@//` is the checkout, and so is `@//` in a `WORKSPACE` - file whose `workspace(name = "")` says so (any `@name//`, where that name is not - written out as a string). A label of another repository is no file of the project. + file whose `workspace(name = "")` says so. Where that name is not written out as a + string, any `@name//` label of a lock or a manifest may be the checkout, and none is taken + for the checkout as a `cargo_config`. A label of another repository is no file of the + project. - A hub that concerns the project is in the way unless its `cargo_config` is one string, the label of the config hosted mode writes: the project's `.cargo/config.toml`, or its `.cargo/config` where the project has that file. Another config is not enough, whatever it @@ -2113,11 +2127,12 @@ name the file gives the ruleset and the extension; the `WORKSPACE` file, for cal - A use of the extension that may hide a call (the name bound to it used for anything but a tag or `use_repo`): a hub that cannot be read may take the project's files. The warning names the line to rewrite. - - A `crates_repository` of the `WORKSPACE` file counts when the module files make no - `crate.from_cargo` hub, unless `.bazeliskrc` or `.bazelversion` names Bazel 9.0.0 or later, - which never reads that file. Bazel 7 reads it, and Bazel 8 when it is told to. So a - `WORKSPACE` file that a move to Bzlmod left behind can withhold a pin on Bazel 8 where the - module files make no such hub: give its call the attribute, or delete the call. + - A `crates_repository` of the `WORKSPACE` file counts when no `crate.from_cargo` hub of the + module files concerns the project, unless `.bazeliskrc` or `.bazelversion` names Bazel + 9.0.0 or later, which never reads that file. Bazel 7 reads it beside the module files, and + Bazel 8 when it is told to. So a `WORKSPACE` file that a move to Bzlmod left behind can + withhold a pin on Bazel 8 where the module files make no hub for the project: give its + call the attribute, or delete the call. - Nothing changes where no hub is seen to concern the project: in no Bazel workspace (the rule is under [Bazel workspaces: `.bazelignore`](#bazel-workspaces-bazelignore)), in a workspace without a crate_universe hub, for a hub that names the config, for a hub whose labels are all @@ -2130,7 +2145,7 @@ Limits: - Not seen, so that the run is the one it was and a hub without the config fails as before: - a hub that another Bazel module declares; - a `crates_repository` inside a `.bzl` file that the `WORKSPACE` file loads, or called - under another name; + under a name that no `load` of the `WORKSPACE` file gives it; - where no hub is seen: a tag called on `use_extension(...)` itself, with no name bound to it; a `MODULE.bazel` that cannot be read (a link that leads out of the Bazel root, a file past the reader's bounds); and what else the module files hide from every reading (a file @@ -2149,6 +2164,9 @@ Limits: is the ruleset's own rule, and this check does not look at it. - `socket-patch vex` attests a hosted pin from the three files alone. It does not look at the hub. +- A hub that names a cargo config which is not there yet gets the pin, which creates the + file. A later `rollback` deletes the file again, and the hub fails on the missing file as it + did before the pin. Create the file with a comment line in it first. ## Cargo: shared registry cache diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index a00e2b99a..436658668 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -4,7 +4,7 @@ "paths": [ "crates/socket-patch-core/src/bazel/" ], - "max": 2618 + "max": 2665 }, "lock-readers": { "paths": [