diff --git a/.github/workflows/bazel-nuget.yml b/.github/workflows/bazel-nuget.yml index 1316b4054..e746a4755 100644 --- a/.github/workflows/bazel-nuget.yml +++ b/.github/workflows/bazel-nuget.yml @@ -1,8 +1,9 @@ name: Bazel patch compatibility (NuGet) -# The NuGet flow of the real-Bazel tests (`nuget::` in `e2e_bazel_build`): +# The NuGet flows of the real-Bazel tests (`nuget::` in `e2e_bazel_build`): # hosted mode for a package that rules_csharp's `nuget.from_lock` reads from -# `packages.lock.json` and `nuget.config`, on each Bazel release below. +# `packages.lock.json` and `nuget.config`, and for a tag that names no +# `nuget_config`, on each Bazel release below. # bazel-compatibility.yml holds the legs and what every ecosystem shares; # this file says when the NuGet legs run and on which releases. # @@ -35,11 +36,14 @@ on: # finds a package in the global packages folder, the reader of the # NuGet settings a project inherits (vendor/nuget_config.rs), the # discovery of a hosted pin in the two files, the upstream restore of - # `rollback`, and the hosted scan's check of that folder. + # `rollback`, and the hosted scan's check of that folder. Then what + # reads the tag in `MODULE.bazel`: the reader of the workspace files, + # which every ecosystem's flow shares (bazel/mod.rs, starlark.rs and + # workspace.rs), and what hosted mode makes of the tag (bazel/nuget.rs). # vendor/nuget_feed.rs is the vendored backend; it is here for the # version and folder helpers that every one of those files calls. - # Nothing another ecosystem's flow owns is here, so a change to one - # ecosystem starts that ecosystem's legs only. + # Nothing else that another ecosystem's flow owns is here, so any other + # change to one ecosystem starts that ecosystem's legs only. # # What it does not cover: shared engine code. The hosted rewrite itself # is there (`rewrite_nuget` in patch/redirect/mod.rs, the file of every @@ -66,6 +70,10 @@ on: - 'crates/socket-patch-core/src/vex/discover/nuget.rs' - 'crates/socket-patch-core/src/patch/redirect/upstream/nuget.rs' - 'crates/socket-patch-cli/src/commands/scan/hosted/nuget.rs' + - 'crates/socket-patch-core/src/bazel/mod.rs' + - 'crates/socket-patch-core/src/bazel/nuget.rs' + - 'crates/socket-patch-core/src/bazel/starlark.rs' + - 'crates/socket-patch-core/src/bazel/workspace.rs' push: branches: [main] # The pull request paths, and what only a push to main covers: the @@ -89,6 +97,10 @@ on: - 'crates/socket-patch-core/src/vex/discover/nuget.rs' - 'crates/socket-patch-core/src/patch/redirect/upstream/nuget.rs' - 'crates/socket-patch-cli/src/commands/scan/hosted/nuget.rs' + - 'crates/socket-patch-core/src/bazel/mod.rs' + - 'crates/socket-patch-core/src/bazel/nuget.rs' + - 'crates/socket-patch-core/src/bazel/starlark.rs' + - 'crates/socket-patch-core/src/bazel/workspace.rs' - 'crates/socket-patch-cli/tests/e2e_bazel_build/main.rs' - 'crates/socket-patch-core/src/patch/redirect/**' - 'crates/socket-patch-core/src/hosted/**' diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 2596ee4c8..3e2f80e33 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -2518,6 +2518,8 @@ command's integration tests check its output. | `redirect_pin_lockless` | `warnings[]` (warning) | scan/get `--mode hosted` (nuget, cargo): the pin was written without a lockfile that records its version, so `vex` cannot attest it and `rollback` / `remove` / `vendor` refuse it as unattributable. The detail names the lockfile to create (`dotnet restore --use-lock-file`, `cargo generate-lockfile`) before re-running the hosted scan. | | `redirect_pypi_stale_install` | `warnings[]` (warning) | Hosted Python redirect: readable installed files differ from the pinned patched wheel's `RECORD` (or from the patch record, when the wheel cannot judge the copy). Read-only, repeated on re-scan, and excludes the package from same-run VEX. See the "Python stale-install guard" section. | | `redirect_nuget_stale_global_package` / `vendor_nuget_stale_global_package` | `warnings[]` (warning, hosted) / `skipped` (warning, vendor advisory event) | scan `--mode hosted` / `vendor` / `scan --mode vendored` (nuget, v5.0 #352): NuGet's global packages folder (`NUGET_PACKAGES`, else `~/.nuget/packages`) already holds the patched package extracted from other bytes (its `.nupkg.metadata` `contentHash` is not the patched one). A patch keeps the upstream id and version and NuGet restores a package already in that folder without asking any source, so `dotnet restore` would keep the upstream bytes (silently without a lock, NU1403 against the re-pinned lock with one). The detail names the directory and the remedy: delete it, then `dotnet restore` (`dotnet nuget locals global-packages --clear` also works but empties the whole machine-wide folder, and the detail says so); CI caches of that folder must drop it too. Read-only like the gem guard (the folder is shared machine-wide), re-fired on every re-run until the copy is gone, skipped on `--dry-run`; a hosted purl it flags is excluded from the same run's `--vex` `assume_applied` and reported stale. A dir without `.nupkg.metadata` (a legacy `packages/` folder) is never judged. | +| `redirect_bazel_nuget_config_undeclared` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (nuget, additive): a rules_csharp `nuget.from_lock` tag of the root `MODULE.bazel` (or of a file it includes) reads a `packages.lock.json` of the project, or may (its `packages_lock_json` is not a label in a plain string), and the tags do not show that the hub takes its package sources from the `nuget.config` hosted mode writes. The hub has one list of sources, which the last tag with a `nuget_config` or `feeds` decides: that tag names another file or has `feeds` alone, no tag has either, or a tag cannot be read with certainty (it is not a call of its own, passes `**`, or its `nuget_config` is not a label in a plain string). Also given, for every NuGet package of the run, where the tags do show it, a tag has `feeds` and the config names no package source yet: the first source in it would take the place of those `feeds`. Where the tags are in more than one file their order is unknown and every tag with a `nuget_config` or `feeds` must name the config. The hub would download the upstream package and fail the build on the re-pinned `contentHash`, so nothing is written for the package: no `nuget.config`, no lock entry. The package is a `skipped` / `redirect_unconfirmed` event; status and exit are unchanged (exit 0). One warning per package that the run would re-pin in a lock a tag reads (a package refused for another reason keeps that reason's code; a package in no such lock is pinned as without Bazel): the detail starts `@: ::`, says what stands against the pin, and names the way on: the attribute to put on the tag (`nuget_config = "//:"`, with the config as the project directory spells it; the file alone where the tag does not name its lock in a plain string or the config lies outside the package of that label), with the sources or `feeds` to name in that config where the tag had others, or, for a tag that names a NuGet config in a directory above the project, the directory to run the command in (`--cwd`). Never given outside a Bazel workspace, for a project without a lock, or when no tag names one of the project's locks: those runs are unchanged, as is one in a workspace where a part of `MODULE.bazel` cannot be read and the tags that are read show the config. See [Bazel workspaces: NuGet](../../docs/ecosystems.md#bazel-workspaces-nuget). | +| `redirect_bazel_nuget_lock_not_repinned` | `warnings[]` (warning; human: `Warning: …`) | scan/get `--mode hosted` in a Bazel workspace (nuget, additive): under the same condition (a `nuget.from_lock` tag reads, or may read, a lock of the project), another tag reads a `packages.lock.json` of the checkout that this run does not re-pin, and that lock holds the package at the patched version with another `contentHash`, or cannot be read; or another tag, beside one that names a lock of the project, does not name its lock in a plain string. The hub holds a package to the `contentHash` of the first lock that names it, so it would refuse the patched package: nothing is written for the package (`skipped` / `redirect_unconfirmed`, exit 0), also when the hub does not read the config yet. Not given when the tags are in one file and a lock of the project that holds the package comes before the other lock, or when the other lock holds the package at another version or already at the patched `contentHash`. The detail starts `@: ::`, names the tag and the lock, and says to run the command in a directory above both locks that holds a solution file or a `nuget.config` (`--cwd`), where the lock of every project is re-pinned; for a tag whose lock is not named, to write its `packages_lock_json` as a label in a plain string. Checked before `redirect_bazel_nuget_config_undeclared`. See [Bazel workspaces: NuGet](../../docs/ecosystems.md#bazel-workspaces-nuget). | | `redirect_gem_stale_install` | `warnings[]` (warning) | scan / get `--mode hosted` (gem): a stale UNPATCHED materialization (installed gem, or committed archive in bundler's cache dir — `vendor/cache` unless `cache_path` moves it) that `bundle install` will reuse instead of fetching the redirected patch, and that the run did not refresh: it is in a shared gem home, or the refresh could not be done (the detail says why) or is turned off. The detail carries the remedy. A human run prints one warning for all of them. Full rules and flavors: the "Gem stale-install guard" section. | | `redirect_gem_install_refreshed` | `warnings[]` (note) | scan / get `--mode hosted` (gem): a stale materialization the run replaced with the sha256-verified patched gem (the detail names the purl and the path). A human run prints a progress line instead. See the "Gem stale-install guard" section. | | `redirect_gem_install_would_refresh` | `warnings[]` (note) | scan / get `--mode hosted --dry-run` (gem): a stale materialization a wet run would replace. Nothing was written or downloaded. | diff --git a/crates/socket-patch-cli/tests/e2e_bazel_build/nuget.rs b/crates/socket-patch-cli/tests/e2e_bazel_build/nuget.rs index 8e8548eb4..a51c1b7e8 100644 --- a/crates/socket-patch-cli/tests/e2e_bazel_build/nuget.rs +++ b/crates/socket-patch-cli/tests/e2e_bazel_build/nuget.rs @@ -111,6 +111,23 @@ //! nuget.org or the patch host. It is there from the first build on, and //! step (c) changes no flag. //! +//! A second flow, [`hosted_no_config`], is the workspace of rules_csharp's +//! own examples: the tag names the lock and no `nuget_config`, and the +//! checkout holds no `nuget.config`. The hub then has nuget.org as its one +//! source, so a lock re-pinned to the patched package would fail the build: +//! the hub downloads the upstream package and refuses it over the lock's +//! `contentHash`. `socket-patch` reads the tag in `MODULE.bazel` and +//! writes nothing: the same `scan` exits 0, reports the patch as skipped +//! and warns with the tag's file and line and the attribute to add, and +//! the build passes on the upstream package as before. With that +//! attribute on the tag and no other step, the same `scan` creates +//! `nuget.config` and re-pins the lock, and the build is the patched one. +//! [`only_a_tag_that_reads_the_lock_without_its_config_is_withheld`] holds +//! the other workspaces to what the command does without Bazel, and +//! [`the_pin_is_written_where_the_hub_takes_it`] the hubs of more than one +//! tag: the last tag with sources decides which config the hub reads, and +//! the first lock that names a package gives its `contentHash`. +//! //! # Known gaps //! //! Three outcomes the flow asserts are gaps of the product, not its @@ -137,7 +154,8 @@ use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; use crate::envelope::{codes_in, event_triples}; use crate::harness::{ - changed, fixture_path, rewrite, Cell, Fixture, Ran, StandIn, Tree, CONSUMER, NOTHING, + cell_without_bazel, changed, fixture_path, rewrite, Cell, Fixture, Ran, StandIn, Tree, + CONSUMER, NOTHING, }; use crate::hosted_memory_common::{read_tree, Patch, ORG}; @@ -171,6 +189,9 @@ const RUN: &str = "@RUN@"; const CONFIG: &str = "nuget.config"; const LOCK: &str = "packages.lock.json"; const BAZEL_LOCK: &str = "MODULE.bazel.lock"; +/// The file of the tag, and the line of it that names [`CONFIG`] there. +const MODULE: &str = "MODULE.bazel"; +const NAMES_CONFIG: &str = " nuget_config = \"//:nuget.config\",\n"; /// The files of the fixture that name a package. const NAMING: [&str; 3] = ["App.csproj", "BUILD.bazel", LOCK]; /// nuget.org as the fixture's `nuget.config` names it, and as NuGet records @@ -323,6 +344,12 @@ fn fixture_file(rel: &str, run: &str) -> String { text.replace(RUN, run) } +/// A file of the fixture's `common` directory that names no package. +fn fixture_file_as_is(rel: &str) -> String { + let path = fixture_path(FIXTURE.dir).join("common").join(rel); + std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{}: {e}", path.display())) +} + /// The entry of a lock that resolves the package `id` to the file of /// `content_hash`, laid out as `dotnet restore` writes it. fn lock_entry(id: &str, content_hash: &str) -> String { @@ -493,6 +520,12 @@ fn downloads(origin: &str) -> Vec { /// set, so that neither a `NUGET_PACKAGES` nor a `DOTNET_CLI_HOME` of the /// caller's shell moves the folder or adds a user-level NuGet config. fn socket_patch(cell: &Cell, standin: &StandIn, home: &Path, args: &[&str]) -> Ran { + on_machine(standin, home, |api, env| cell.socket_patch(api, args, env)) +} + +/// What `run` gives for the Socket API and the environment of +/// [`socket_patch`]. +fn on_machine(standin: &StandIn, home: &Path, run: impl Fn(&str, &[(&str, &str)]) -> T) -> T { let origin = standin.origin(); let upstream = format!("{origin}/{NUGET_ORG}"); let packages = packages_folder(home); @@ -502,7 +535,7 @@ fn socket_patch(cell: &Cell, standin: &StandIn, home: &Path, args: &[&str]) -> R ("DOTNET_CLI_HOME", home.as_ref()), ("NUGET_PACKAGES", packages.as_ref()), ]; - cell.socket_patch(&origin, args, &env) + run(&origin, &env) } /// A text file of a tree. @@ -820,6 +853,408 @@ fn hosted_from_lock() { ); } +/// The codes of the warnings with which hosted mode says that it wrote +/// nothing for a package: the hub does not read `nuget.config`, or it has +/// the package from a lock that the run does not re-pin. +const WITHHELD: &str = "redirect_bazel_nuget_config_undeclared"; +const NOT_REPINNED: &str = "redirect_bazel_nuget_lock_not_repinned"; + +/// For a finished `scan --mode hosted --yes` in a checkout whose tag names +/// no `nuget_config`: it passed and reports the package `purl` as skipped +/// and no file as rewritten, and its one warning is about `package`, under +/// the name the purl has for it. The warning names the tag by its file and +/// line, and says to put on it the attribute that [`NAMES_CONFIG`] is in +/// the fixture; it does not send the user to vendored mode, which no +/// Bazel build of NuGet packages is known to take. Gives the warning's +/// text. +fn withheld(test: &str, scan: &Ran, package: &Package, purl: &str) -> String { + let said = &scan.dump; + assert_eq!(scan.code, 0, "{test}: {said}"); + assert_eq!(scan.envelope["status"], "success", "{test}: {said}"); + assert_eq!(scan.envelope["summary"]["applied"], 0, "{test}: {said}"); + let skipped = [( + purl.to_string(), + "skipped".to_string(), + "redirect_unconfirmed".to_string(), + )]; + assert_eq!(event_triples(&scan.envelope), skipped, "{test}: {said}"); + let rewritten = &scan.envelope["redirect"]["rewrittenFiles"]; + assert_eq!(*rewritten, json!([]), "{test}: {said}"); + let warnings = &scan.envelope["warnings"]; + assert_eq!(codes_in(warnings), [WITHHELD], "{test}: {said}"); + let detail = warnings[0]["detail"].as_str().unwrap_or_default(); + let starts = format!( + "{}@{VERSION}: {MODULE}:9: this `nuget.from_lock` names no `nuget_config`, so \ + rules_csharp does not read {CONFIG}", + package.lower() + ); + assert!(detail.starts_with(&starts), "{test}: {detail}"); + let attribute = NAMES_CONFIG.trim().trim_end_matches(','); + let way = format!("Put `{attribute}` on that tag and run this command again"); + assert!(detail.contains(&way), "{test}: {detail}"); + assert!(!detail.contains("vendored"), "{test}: {detail}"); + detail.to_string() +} + +/// The flow of a tag that names no `nuget_config`, in a checkout without a +/// `nuget.config`, on the machine that restored the project. +#[test] +#[ignore = "real Bazel; run with --ignored"] +fn hosted_no_config() { + let test = "nuget::hosted_no_config"; + let started = Instant::now(); + let Some(mut cell) = Cell::open(test, &FIXTURE) else { + return; + }; + let release = cell.release().to_string(); + + let run = format!("R{}", uuid::Uuid::new_v4().simple()); + let upstream = Package::new(NAME, &run, &format!("PRISTINE {run}")); + let neighbour = Package::new(NEIGHBOUR, &run, "NEIGHBOUR"); + let patch = Patched::new(&run); + let (purl, unpatched, patched) = (upstream.purl(), upstream.marker(), patch.package.marker()); + let standin = registry(&[&neighbour, &upstream]); + grant(&standin, &upstream, &patch, &patch.package.nupkg); + cell.route_downloads(&downloads(&standin.origin())); + check_out(&cell, &run, &[&neighbour, &upstream]); + // The fixture's tag without the line that names its `nuget.config`, + // and no such file. + let module = std::fs::read_to_string(cell.path(MODULE)).expect(MODULE); + cell.write(MODULE, module.replace(NAMES_CONFIG, "")); + std::fs::remove_file(cell.path(CONFIG)).expect("remove nuget.config"); + let home = tempfile::Builder::new() + .prefix("sp-bazel-nuget-") + .tempdir() + .expect("create the .NET home"); + for package in [&neighbour, &upstream] { + package.restore_into(home.path()); + } + + // The unpatched build: the hub has nuget.org alone. + let before = cell.builds_unpatched(&unpatched); + assert!(!before.contains_key(CONFIG), "{test}: a {CONFIG} is left"); + + // The scan writes nothing: a lock re-pinned to the patched package + // would fail the next build, because the hub would still download the + // upstream one. So the build passes as before, and changes no file. + let scan = socket_patch(&cell, &standin, home.path(), &SCAN); + withheld(test, &scan, &upstream, &purl); + assert_eq!(changed(&before, &cell.tree()), NOTHING, "{test}"); + cell.builds(&unpatched); + assert_eq!(changed(&before, &cell.tree()), NOTHING, "{test}"); + + // What the warning says to do, and no other step: the attribute on the + // tag, and the same command. It creates the file the tag now names and + // re-pins the lock, and the build is the patched one, from the Socket + // source. + cell.write(MODULE, &module); + let wired = cell.tree(); + let scan = socket_patch(&cell, &standin, home.path(), &SCAN); + let pinned = cell.pinned_by(&scan, &purl, &patch.uuid, &wired, &[CONFIG, LOCK]); + assert_eq!( + text(&pinned, LOCK), + lock_pinning(&run, &[&neighbour, &patch.package]), + "{test}: {LOCK}" + ); + cell.builds(&patched); + assert_eq!(changed(&pinned, &cell.tree()), [BAZEL_LOCK], "{test}"); + let fetches = standin.asked(&patch.file(&patch.package)); + assert!(fetches > 0, "{test}: the Socket source was not asked"); + + println!( + "{test}: on Bazel {release}, `scan --mode hosted` wrote nothing for a tag that names no \ + nuget_config and warned ({WITHHELD}), and the build still printed {unpatched:?}; with \ + the attribute on the tag the same scan wrote {CONFIG} and {LOCK}, and the build printed \ + {patched:?} from the Socket source ({fetches} requests for the package); {:?}", + started.elapsed() + ); +} + +/// Without a Bazel: `scan --mode hosted --yes` through the binary, in +/// checkouts of the fixture's project that hold no `nuget.config`. What it +/// gives a person, then a script, and the files it leaves are the same in +/// a checkout that is no Bazel workspace, in one whose tag names the +/// config, and in one whose tag reads the lock of another project. Only +/// the tag that reads this lock and names no config is withheld: nothing +/// is written, a person reads the warning a script gets, and with the +/// attribute on the tag the command gives what it gives without Bazel. +/// Where the two files already hold the pin, it warns and leaves them. +#[test] +fn only_a_tag_that_reads_the_lock_without_its_config_is_withheld() { + let test = "nuget::only_a_tag_that_reads_the_lock_without_its_config_is_withheld"; + let (upstream, neighbour) = ( + Package::new(NAME, "R1", "PRISTINE"), + Package::new(NEIGHBOUR, "R1", "NEIGHBOUR"), + ); + let (patch, purl) = (Patched::new("R1"), upstream.purl()); + let standin = StandIn::start(); + grant(&standin, &upstream, &patch, &patch.package.nupkg); + let home = tempfile::tempdir().expect("create the .NET home"); + for package in [&neighbour, &upstream] { + package.restore_into(home.path()); + } + let era = fixture_path(FIXTURE.dir).join(FIXTURE.eras[0].1); + let module = std::fs::read_to_string(era.join(MODULE)).expect(MODULE); + let bare = module.replace(NAMES_CONFIG, ""); + let checkout = |module: Option<&str>| { + let cell = cell_without_bazel("9.3.2"); + check_out(&cell, "R1", &[&neighbour, &upstream]); + cell.write(".git/HEAD", "ref: refs/heads/main\n"); + if let Some(module) = module { + cell.write(MODULE, module); + } + cell + }; + // What the command gives a person, then a script, and the files it + // leaves beside `MODULE.bazel`. + let gives = |cell: &Cell| { + let scan = |api: &str, env: &[(&str, &str)]| cell.socket_patch_person(api, &SCAN, env); + let person = on_machine(&standin, home.path(), scan); + let ran = socket_patch(cell, &standin, home.path(), &SCAN); + let mut files = cell.tree(); + files.remove(MODULE); + (person, ran, files) + }; + let said = |(person, ran, files): &([String; 2], Ran, Tree)| { + ( + person.clone(), + ran.code, + ran.envelope.clone(), + files.clone(), + ) + }; + + // Without Bazel the command pins the package: it creates the config + // and re-pins the lock. + let plain = checkout(None); + let before = plain.tree(); + let without = gives(&plain); + let pinned = changed(&before, &plain.tree()); + assert_eq!(pinned, [CONFIG, LOCK], "{test}: {}", without.1.dump); + assert!( + without.0[0].contains("Switched 1 package"), + "{:?}", + without.0 + ); + let elsewhere = bare.replace("//:packages", "//api:packages"); + for same in [&module, &elsewhere] { + assert_eq!( + said(&gives(&checkout(Some(same)))), + said(&without), + "{same}" + ); + } + + let cell = checkout(Some(&bare)); + let before = cell.tree(); + let (person, scan, _) = gives(&cell); + let detail = withheld(test, &scan, &upstream, &purl); + assert_eq!(changed(&before, &cell.tree()), NOTHING, "{test}"); + let told = [ + "No patches could be switched to hosted:".to_string(), + format!( + " {purl}: no lockfile entry pinning it could be rewritten (see the warning below)" + ), + format!("Warning: {detail}"), + ]; + // The last lines of stderr, after what every scan says there. + let mut lines: Vec<&str> = person[1].lines().collect(); + let last = lines.split_off(lines.len().saturating_sub(told.len())); + assert_eq!(last, told, "{test}: {person:?}"); + assert!(person[0].starts_with("Switched 0 packages"), "{person:?}"); + cell.write(MODULE, &module); + assert_eq!(said(&gives(&cell)), said(&without), "{test}"); + + // A checkout that an earlier release pinned under such a tag: the pin + // stays in the two files and is reported as it is, beside the warning. + plain.write(MODULE, &bare); + let before = plain.tree(); + let scan = socket_patch(&plain, &standin, home.path(), &SCAN); + assert_eq!(changed(&before, &plain.tree()), NOTHING, "{test}"); + let applied = [(purl, "applied".to_string(), String::new())]; + assert_eq!(event_triples(&scan.envelope), applied, "{}", scan.dump); + let warned = codes_in(&scan.envelope["warnings"]); + assert_eq!(warned[0], WITHHELD, "{}", scan.dump); +} + +/// Without a Bazel: `scan --mode hosted --yes` through the binary in the +/// workspaces of a hub with more than one tag, or with a tag that is not +/// the fixture's. Each is a checkout of the fixture's project, also below +/// the root as `app` and `lib`, with the tags of its row; the scan runs in +/// the directory of the row. It pins the package and rewrites the files +/// named, or it writes nothing and its one warning has the code and the +/// words of the row. What Bazel builds in each was measured on 9.3.2 and +/// is not run here. +#[test] +fn the_pin_is_written_where_the_hub_takes_it() { + let test = "nuget::the_pin_is_written_where_the_hub_takes_it"; + let (upstream, neighbour) = ( + Package::new(NAME, "R1", "PRISTINE"), + Package::new(NEIGHBOUR, "R1", "NEIGHBOUR"), + ); + let (patch, purl) = (Patched::new("R1"), upstream.purl()); + let standin = StandIn::start(); + grant(&standin, &upstream, &patch, &patch.package.nupkg); + let home = tempfile::tempdir().expect("create the .NET home"); + for package in [&neighbour, &upstream] { + package.restore_into(home.path()); + } + let era = fixture_path(FIXTURE.dir).join(FIXTURE.eras[0].1); + let module = std::fs::read_to_string(era.join(MODULE)).expect(MODULE); + let (top, _) = module.split_once("nuget.from_lock(").expect("the tag"); + // A file of tags, each the lock it reads and the rest of its lines. + let with = |tags: &[(&str, &str)]| { + let tag = |(lock, more): &(&str, &str)| { + format!("nuget.from_lock(\n{more} packages_lock_json = \"{lock}\",\n)\n") + }; + let tags: String = tags.iter().map(tag).collect(); + format!("{top}{tags}use_repo(nuget, \"nuget\")\n") + }; + let (root, app, lib) = ( + "//:packages.lock.json", + "//app:packages.lock.json", + "//lib:packages.lock.json", + ); + let (own, of_app, of_api) = ( + NAMES_CONFIG, + " nuget_config = \"//app:nuget.config\",\n", + " nuget_config = \"//api:nuget.config\",\n", + ); + let feeds = format!(" feeds = [\"{SOURCE}\"],\n"); + let computed = "\"0.5.0\", repo_name = \"rules_\" + \"csharp\")"; + let renamed = module.replacen("\"0.5.0\")", computed, 1); + assert_ne!(renamed, module); + let (app_lock, app_config) = ("app/packages.lock.json", "app/nuget.config"); + let empty = "{\n \"version\": 1,\n \"dependencies\": {\n \"net8.0\": {}\n }\n}\n"; + // The module file, the file added to the checkout, the directory of + // the scan, and then the files it rewrites, or the code and the words + // of its warning. + type Row<'a> = (String, &'a str, &'a str, &'a [&'a str], &'a [&'a str]); + let rows: [Row; 9] = [ + // The last tag with sources names the config; a line that the + // reader cannot read is beside the tag, which names the config. + ( + with(&[("//api:packages.lock.json", of_api), (root, own)]), + "api/packages.lock.json", + "", + &[CONFIG, LOCK], + &[], + ), + (renamed, "", "", &[CONFIG, LOCK], &[]), + // Feeds and no config; a config that the directory spells its own + // way. + ( + with(&[(root, &feeds)]), + "", + "", + &[], + &[ + WITHHELD, + "MODULE.bazel:9: this `nuget.from_lock` has `feeds` and no `nuget_config`", + "name its `feeds` in nuget.config", + ], + ), + ( + with(&[(root, "")]), + "NuGet.Config", + "", + &[], + &[ + WITHHELD, + "does not read NuGet.Config, where", + "Put `nuget_config = \"//:NuGet.Config\"` on that tag", + ], + ), + // Two locks hold the package and the scan re-pins one: the hub + // has the package from the first. + ( + with(&[(lib, of_app), (app, of_app)]), + "", + "app", + &[], + &[ + NOT_REPINNED, + "MODULE.bazel:9: this `nuget.from_lock` reads lib/packages.lock.json, which holds \ + the package as well and which this run does not re-pin", + "Run this command in a directory above both locks", + ], + ), + ( + with(&[(app, of_app), (lib, of_app)]), + "", + "app", + &[app_config, app_lock], + &[], + ), + ( + with(&[(lib, own), (app, own)]), + "", + "", + &[app_lock, "lib/packages.lock.json", CONFIG, LOCK], + &[], + ), + // The tag names a config above the project: the scan writes it + // where it runs in that directory. + ( + with(&[(app, own)]), + "", + "app", + &[], + &[ + WITHHELD, + "names `//:nuget.config` as its `nuget_config`, not app/nuget.config", + "Run this command in the directory of that file (`--cwd`)", + ], + ), + (with(&[(app, own)]), "", "", &[app_lock, CONFIG, LOCK], &[]), + ]; + for (module, added, cwd, rewritten, warned) in rows { + let cell = cell_without_bazel("9.3.2"); + check_out(&cell, "R1", &[&neighbour, &upstream]); + cell.write(".git/HEAD", "ref: refs/heads/main\n"); + cell.write(MODULE, &module); + // The project again, below each directory the tags name. + let named = |dir: &&str| module.contains(&format!("//{dir}:")); + for dir in ["app", "lib"].into_iter().filter(named) { + for file in ["App.csproj", LOCK] { + let text = std::fs::read(cell.path(file)).expect(file); + cell.write(&format!("{dir}/{file}"), text); + } + } + match added { + "" => {} + "NuGet.Config" => cell.write(added, fixture_file_as_is(CONFIG)), + _ => cell.write(added, empty), + } + let before = cell.tree(); + let args: Vec<&str> = match cwd { + "" => SCAN.to_vec(), + _ => SCAN.iter().copied().chain(["--cwd", cwd]).collect(), + }; + let scan = socket_patch(&cell, &standin, home.path(), &args); + let said = format!("{test}: {cwd:?} {module}\n{}", scan.dump); + assert_eq!(scan.code, 0, "{said}"); + assert_eq!(changed(&before, &cell.tree()), rewritten, "{said}"); + let (action, code) = match warned.is_empty() { + true => ("applied", ""), + false => ("skipped", "redirect_unconfirmed"), + }; + let event = [(purl.clone(), action.to_string(), code.to_string())]; + assert_eq!(event_triples(&scan.envelope), event, "{said}"); + let warnings = &scan.envelope["warnings"]; + let Some((code, words)) = warned.split_first() else { + let codes = codes_in(warnings); + assert!(!codes.iter().any(|code| code.contains("bazel")), "{said}"); + continue; + }; + assert_eq!(codes_in(warnings), [*code], "{said}"); + let detail = warnings[0]["detail"].as_str().unwrap_or_default(); + for words in words { + assert!(detail.contains(words), "{said}"); + } + } +} + /// Without a Bazel, what stays fixed now that no id and no sum is /// committed: the fixture's directories and the legs of its workflow are /// those of its table; every era names the two files the hub reads; the @@ -843,6 +1278,8 @@ fn a_checkout_names_and_pins_the_packages_of_its_run() { ] { assert!(module.contains(&named), "{}: no {named}", path.display()); } + // The one line a flow takes out for a tag without the config. + assert_eq!(module.matches(NAMES_CONFIG).count(), 1, "{module}"); } let common = root.join("common"); let read = |rel: &str| std::fs::read_to_string(common.join(rel)).expect(rel); diff --git a/crates/socket-patch-core/src/bazel/mod.rs b/crates/socket-patch-core/src/bazel/mod.rs index f2d19a28d..e83a2a891 100644 --- a/crates/socket-patch-core/src/bazel/mod.rs +++ b/crates/socket-patch-core/src/bazel/mod.rs @@ -11,6 +11,7 @@ pub const WORKSPACE_FILES: [&str; 4] = pub mod built; pub mod ignore; pub mod maven; +pub mod nuget; pub mod rc; pub mod starlark; pub mod workspace; diff --git a/crates/socket-patch-core/src/bazel/nuget.rs b/crates/socket-patch-core/src/bazel/nuget.rs new file mode 100644 index 000000000..da4b8811e --- /dev/null +++ b/crates/socket-patch-core/src/bazel/nuget.rs @@ -0,0 +1,1054 @@ +//! Whether the Bazel build of a workspace takes what hosted mode writes +//! for a NuGet package. +//! +//! rules_csharp hands NuGet packages to Bazel through one tag of a module +//! extension. `nuget.from_lock(packages_lock_json = ...)` downloads every +//! package of a `packages.lock.json` and fails the build on a file that +//! does not match the lock's `contentHash`. All the tags of the extension +//! make one hub (`_nuget_impl` in `csharp/extensions.bzl` at 0.5.0): +//! +//! - It has one list of package sources, asked before nuget.org. Each tag +//! in turn puts its `feeds` in the place of the list, and then the +//! sources of the `nuget.config` it names as `nuget_config`, when that +//! file has any. So the last tag with either decides. +//! - It holds a package to the `contentHash` of the first lock that names +//! it. +//! +//! Hosted mode re-pins the `contentHash` of a patched package in the +//! project's locks and names the Socket source of the patch in the +//! project's `nuget.config`. A hub that reads a re-pinned lock and not +//! that file downloads the upstream package and refuses it; one that reads +//! the file and has the package from a lock this run does not re-pin +//! downloads the patched package and refuses that. Either way the command +//! exited 0 and the next `bazel build` failed (measured on Bazel 9.3.2). +//! So [`withheld`] reads what the hub makes of a project, and +//! [`Hub::against`] tells hosted mode when to write nothing for a package, +//! and what to say instead. +//! +//! The answer is read from the checkout through [`super::workspace`]: +//! `MODULE.bazel` and the files it includes, and the locks the tags name. +//! Nothing is evaluated and no Bazel is run. The ruleset has no +//! `WORKSPACE` form, so that file is not looked at. + +use std::path::Path; + +use serde::{Deserialize, Serialize}; + +use super::starlark::Value; +use super::workspace::{disk, disk_up_to, Entry, Extension, Repo, Tag, Workspace}; +use crate::formats::nuget::lock::{locked_at, nuget_lock_entries, parse_lock}; +use crate::formats::nuget::parse_config; +use crate::patch::redirect::NUGET_CONFIG_FILE_NAMES as NAMES; + +/// The extension whose `from_lock` tags make the hub. +pub const EXTENSION: Extension<'static> = Extension { + module: "rules_csharp", + file: "//csharp:extensions.bzl", + name: "nuget", +}; + +/// The warnings of a hosted run for a NuGet package it wrote nothing for: +/// the hub would not download from the Socket source, or it has the +/// package from a lock that the run does not re-pin. +pub const CONFIG_UNDECLARED: &str = "redirect_bazel_nuget_config_undeclared"; +pub const LOCK_NOT_REPINNED: &str = "redirect_bazel_nuget_lock_not_repinned"; + +/// The key under which a hosted run hands the answer of [`withheld`] to +/// the NuGet rewriter among its candidate files: no path of a file. +pub const WITHHELD_KEY: &str = ""; + +/// The most a lock that a tag names may hold, to be read. +const MAX_LOCK: usize = 16 << 20; + +/// What stands against a tag that is not written out, and what a warning +/// ends with in a workspace where a tag has `local_package_dirs`. +const NOT_PLAIN: &str = "is not a call of its own that names its `nuget_config` in a plain string"; +const DIRS: &str = "; rules_csharp asks the directories of `local_package_dirs` before every \ + source, so the upstream package must not lie in one"; +/// What the build does with a re-pinned lock and without the Socket source. +const WITHOUT: &str = + ", where hosted mode names the Socket source of a patch; without that source \ + rules_csharp downloads the upstream package, and the Bazel build fails on \ + the `contentHash` that hosted mode re-pins in the lock"; + +/// What a lock warning says of a tag whose lock is not named plainly +/// beside one that names the project's, and the way on of each kind. +const UNNAMED: &str = "names its lock in no plain string, so that lock may hold the package and \ + may be one this run does not re-pin"; +const NAME_IT: &str = "Write its `packages_lock_json` as a label in a plain string and run this \ + command again"; +const ABOVE_BOTH: &str = "Run this command in a directory above both locks that holds a solution \ + file or a `nuget.config` (`--cwd`): there it re-pins the lock of every \ + project"; + +/// What the hub of a Bazel workspace makes of one project. +#[derive(Debug, PartialEq, Serialize, Deserialize)] +pub struct Hub { + /// Why the hub does not take its package sources from the project's + /// `nuget.config`, as the warning of a package says it after its name. + /// `None`: the tags show that it does. + pub why: Option, + /// [`Self::why`] stands against every package of the run, not only + /// against one that a lock the hub reads holds: the hub does read the + /// config, which must not get a source yet. + pub every: bool, + /// The lock of each tag that names one this reader can tell, in the + /// order of the tags. + pub locks: Vec, + /// That order is the one the hub reads them in: every tag is in one + /// file, where the reader lists the calls as Bazel runs them. + pub ordered: bool, +} + +/// The lock a tag reads. +#[derive(Debug, PartialEq, Serialize, Deserialize)] +pub enum Read { + /// A lock of the project, as the project names it: one this run + /// re-pins. + Ours(String), + /// Another file of the checkout: where its tag is, its path from the + /// Bazel root, and its text (`None`: it cannot be read). + Other(String, String, Option), + /// A label that cannot be read, and where its tag is: any lock. Where + /// no tag names a lock of the project it is taken for the project's; + /// beside a tag that does, it is another lock, which cannot be read. + Any(String), +} + +impl Hub { + /// Why hosted mode writes nothing for the package `id` at the + /// normalized version `version`, whose patched `contentHash` is + /// `hash`: the code of the warning, and what it says after the + /// package's name. `None`: it writes what it writes without Bazel. + /// `pinned` says whether a lock of the project (`None`: any of them) + /// holds the package at that version, which is what the run re-pins. + /// + /// First the locks this run does not re-pin: one that holds the + /// package at that version with another `contentHash`, or cannot be + /// read, stands against the pin, unless the hub has the package from + /// a lock before it. That is asked for every package, also where the + /// hub does not read the config yet: the run names the package's + /// source in the config that the other warning says to put on a tag, + /// and the hub would then take the patched package against that lock. + /// Then the config ([`Self::why`]), for a package that the run re-pins + /// in a lock the hub reads. A package in no such lock is not the + /// hub's: its pin changes nothing a build reads. + /// + /// `docs` are [`Self::docs`], read once for all the packages of a run. + pub fn against( + &self, + docs: &[Option], + pinned: &dyn Fn(Option<&str>) -> bool, + id: &str, + version: &str, + hash: &str, + ) -> Option<(&'static str, String)> { + let named = self.locks.iter().any(|lock| matches!(lock, Read::Ours(_))); + let (mut read, mut first) = (false, false); + for (lock, doc) in self.locks.iter().zip(docs) { + let (at, reads, way) = match lock { + Read::Any(_) if !named => { + read |= pinned(None); + continue; + } + Read::Ours(lock) => { + let ours = pinned(Some(lock)); + (read, first) = (read || ours, first || ours); + continue; + } + _ if first && self.ordered => continue, + Read::Any(at) => (at, UNNAMED.to_string(), NAME_IT), + Read::Other(at, path, _) => { + let state = match doc { + None => "cannot be read and may hold the package", + Some(doc) + if locked_at(doc, id, version) + .any(|e| e.content_hash != Some(hash)) => + { + "holds the package as well" + } + Some(doc) => { + first |= nuget_lock_entries(doc).any(|e| e.id.eq_ignore_ascii_case(id)); + continue; + } + }; + let reads = + format!("reads {path}, which {state} and which this run does not re-pin"); + (at, reads, ABOVE_BOTH) + } + }; + let why = format!( + "{at}: this `nuget.from_lock` {reads}; rules_csharp holds a package to the \ + `contentHash` of the first lock that names it, so the Bazel build can fail on \ + the patched package. So hosted mode wrote nothing for it. {way}" + ); + return Some((LOCK_NOT_REPINNED, why)); + } + let why = self.why.clone().filter(|_| read || self.every)?; + Some((CONFIG_UNDECLARED, why)) + } + + /// The locks of the other tags as documents, one for each of + /// [`Self::locks`]: `None` for one that is not such a lock, has no + /// text or is no JSON. + pub fn docs(&self) -> Vec> { + let doc = |lock: &Read| match lock { + Read::Other(_, _, Some(text)) => parse_lock(text).ok(), + _ => None, + }; + self.locks.iter().map(doc).collect() + } +} + +/// What the rules_csharp hub of the Bazel workspace makes of the project +/// at `project`, whose `nuget.config` is `config` and whose locks are +/// `locks` (paths from the project, with `/` between their parts): a +/// [`Hub`] as JSON, for [`WITHHELD_KEY`]. `None`: hosted mode writes what +/// it writes without Bazel. +/// +/// `None` for a project in no Bazel workspace ([`super::bazel_root`]) and +/// for one without a lock. `None` as well unless a `nuget.from_lock` call +/// that the reader lists names one of the locks, or names its lock in a +/// way that cannot be read (a computed label, a `**` dict): a project +/// whose lock no tag reads is not Bazel's to build. +/// +/// `config` is taken as the project's directory spells it: a disk that +/// folds case answers to every spelling, and a label has to name the file +/// as a disk that does not would. A label that differs from it only in +/// case names the same file, unless the directory holds both, or holds a +/// spelling that is not NuGet's on a disk that does not fold case, where +/// the rewriter writes a file of its own beside it. +/// +/// Not seen: a tag of another Bazel module, whose sources and locks the +/// hub takes as well; a tag that an unreadable part of the workspace +/// hides; a lock of another repository; and what a directory of +/// `local_package_dirs` holds, which the hub asks before every source. +pub fn withheld(project: &Path, config: &str, locks: &[String]) -> Option { + if locks.is_empty() { + return None; + } + let root = super::bazel_root(project)?; + // Bazel has `/` between the parts of a path on every system. + let dir = root.project.components(); + let dir: String = dir + .map(|part| format!("{}/", part.as_os_str().to_string_lossy())) + .collect(); + let listed = std::fs::read_dir(project).into_iter().flatten().flatten(); + let spelled: Vec = listed + .filter_map(|entry| entry.file_name().into_string().ok()) + .filter(|name| name.eq_ignore_ascii_case(config)) + .collect(); + let exact = spelled.iter().any(|name| name == config); + // A spelling in another case is the file the rewriter writes only + // where the disk folds case: there its own name finds the file. + let folds = !exact && project.join(config).exists(); + let strict = spelled.len() > 1 || !(exact || folds || spelled.is_empty()); + let config = spelled + .first() + .filter(|_| folds) + .map_or(config, |name| name); + let read = disk_up_to(&root.dir, MAX_LOCK); + let sources = match read(&format!("{dir}{config}")) { + Entry::Text(text) => parse_config(&text).is_some_and(|c| !c.sources.is_empty()), + _ => false, + }; + let workspace = Workspace::read(&disk(&root.dir)); + let mut hub = judge(&workspace, &dir, config, locks, strict, sources)?; + for lock in &mut hub.locks { + if let Read::Other(_, path, text) = lock { + *text = match read(path) { + Entry::Text(text) => Some(text), + _ => None, + }; + } + } + serde_json::to_string(&hub).ok() +} + +/// [`withheld`] for the workspace `ws` and the project in its directory +/// `dir` (empty, or with a `/` at its end), before the locks of the other +/// tags are read. `exact`: a label names `config` only in its own case. +/// `sources`: `config` has package sources of its own. +/// +/// The hub reads `config` when the last tag that has a `nuget_config` or +/// `feeds` names it, as a label in a string; where the tags are in more +/// than one file, their order is not known, and every tag that has either +/// must name it. A tag that is not a plain call of its own, or whose +/// `nuget_config` is no string, makes that uncertain, and an uncertain +/// reading withholds: a build that may break outranks a patch that is +/// missed and explained. A doubt of the reader elsewhere in the files can +/// only hide a tag, and changes nothing here. +/// +/// A config without a package source does not replace the `feeds` of the +/// tags (`if parsed_sources: feeds = parsed_sources`), and it would once +/// hosted mode names the Socket source in it: where a tag has `feeds`, +/// that is withheld as well, until the config names them. +fn judge( + ws: &Workspace, + dir: &str, + config: &str, + locks: &[String], + exact: bool, + sources: bool, +) -> Option { + let tags = ws.tags(&EXTENSION, "from_lock"); + let given = |tag: &Tag, attribute: &str| tag.call.arg(attribute).is_some(); + // The file of the checkout that a tag names in a plain string. + let file = |tag: &Tag, attribute: &str| match tag.call.arg(attribute) { + Some(Value::Str(label)) => ws.label(label).filter(|label| label.repo == Repo::Main), + _ => None, + }; + let at = |tag: &Tag| format!("{}:{}", tag.file, tag.call.line); + let read = |tag: &Tag| match tag.call.arg("packages_lock_json") { + Some(Value::Str(_)) => { + let path = file(tag, "packages_lock_json")?.path(); + let ours = |lock: &&String| path.eq_ignore_ascii_case(&format!("{dir}{lock}")); + Some(match locks.iter().find(ours) { + Some(lock) => Read::Ours(lock.clone()), + None => Read::Other(at(tag), path, None), + }) + } + Some(_) => Some(Read::Any(at(tag))), + None => tag.call.unpacks().then(|| Read::Any(at(tag))), + }; + let reads: Vec> = tags.calls.iter().map(read).collect(); + let mine = |read: &Option| matches!(read, Some(Read::Ours(_) | Read::Any(_))); + let reader = &tags.calls[reads.iter().position(mine)?]; + + let config = format!("{dir}{config}"); + let ours = |tag: &&Tag| { + let named = file(tag, "nuget_config").map(|label| label.path()); + named.is_some_and(|path| path == config || !exact && path.eq_ignore_ascii_case(&config)) + }; + let sets = |tag: &&Tag| given(tag, "nuget_config") || given(tag, "feeds"); + let ordered = tags.calls.iter().all(|tag| tag.file == reader.file); + // The tag that stands against the config: the last that gives the hub + // its sources, or the first of them where the order is not known. + let other = match tags.calls.iter().rfind(sets) { + Some(last) if ordered => Some(last).filter(|tag| !ours(tag)), + _ => tags.calls.iter().filter(sets).find(|tag| !ours(tag)), + }; + let plain = |tag: &&Tag| { + let config = matches!(tag.call.arg("nuget_config"), None | Some(Value::Str(_))); + tag.call.whole && !tag.call.unpacks() && config + }; + let vague = tags.calls.iter().find(|tag| !plain(tag)); + let shown = other.is_none() && tags.calls.iter().any(|tag| sets(&tag)); + // The tag whose `feeds` the config would replace once it has a source. + let fed = tags.calls.iter().find(|tag| given(tag, "feeds")); + let fed = fed.filter(|_| shown && !sources); + let locks: Vec = reads.into_iter().flatten().collect(); + let every = vague.is_none() && fed.is_some(); + let hub = |why, locks| Hub { + why, + every, + locks, + ordered, + }; + if vague.is_none() && shown && fed.is_none() { + // The config is read: only another lock can stand against a pin. + return (locks.len() > 1).then(|| hub(None, locks)); + } + + // The label of `config` for the tag, in the package of the lock, when + // the tag names the lock plainly and `config` lies in that package: + // one that Bazel reads already. Elsewhere the package of `config` is + // not known, and the answer names the file. + let package = file(reader, "packages_lock_json").map(|label| label.package); + let inside = |package: &String| match config.strip_prefix(package.as_str()) { + Some(rest) => package.is_empty() || rest.starts_with('/'), + None => false, + }; + let attribute = match package.as_ref().filter(|package| inside(package)) { + Some(package) => { + let name = config[package.len()..].trim_start_matches('/'); + format!("`nuget_config = \"//{package}:{name}\"`") + } + None => format!("a `nuget_config` that names {config}"), + }; + let put = format!("Put {attribute} on that tag"); + let again = format!( + " and run this command again: it then names the source in {config}, which it creates \ + where there is none" + ); + let unread = format!("so rules_csharp does not read {config}"); + let tag = vague.or(other).or(fed).unwrap_or(reader); + let mut harm = WITHOUT.to_string(); + let (why, way) = match tag.call.arg("nuget_config") { + _ if vague.is_some() => ( + format!("{NOT_PLAIN}, so socket-patch cannot tell whether rules_csharp reads {config}"), + format!("Write that plainly, with {attribute} on the tag,{again}"), + ), + _ if other.is_none() && fed.is_some() => { + harm = format!( + ": with the Socket source that hosted mode names there, rules_csharp takes the \ + package sources of {config} in place of those `feeds`, and the Bazel build \ + cannot download a package that only they hold" + ); + ( + format!("has `feeds`, and {config} names no package source"), + format!("Name those `feeds` in {config} and run this command again"), + ) + } + Some(Value::Str(theirs)) => { + // A config of NuGet's in a directory above the project's is + // one the same command writes, when it runs there. + let above = file(tag, "nuget_config").is_some_and(|label| { + let path = label.path(); + let (up, name) = path.split_at(path.rfind('/').map_or(0, |slash| slash + 1)); + let nuget = NAMES.iter().any(|nuget| nuget.eq_ignore_ascii_case(name)); + nuget && dir.len() > up.len() && dir.starts_with(up) + }); + let way = match above { + true => "Run this command in the directory of that file (`--cwd`): it then names \ + the source there" + .to_string(), + false => format!( + "{put}, name in {config} the package sources of `{theirs}` that the build \ + needs (rules_csharp takes its package sources from one file){again}" + ), + }; + let why = format!("names `{theirs}` as its `nuget_config`, not {config}"); + (why, way) + } + _ if given(tag, "feeds") => ( + format!("has `feeds` and no `nuget_config`, {unread}"), + format!( + "{put}, name its `feeds` in {config} (rules_csharp takes the package sources of a \ + `nuget_config` in place of `feeds`){again}" + ), + ), + _ => ( + format!("names no `nuget_config`, {unread}"), + format!("{put}{again}"), + ), + }; + // What a directory holds is asked before every source, and not seen. + let held = |tag: &Tag| given(tag, "local_package_dirs"); + let dirs = tags.calls.iter().any(held); + let dirs = if dirs { DIRS } else { "" }; + let why = format!( + "{}: this `nuget.from_lock` {why}{harm}. So hosted mode wrote nothing for it. {way}, and \ + the Bazel build takes the patched package{dirs}", + at(tag) + ); + Some(hub(Some(why), locks)) +} + +#[cfg(test)] +mod tests { + use super::super::workspace::Entry; + use super::*; + + /// The tag of the fixture that hosted mode patches on a real Bazel, and + /// the same tag without its `nuget_config`. + const WIRED: &str = r#"nuget.from_lock( + nuget_config = "//:nuget.config", + packages_lock_json = "//:packages.lock.json", +)"#; + const BARE: &str = "nuget.from_lock(packages_lock_json = \"//:packages.lock.json\")"; + /// How a file that asks for the extension starts. + const PROXY: &str = + "nuget = use_extension(\"@rules_csharp//csharp:extensions.bzl\", \"nuget\")\n"; + + /// A root `MODULE.bazel` that asks for the extension as `nuget` and + /// goes on with `rest`. + fn module(rest: &str) -> String { + format!( + "module(name = \"shop\", version = \"0.0.0\")\n\ + bazel_dep(name = \"rules_csharp\", version = \"0.5.0\")\n\ + {PROXY}{rest}\nuse_repo(nuget, \"nuget\")\n" + ) + } + + /// What [`judge`] makes of the project in the directory `dir` (empty, + /// or with a `/` at its end), with its `nuget.config` and its one + /// lock, in the workspace of `files`. + fn hub(files: &[(&str, &str)], dir: &str) -> Option { + let view = |path: &str| match files.iter().find(|(name, _)| *name == path) { + Some((_, text)) => Entry::Text(text.to_string()), + None => Entry::Absent, + }; + let lock = ["packages.lock.json".to_string()]; + judge( + &Workspace::read(&view), + dir, + "nuget.config", + &lock, + false, + true, + ) + } + + /// What stands against the config there ([`Hub::why`]). + fn said(files: &[(&str, &str)], dir: &str) -> Option { + hub(files, dir)?.why + } + + /// [`said`] for the project at the root of a workspace whose + /// `MODULE.bazel` is [`module`] with `rest`. + fn at_root(rest: &str) -> Option { + said(&[("MODULE.bazel", &module(rest))], "") + } + + /// How every answer goes on after what stands against the config, and + /// how one ends that has the same command write the project's config. + const WHERE: &str = ", where hosted mode names the Socket source of a patch; without that \ + source rules_csharp downloads the upstream package, and the Bazel build \ + fails on the `contentHash` that hosted mode re-pins in the lock. So \ + hosted mode wrote nothing for it. "; + const AGAIN: &str = " and run this command again: it then names the source in nuget.config, \ + which it creates where there is none, and the Bazel build takes the \ + patched package"; + const ATTRIBUTE: &str = "`nuget_config = \"//:nuget.config\"`"; + + #[test] + fn hosted_mode_writes_where_the_hub_reads_the_config_or_no_tag_reads_the_lock() { + let another = "nuget.from_lock(packages_lock_json = \"//api:packages.lock.json\")"; + let theirs = WIRED.replace("//:", "//api:"); + let feeds = "feeds = [\"https://feed.example/v3/index.json\"],\n nuget_config"; + for rest in [ + // The tag names the config: as the fixture has it, by the name + // of the root module, by the main repository, in another case. + WIRED.to_string(), + WIRED.replace("//:nuget", "@shop//:nuget"), + WIRED.replace("//:nuget", "@//:nuget"), + WIRED.replace("nuget.config", "NuGet.Config"), + // And with a lock that cannot be read. + WIRED.replace("\"//:packages.lock.json\"", "LOCK"), + // No tag reads the lock: there is none, it reads the lock of + // another project or of another repository, or it names none. + String::new(), + another.to_string(), + BARE.replace("//:", "@other//:"), + "nuget.from_lock()".to_string(), + // A doubt of the reader can only hide a tag: it changes + // nothing, whether a tag reads the lock or not. + format!("{another}\nMORE = [nuget]"), + format!("{WIRED}\nMORE = [nuget]"), + format!("{WIRED}\ninclude(NAME)"), + format!("{WIRED}\ninclude(\"//:gone.MODULE.bazel\")"), + // One list of sources for the whole extension, and the last + // tag that has any decides: another tag names the config, the + // project's own tag comes after one that names another file, + // and the config of a tag stands in the place of its feeds. + format!("{BARE}\n{}", WIRED.replace("//:p", "//api:p")), + format!("{theirs}\n{WIRED}"), + format!("{}\n{WIRED}", another.replace(")", ", feeds = [])")), + WIRED.replace("nuget_config", feeds), + // What a directory holds is not seen. + WIRED.replace( + "nuget_config", + "local_package_dirs = [\"/opt\"],\n nuget_config", + ), + ] { + assert_eq!(at_root(&rest), None, "{rest}"); + } + // A `bazel_dep` that cannot be read, beside the fixture's tag. + let renamed = module(WIRED).replace( + "\"0.5.0\")", + "\"0.5.0\", repo_name = \"rules_\" + \"csharp\")", + ); + assert_eq!(said(&[("MODULE.bazel", &renamed)], ""), None); + // Another extension of that name, and a workspace with no tag of + // the extension, with a `WORKSPACE` file beside it. + let theirs = module(BARE).replace("@rules_csharp//csharp", "@rules_other//csharp"); + assert_eq!(hub(&[("MODULE.bazel", &theirs)], ""), None); + let legacy = [ + ("MODULE.bazel", ""), + ("WORKSPACE", "nuget_repo(name = \"nuget\")\n"), + ]; + assert_eq!(hub(&legacy, ""), None); + } + + #[test] + fn a_tag_that_reads_the_lock_without_the_config_withholds_and_names_the_way_on() { + let none = "this `nuget.from_lock` names no `nuget_config`, so rules_csharp does not read \ + nuget.config"; + let bare = format!("MODULE.bazel:4: {none}{WHERE}Put {ATTRIBUTE} on that tag{AGAIN}"); + assert_eq!(at_root(BARE), Some(bare.clone())); + // A tag whose lock cannot be read may read the project's. The + // package of the config is then not known, and the file is named. + let computed = BARE.replace("\"//:packages.lock.json\"", "LOCK"); + let file = bare.replace(ATTRIBUTE, "a `nuget_config` that names nuget.config"); + assert_ne!(file, bare); + assert_eq!(at_root(&computed), Some(file)); + // What a directory of the tag holds is asked before every source. + let dirs = BARE.replace(")", ", local_package_dirs = [\"/opt/packages\"])"); + let first = "; rules_csharp asks the directories of `local_package_dirs` before every \ + source, so the upstream package must not lie in one"; + assert_eq!(at_root(&dirs), Some(bare + first)); + + // A tag with feeds: the config would stand in their place. + let feeds = BARE.replace(")", ", feeds = [\"https://feed.example/v3/index.json\"])"); + let has = "this `nuget.from_lock` has `feeds` and no `nuget_config`, so rules_csharp \ + does not read nuget.config"; + let way = format!( + "Put {ATTRIBUTE} on that tag, name its `feeds` in nuget.config (rules_csharp takes \ + the package sources of a `nuget_config` in place of `feeds`){AGAIN}" + ); + let answer = at_root(&feeds).expect("feeds"); + assert_eq!(answer, format!("MODULE.bazel:4: {has}{WHERE}{way}")); + // Also when it reads another lock, after the project's own tag. + let later = feeds.replace("//:", "//api:"); + let answer = at_root(&format!("{WIRED}\n{later}")).expect("later feeds"); + assert_eq!(answer, format!("MODULE.bazel:8: {has}{WHERE}{way}")); + + // A tag that names another file, in this repository or in another: + // the answer names it as it is written, and where. Its sources + // have to go with the tag. + let names = |other: &str| { + format!( + "this `nuget.from_lock` names `{other}` as its `nuget_config`, not nuget.config" + ) + }; + let way = |other: &str| { + format!( + "Put {ATTRIBUTE} on that tag, name in nuget.config the package sources of \ + `{other}` that the build needs (rules_csharp takes its package sources from one \ + file){AGAIN}" + ) + }; + for other in [ + "//config:nuget.config", + "@other//:nuget.config", + ":nuget.config.in", + ] { + let answer = at_root(&WIRED.replace("//:nuget.config", other)).expect(other); + let whole = format!("MODULE.bazel:4: {}{WHERE}{}", names(other), way(other)); + assert_eq!(answer, whole); + } + // The last tag that has sources decides: one that reads another + // lock and names another config, after the project's own tag. + let theirs = WIRED.replace("//:", "//api:"); + let other = "//api:nuget.config"; + let last = format!("MODULE.bazel:8: {}{WHERE}{}", names(other), way(other)); + assert_eq!(at_root(&format!("{WIRED}\n{theirs}")), Some(last)); + // In two files their order is not known: the tag that names another + // config stands against the pin wherever it is. + let included = format!("{PROXY}{WIRED}\n"); + let root = module(&format!("{theirs}\ninclude(\"//:more.MODULE.bazel\")")); + let files = [ + ("MODULE.bazel", root.as_str()), + ("more.MODULE.bazel", included.as_str()), + ]; + let first = format!("MODULE.bazel:4: {}{WHERE}{}", names(other), way(other)); + assert_eq!(said(&files, ""), Some(first)); + } + + #[test] + fn a_config_above_the_project_is_written_by_a_run_in_its_directory() { + let tag = |config: &str| { + let lock = "packages_lock_json = \"//svc/app:packages.lock.json\""; + module(&format!( + "nuget.from_lock(nuget_config = \"{config}\", {lock})" + )) + }; + let there = "Run this command in the directory of that file (`--cwd`): it then names the \ + source there, and the Bazel build takes the patched package"; + for (config, above) in [ + ("//:nuget.config", true), + ("//svc:NuGet.Config", true), + ("//:svc/nuget.config", true), + // Beside the project, below it, and no config of NuGet's. + ("//svc/web:nuget.config", false), + ("//svc/app/obj:nuget.config", false), + ("//:nuget.config.in", false), + ] { + let said = said(&[("MODULE.bazel", &tag(config))], "svc/app/").expect(config); + let names = format!( + "MODULE.bazel:4: this `nuget.from_lock` names `{config}` as its `nuget_config`, \ + not svc/app/nuget.config{WHERE}" + ); + assert!(said.starts_with(&names), "{said}"); + assert_eq!(said.ends_with(&format!(". {there}")), above, "{said}"); + let put = "Put `nuget_config = \"//svc/app:nuget.config\"` on that tag, name in \ + svc/app/nuget.config the package sources of"; + assert_eq!(said.contains(put), !above, "{said}"); + } + } + + #[test] + fn the_config_is_named_by_a_label_of_the_workspace() { + // A project below the root whose directory is a package of its + // own, part of the root package, and part of the package above it; + // the tag in a file the root includes. + for (lock, label) in [ + ("//svc/app:packages.lock.json", "//svc/app:nuget.config"), + ("//:svc/app/packages.lock.json", "//:svc/app/nuget.config"), + ("//svc:app/packages.lock.json", "//svc:app/nuget.config"), + ] { + let tag = format!("nuget.from_lock(packages_lock_json = \"{lock}\")"); + let included = format!("{PROXY}{tag}\n"); + let root = ("MODULE.bazel", "include(\"//svc:nuget.MODULE.bazel\")\n"); + let files = [root, ("svc/nuget.MODULE.bazel", included.as_str())]; + let answer = said(&files, "svc/app/").expect(lock); + let starts = "svc/nuget.MODULE.bazel:2: this `nuget.from_lock` names no"; + assert!(answer.starts_with(starts), "{answer}"); + let names = "does not read svc/app/nuget.config, where"; + assert!(answer.contains(names), "{answer}"); + let way = format!(". Put `nuget_config = \"{label}\"` on that tag and run"); + assert!(answer.contains(&way), "{lock}: {answer}"); + // With it, nothing is withheld, and neither is the project + // beside it, whose lock no tag reads. + let wired = tag.replace("(", &format!("(nuget_config = \"{label}\", ")); + let included = included.replace(&tag, &wired); + let files = [root, ("svc/nuget.MODULE.bazel", included.as_str())]; + for dir in ["svc/app/", "svc/web/", ""] { + assert_eq!(hub(&files, dir), None, "{lock} {dir}"); + } + } + // A lock below the config's directory, in a package of its own. + let tag = "nuget.from_lock(packages_lock_json = \"//svc/app:packages.lock.json\")"; + let view = |path: &str| match path { + "MODULE.bazel" => Entry::Text(module(tag)), + _ => Entry::Absent, + }; + let (ws, lock) = ( + Workspace::read(&view), + ["app/packages.lock.json".to_string()], + ); + let said = judge(&ws, "svc/", "nuget.config", &lock, false, true).expect("withheld"); + let way = "Put a `nuget_config` that names svc/nuget.config on that tag"; + assert!(said.why.expect("why").contains(way)); + // A label in another case names the config, unless the directory + // holds both spellings. + let view = |path: &str| match path { + "MODULE.bazel" => Entry::Text(module(&WIRED.replace("nuget.config", "NuGet.Config"))), + _ => Entry::Absent, + }; + let (ws, lock) = (Workspace::read(&view), ["packages.lock.json".to_string()]); + assert_eq!(judge(&ws, "", "nuget.config", &lock, false, true), None); + let said = judge(&ws, "", "nuget.config", &lock, true, true).and_then(|hub| hub.why); + let said = said.expect("another file"); + let names = "names `//:NuGet.Config` as its `nuget_config`, not nuget.config"; + assert!(said.contains(names), "{said}"); + assert!( + said.contains(&format!(". Put {ATTRIBUTE} on that tag, name in")), + "{said}" + ); + } + + #[test] + fn a_tag_that_is_not_written_out_withholds_and_says_so() { + for rest in [ + WIRED.replace("\"//:nuget.config\"", "CONFIG"), + "nuget.from_lock(**TAG)".to_string(), + format!("[{} for _ in [0]]", WIRED.replace('\n', " ")), + format!("True and {}", WIRED.replace('\n', " ")), + // Also beside the project's own tag, which names the config. + format!("{WIRED}\nnuget.from_lock(**TAG)"), + ] { + let said = at_root(&rest).unwrap_or_else(|| panic!("not withheld: {rest}")); + let line = 4 + rest.matches(WIRED).count() * 4; + // The tag that reads the lock names it plainly, or none does. + let attribute = match rest.contains("//:packages.lock.json") { + true => ATTRIBUTE, + false => "a `nuget_config` that names nuget.config", + }; + let whole = format!( + "MODULE.bazel:{line}: this `nuget.from_lock` {NOT_PLAIN}, so socket-patch cannot \ + tell whether rules_csharp reads nuget.config{WHERE}Write that plainly, with \ + {attribute} on the tag,{AGAIN}" + ); + assert_eq!(said, whole, "{rest}"); + } + } + + /// A lock that resolves `id` to version 1.0.0 with `hash`. + fn lock(id: &str, hash: &str) -> Option { + let entry = format!("{{\"resolved\": \"1.0.0\", \"contentHash\": \"{hash}\"}}"); + Some(format!( + "{{\"dependencies\": {{\"net8.0\": {{\"{id}\": {entry}}}}}}}" + )) + } + + #[test] + fn the_locks_of_the_other_tags_are_listed_in_the_order_of_the_hub() { + let lib = WIRED.replace("//:p", "//lib:p"); + let other = |line: usize| { + let at = format!("MODULE.bazel:{line}"); + Read::Other(at, "lib/packages.lock.json".to_string(), None) + }; + let ours = || Read::Ours("packages.lock.json".to_string()); + let read = |rest: &str| hub(&[("MODULE.bazel", &module(rest))], "").expect(rest); + // The config is read: the answer is about the other lock alone. + let computed = BARE.replace("\"//:packages.lock.json\"", "LOCK"); + let listed = read(&format!("{lib}\n{WIRED}\n{computed}")); + assert_eq!((listed.why, listed.ordered), (None, true)); + let unnamed = Read::Any("MODULE.bazel:12".to_string()); + assert_eq!(listed.locks, [other(4), ours(), unnamed]); + // And about the config as well. + let listed = read(&format!("{BARE}\n{}", lib.replace("//:n", "//lib:n"))); + assert_eq!((listed.why.is_some(), listed.ordered), (true, true)); + assert_eq!(listed.locks, [ours(), other(5)]); + // Two files: the order of their tags is not known. + let included = format!("{PROXY}{lib}\n"); + let root = module(&format!("{WIRED}\ninclude(\"//:more.MODULE.bazel\")")); + let files = [ + ("MODULE.bazel", root.as_str()), + ("more.MODULE.bazel", included.as_str()), + ]; + assert!(!hub(&files, "").expect("two files").ordered); + } + + #[test] + fn a_package_that_another_lock_of_the_hub_holds_unpinned_is_withheld() { + let other = |text: Option| Read::Other("M:4".into(), "lib/p.json".into(), text); + let ours = || Read::Ours("p.json".to_string()); + // What the run re-pins: the package in the project's lock, or not. + let against = |locks: Vec, ordered: bool, pinned: bool| { + let why = Some("why".to_string()); + let hub = Hub { + why, + every: false, + locks, + ordered, + }; + let said = hub.against(&hub.docs(), &|_| pinned, "newtonsoft.json", "1.0.0", "NEW"); + said.map(|(code, why)| (code, why.starts_with("M:4: ") || why == "why")) + }; + let any = || Read::Any("M:4".to_string()); + let (config, elsewhere) = ( + Some((CONFIG_UNDECLARED, true)), + Some((LOCK_NOT_REPINNED, true)), + ); + let upstream = || lock("Newtonsoft.Json", "OLD"); + for (locks, ordered, pinned, said) in [ + // The hub has the package from the other lock first. + (vec![other(upstream()), ours()], true, true, elsewhere), + // From the project's lock first, when the order is known. + (vec![ours(), other(upstream())], true, true, config), + (vec![ours(), other(upstream())], false, true, elsewhere), + // The project's lock does not hold it: the other is the first. + (vec![ours(), other(upstream())], true, false, elsewhere), + (vec![any(), other(upstream())], true, true, elsewhere), + // A lock that cannot be read may hold it. + (vec![other(None), ours()], true, true, elsewhere), + (vec![other(Some("{".into())), ours()], true, true, elsewhere), + // The other lock holds the patched package, another package, + // or another version, which is then the one the hub takes + // whatever a later lock holds. + ( + vec![other(lock("Newtonsoft.Json", "NEW")), ours()], + true, + true, + config, + ), + ( + vec![other(lock("Other", "OLD")), ours()], + true, + true, + config, + ), + ( + vec![ + other(upstream().map(|l| l.replace("1.0.0", "2.0.0"))), + other(upstream()), + ], + true, + false, + None, + ), + // A package in no lock that the hub reads and the run re-pins + // is written as without Bazel. + (vec![ours()], true, false, None), + (vec![any()], true, false, None), + (vec![any()], true, true, config), + // Beside a tag that names the project's lock, a tag whose lock + // cannot be read reads another one, which may hold the package + // first. + (vec![any(), ours()], true, true, elsewhere), + (vec![ours(), any()], true, true, config), + (vec![ours(), any()], true, false, elsewhere), + (vec![ours(), any()], false, true, elsewhere), + ] { + assert_eq!(against(locks, ordered, pinned), said); + } + let hub = Hub { + why: None, + every: false, + locks: vec![other(None), ours()], + ordered: true, + }; + let (_, said) = hub + .against(&hub.docs(), &|_| true, "a", "1.0.0", "NEW") + .expect("withheld"); + assert_eq!( + said, + "M:4: this `nuget.from_lock` reads lib/p.json, which cannot be read and may hold the \ + package and which this run does not re-pin; rules_csharp holds a package to the \ + `contentHash` of the first lock that names it, so the Bazel build can fail on the \ + patched package. So hosted mode wrote nothing for it. Run this command in a \ + directory above both locks that holds a solution file or a `nuget.config` (`--cwd`): \ + there it re-pins the lock of every project" + ); + // Where the config is read and no other lock stands against it, + // nothing is withheld. + let read = Hub { + locks: vec![ours()], + ..hub + }; + assert_eq!(read.against(&[None], &|_| true, "a", "1.0.0", "NEW"), None); + // The tag whose lock is not named, before the project's own. + let unnamed = Hub { + locks: vec![any(), ours()], + ..read + }; + let (code, said) = unnamed + .against(&unnamed.docs(), &|_| true, "a", "1.0.0", "NEW") + .expect("withheld"); + assert_eq!(code, LOCK_NOT_REPINNED); + assert_eq!( + said, + "M:4: this `nuget.from_lock` names its lock in no plain string, so that lock may \ + hold the package and may be one this run does not re-pin; rules_csharp holds a \ + package to the `contentHash` of the first lock that names it, so the Bazel build \ + can fail on the patched package. So hosted mode wrote nothing for it. Write its \ + `packages_lock_json` as a label in a plain string and run this command again" + ); + } + + #[test] + fn a_config_without_a_source_does_not_take_the_place_of_feeds() { + let feeds = "feeds = [\"https://feed.example/v3/index.json\"]"; + let both = WIRED.replace("nuget_config", &format!("{feeds},\n nuget_config")); + let earlier = format!( + "nuget.from_lock({feeds}, packages_lock_json = \"//api:packages.lock.json\")\n{WIRED}" + ); + let judged = |rest: &str, sources: bool| { + let view = |path: &str| match path { + "MODULE.bazel" => Entry::Text(module(rest)), + _ => Entry::Absent, + }; + let lock = ["packages.lock.json".to_string()]; + judge( + &Workspace::read(&view), + "", + "nuget.config", + &lock, + false, + sources, + ) + }; + let why = |rest: &str, sources: bool| judged(rest, sources).and_then(|hub| hub.why); + // The config has a source: it stands in the place of the feeds + // already, and hosted mode adds its own. + assert_eq!(judged(&both, true), None); + assert_eq!(why(&earlier, true), None); + // No tag has feeds: the hub has nuget.org alone, with or without it. + assert_eq!(judged(WIRED, false), None); + // It has none: the first source in it would drop the feeds. + for rest in [&both, &earlier] { + assert_eq!( + why(rest, false).expect("withheld"), + "MODULE.bazel:4: this `nuget.from_lock` has `feeds`, and nuget.config names no \ + package source: with the Socket source that hosted mode names there, \ + rules_csharp takes the package sources of nuget.config in place of those \ + `feeds`, and the Bazel build cannot download a package that only they hold. So \ + hosted mode wrote nothing for it. Name those `feeds` in nuget.config and run \ + this command again, and the Bazel build takes the patched package" + ); + } + // The config is the hub's: no package of the run may put a source + // in it, also one that no lock the hub reads holds. + let hub = judged(&both, false).expect("withheld"); + let said = hub.against(&hub.docs(), &|_| false, "a", "1.0.0", "NEW"); + assert_eq!(said.map(|(code, _)| code), Some(CONFIG_UNDECLARED)); + let hub = Hub { + every: false, + ..hub + }; + assert_eq!( + hub.against(&hub.docs(), &|_| false, "a", "1.0.0", "NEW"), + None + ); + } + + #[test] + fn a_project_is_read_from_its_checkout() { + let tmp = tempfile::tempdir().unwrap(); + let root = std::fs::canonicalize(tmp.path()).unwrap(); + let project = root.join("svc/app"); + std::fs::create_dir_all(&project).unwrap(); + std::fs::create_dir(root.join(".git")).unwrap(); + let lock = ["packages.lock.json".to_string()]; + // The config as the rewriter names it: the first of NuGet's + // spellings that the directory answers to. + let read = |project: &Path| { + let named = NAMES.iter().find(|name| project.join(name).exists()); + let hub = withheld(project, named.unwrap_or(&NAMES[0]), &lock)?; + Some(serde_json::from_str::(&hub).expect("a hub")) + }; + // No Bazel workspace. + assert_eq!(read(&project), None); + let tag = "nuget.from_lock(packages_lock_json = \"//svc/app:packages.lock.json\")"; + let lib = tag.replace("svc/app", "lib"); + std::fs::write(root.join("MODULE.bazel"), module(&format!("{tag}\n{lib}"))).unwrap(); + let hub = read(&project).expect("withheld"); + let said = hub.why.expect("why"); + assert!( + said.contains("does not read svc/app/nuget.config"), + "{said}" + ); + // The lock of the other tag is read, when there is one. + let other = |text: Option<&str>| { + let (at, path) = ("MODULE.bazel:5", "lib/packages.lock.json"); + Read::Other(at.into(), path.into(), text.map(String::from)) + }; + assert_eq!(hub.locks, [Read::Ours(lock[0].clone()), other(None)]); + std::fs::create_dir(root.join("lib")).unwrap(); + std::fs::write(root.join("lib/packages.lock.json"), "{}").unwrap(); + assert_eq!( + read(&project).expect("withheld").locks[1], + other(Some("{}")) + ); + // The config is named as the directory spells it. + std::fs::write(project.join("NuGet.Config"), "").unwrap(); + let said = read(&project).and_then(|hub| hub.why).expect("withheld"); + let spelled = "does not read svc/app/NuGet.Config, where"; + assert!(said.contains(spelled), "{said}"); + let label = "Put `nuget_config = \"//svc/app:NuGet.Config\"` on that tag"; + assert!(said.contains(label), "{said}"); + // Where the disk holds two spellings, a tag that names the other + // one does not name the config. + std::fs::write(project.join("nuget.config"), "").unwrap(); + let wired = tag.replace("(", "(nuget_config = \"//svc/app:NuGet.Config\", "); + std::fs::write(root.join("MODULE.bazel"), module(&wired)).unwrap(); + let two = std::fs::read_dir(&project).unwrap().count() == 2; + assert_eq!(read(&project).is_some(), two); + // A spelling that is not NuGet's is the rewriter's file only on a + // disk that folds case; elsewhere the rewriter writes a file of + // its own, and a tag that names the other one does not read it. + for name in ["nuget.config", "NuGet.Config"] { + std::fs::remove_file(project.join(name)).ok(); + } + std::fs::write(project.join("Nuget.Config"), "").unwrap(); + let folds = project.join("nuget.config").exists(); + let odd = wired.replace("NuGet.Config", "Nuget.Config"); + std::fs::write(root.join("MODULE.bazel"), module(&odd)).unwrap(); + assert_eq!(read(&project).is_some(), !folds); + // The config has no source: a tag with `feeds` keeps them until + // it has. + let fed = odd.replace("(", "(feeds = [\"https://feed.example/v3/index.json\"], "); + std::fs::write(root.join("MODULE.bazel"), module(&fed)).unwrap(); + let said = read(&project).and_then(|hub| hub.why).expect("withheld"); + let stands = match folds { + true => "has `feeds`, and svc/app/Nuget.Config names no package source", + false => "as its `nuget_config`, not svc/app/nuget.config", + }; + assert!(said.contains(stands), "{said}"); + let source = "\ + "; + std::fs::write(project.join("Nuget.Config"), source).unwrap(); + assert_eq!(read(&project).is_some(), !folds); + // The project at the root, whose lock the tag does not read. And + // a project without a lock, which no tag can read: not the one + // whose name cannot be read either. + assert_eq!(read(&root), None); + let computed = "nuget.from_lock(packages_lock_json = LOCK)"; + std::fs::write(root.join("MODULE.bazel"), module(computed)).unwrap(); + assert!(read(&root).is_some()); + assert_eq!(withheld(&root, "nuget.config", &[]), None); + } +} diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 09b507b2e..8a930ed5e 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -1281,6 +1281,26 @@ pub async fn read_candidate_files_in( crate::patch::redirect::NUGET_LOCKS_KEY.to_string(), lines.join("\n"), ); + // In a Bazel workspace a rules_csharp hub reads those locks, with + // the locks of its other tags, and takes its package sources from + // the config the rewriter writes only when a tag names it. What + // the hub makes of the project is read from the checkout here and + // rides a synthetic key, for the rewriter to judge each package. + // A project in no Bazel workspace, or whose locks no tag reads, + // gets none, and its run is the one it was. The root is the raw + // one: which files of the workspace this reads is not known + // beforehand, so a recording gives up instead of listing them. + if let Some(root) = view.disk_root() { + use crate::bazel::nuget::{withheld, WITHHELD_KEY}; + let config = crate::patch::redirect::nuget_config_path(&out.files); + let locks = lines.iter().filter_map(|line| line.strip_prefix("lock\t")); + let locks: Vec = locks.map(String::from).collect(); + let root = root.to_path_buf(); + let read = move || withheld(&root, config, &locks); + let read = crate::utils::fs::run_blocking(read).await; + out.files + .extend(read.map(|hub| (WITHHELD_KEY.to_string(), hub))); + } } for path in view.python_lock_paths() { diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 61cea1b9c..ee96c4f09 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -7264,6 +7264,16 @@ pub fn artifact_url_spellings(artifact_url: &str) -> [String; 2] { pub(crate) const NUGET_CONFIG_FILE_NAMES: [&str; 3] = ["nuget.config", "NuGet.config", "NuGet.Config"]; +/// The config file the NuGet rewriter writes among `files`: NuGet reads the +/// first of its spellings present in a directory, and a fresh +/// `nuget.config` beside a `NuGet.config` would shadow it. +pub(crate) fn nuget_config_path(files: &BTreeMap) -> &'static str { + NUGET_CONFIG_FILE_NAMES + .into_iter() + .find(|name| files.contains_key(*name)) + .unwrap_or(NUGET_CONFIG_FILE_NAMES[0]) +} + fn default_nuget_config() -> String { "\n\n \n \n \n\n".to_string() } @@ -7492,12 +7502,7 @@ fn rewrite_nuget( if nuget.is_empty() { return; } - // NuGet reads the first of these spellings present in a directory; a - // fresh `nuget.config` beside a `NuGet.config` would shadow it. - let config_path = NUGET_CONFIG_FILE_NAMES - .into_iter() - .find(|name| files.contains_key(*name)) - .unwrap_or(NUGET_CONFIG_FILE_NAMES[0]); + let config_path = nuget_config_path(files); // The source keys the configs below this one define (the engine reads // them from disk; absent in memory). let inherited: Option> = files @@ -7587,6 +7592,14 @@ fn rewrite_nuget( } } + // What the rules_csharp hub of a Bazel workspace makes of the project + // (the engine reads it from the checkout; absent outside one). + let hub: Option = files + .get(crate::bazel::nuget::WITHHELD_KEY) + .and_then(|hub| serde_json::from_str(hub).ok()); + // The locks of its other tags, parsed once for every dep. + let hub = hub.map(|hub| (hub.docs(), hub)); + for dep in &nuget { // The uuid lands in the source key, the mapping and the set-aside // comment: one carrying markup (`-->`, a quote, `<`) could write live @@ -7735,6 +7748,27 @@ fn rewrite_nuget( continue; } }; + // In a Bazel workspace whose rules_csharp hub would not take this + // pin (it does not read this config, or has the package from a + // lock this run does not re-pin), the build fails on the + // `contentHash` ([`crate::bazel::nuget::Hub::against`]). Nothing + // is written for a dep that every check above would let through. + let pinned = |lock: Option<&str>| { + locks.iter().any(|(rel, _, doc, _)| { + let held = crate::formats::nuget::lock::locked_at(doc, &id_lower, &version_norm); + lock.is_none_or(|lock| lock == rel) && held.count() > 0 + }) + }; + let against = |(docs, hub): &(Vec>, crate::bazel::nuget::Hub)| { + hub.against(docs, &pinned, &id_lower, &version_norm, &content_hash) + }; + if let Some((code, why)) = hub.as_ref().and_then(against) { + result.warnings.push(RewriteWarning { + code: code.into(), + detail: format!("{}@{}: {why}", dep.name, dep.version), + }); + continue; + } if !moved.is_empty() { result.warnings.push(RewriteWarning { code: "redirect_nuget_mapping_set_aside".into(), @@ -32286,6 +32320,73 @@ packages: ); } + /// What the engine hands over of the rules_csharp hub of a Bazel + /// workspace ([`crate::bazel::nuget::withheld`]): a dep the hub would + /// not take is skipped with the hub's word after its name and nothing + /// is written, where the same files without it are pinned. A dep the + /// rewriter refuses for another reason keeps that reason, and one in + /// no lock that the hub reads is pinned as without it. + #[test] + fn nuget_under_a_bazel_hub_that_would_not_take_the_pin_writes_nothing() { + use crate::bazel::nuget::{Hub, Read, WITHHELD_KEY}; + let lock = ("packages.lock.json".to_string(), simple_lock("UPSTREAM==")); + let mut files = BTreeMap::from([lock]); + let pinned = rewrite_registry_redirect(&files, &[nuget_override()]); + let written: Vec<&str> = pinned.files.keys().map(String::as_str).collect(); + assert_eq!(written, ["nuget.config", "packages.lock.json"]); + + let hub = |why: Option<&str>, locks: Vec| { + let (why, ordered) = (why.map(String::from), true); + let hub = Hub { + why, + every: false, + locks, + ordered, + }; + serde_json::to_string(&hub).expect("a hub") + }; + let ours = || Read::Ours("packages.lock.json".to_string()); + let (at, other) = ("MODULE.bazel:8".to_string(), "lib/packages.lock.json"); + let other = Read::Other(at, other.to_string(), Some(simple_lock("UPSTREAM=="))); + let mut refused = nuget_override(); + (refused.patch_uuid, refused.integrity.sha512) = ("other".into(), None); + let missing = "Newtonsoft.Json has no sha512 integrity"; + for (hub, code, starts) in [ + ( + hub(Some("MODULE.bazel:4: why"), vec![ours()]), + "redirect_bazel_nuget_config_undeclared", + "Newtonsoft.Json@13.0.3: MODULE.bazel:4: why", + ), + ( + hub(None, vec![other, ours()]), + "redirect_bazel_nuget_lock_not_repinned", + "Newtonsoft.Json@13.0.3: MODULE.bazel:8: this `nuget.from_lock` reads \ + lib/packages.lock.json, which holds the package as well and which this run", + ), + ] { + files.insert(WITHHELD_KEY.to_string(), hub); + for with_config in [false, true] { + if with_config { + files.insert("nuget.config".to_string(), default_nuget_config()); + } + let r = rewrite_registry_redirect(&files, &[nuget_override(), refused.clone()]); + assert!(r.files.is_empty() && r.edits.is_empty(), "{r:?}"); + let said: Vec<_> = r.warnings.iter().map(|w| (&*w.code, &*w.detail)).collect(); + assert_eq!(said.len(), 2, "{said:?}"); + assert_eq!(said[0].0, code); + assert!(said[0].1.starts_with(starts), "{said:?}"); + assert_eq!(said[1], ("redirect_nuget_missing_sha512", missing)); + } + files.remove("nuget.config"); + } + // The hub reads another lock of the project: this pin is not its. + let elsewhere = vec![Read::Ours("src/packages.lock.json".to_string())]; + let elsewhere = hub(Some("why"), elsewhere); + files.insert(WITHHELD_KEY.to_string(), elsewhere); + let r = rewrite_registry_redirect(&files, &[nuget_override()]); + assert_eq!(r.files, pinned.files); + } + /// A PRESENT but unparseable packages.lock.json refuses the whole nuget /// redirect up front: landing the source + mapping while the lock kept /// the upstream contentHash would NU1403 every restore, with the ledger diff --git a/docs/ecosystems.md b/docs/ecosystems.md index ab1d2884b..8d5c3290e 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -22,7 +22,7 @@ The backticked slug in each row is the value `-e`/`--ecosystems` accepts (e.g. | Go (`golang`) | ✅ `go.mod` `replace` → `.socket/go-patches/` — see [Go: directory replaces and go.sum](#go-directory-replaces-and-gosum) | ✅ `replace` → the committed vendor tree | ✅ (free tier) fork-style `replace` → `patch.socket.dev/gopatch/` + committed `go.sum` pin; see [Go notes](#go-directory-replaces-and-gosum). Paid hosted patches are unsupported; `redirect_golang_unsupported` names the vendored remedy | | Maven (`maven`) — Maven and Gradle | ✅ in place in every copy the build consumes: each `~/.m2` copy it reads and each Gradle `files-2.1` copy; `~/.m2` `.sha1`/`.md5` sidecars are rewritten, Gradle copies get advisories; jar-member records swap in the patch service's whole jar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) and [Gradle](#gradle) | ✅ suffixed Maven repository (`-socket.` pin + `.mvn/maven.config` + fallback file repository) for every pom root, single-module or reactor, or Gradle 6.8+ same-GAV repository with settings wiring and SHA-256 checks (a root with both `pom.xml` and a Gradle build wires both); see [JVM vendoring](design/maven-vendoring.md) and [Gradle](#gradle) | ✅ fail-closed by a Socket-only `-socket.` suffix: pom projects get a pinned `` (`${property}` versions are refused); Gradle 6.8+ builds get an owned settings script, lock-entry rewrites and a resolution tripwire — see [Maven & NuGet caveats](#maven--nuget-caveats) and [Gradle](#gradle) | | sbt / Mill / scala-cli (`maven`) | ✅ Coursier caches (sbt 1.3+, sbt 2, Mill, scala-cli) and Ivy caches (sbt 0.13–1.2, `useCoursier := false`) patched in place, Coursier checksum sidecars resynced — see [Scala build tools](#scala-build-tools-sbt-mill-scala-cli) | ✅ sbt 0.13.18+: generated `socket-patch-vendor.sbt` over the committed suffixed `.socket/vendor/maven2` tree; scala-cli directory builds: owned `socket-patch.scala` + same-GAV `.socket/vendor/coursier` tree (Linux / macOS); Mill: not wired (agent or hosted guidance) | ✅ sbt 0.13.18+: one generated `socket-patch.sbt`, gated on sbt's own `sbt update` records; Mill / scala-cli: paste-able snippets only (`redirect_mill_manual_snippet`, `redirect_scala_cli_manual_snippet`) | -| NuGet (`nuget`) | ✅ in-place patching deletes `.nupkg.metadata` and advises on the `.nupkg.sha512` tamper-evidence sidecar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) | ✅ committed folder feed (its `.gitignore` re-includes the nupkg against `*.nupkg` rules) + `packageSourceMapping` + `packages.lock.json` contentHash pin | ✅ `nuget.config` source + source-mapping, `packages.lock.json` contentHash rewrite. See the locked-mode note in [Maven & NuGet caveats](#maven--nuget-caveats) | +| NuGet (`nuget`) | ✅ in-place patching deletes `.nupkg.metadata` and advises on the `.nupkg.sha512` tamper-evidence sidecar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) | ✅ committed folder feed (its `.gitignore` re-includes the nupkg against `*.nupkg` rules) + `packageSourceMapping` + `packages.lock.json` contentHash pin | ✅ `nuget.config` source + source-mapping, `packages.lock.json` contentHash rewrite. See the locked-mode note in [Maven & NuGet caveats](#maven--nuget-caveats). In a Bazel workspace whose rules_csharp `nuget.from_lock` hub would not take the pin (it reads the lock and not that `nuget.config`, or has the package from a lock the run does not re-pin), nothing is written (`redirect_bazel_nuget_config_undeclared`, `redirect_bazel_nuget_lock_not_repinned`); see [Bazel workspaces: NuGet](#bazel-workspaces-nuget) | | Composer (`composer`) | ✅ in place (`vendor/`) | ✅ `composer.lock` `dist: path` rewrite | ✅ `composer.lock` dist url + shasum rewrite; the entry's `source` and `dist.mirrors` are removed. See [composer-compatibility.md](testing/composer-compatibility.md) | | Deno (`deno`) | ✅ in place (the only mode for Deno) | ❌ refused (`vendor_unsupported_ecosystem`) | ❌ not supported | @@ -1475,6 +1475,127 @@ Honest limits of the Maven and NuGet flows — documented behavior, not bugs: * **NuGet package signatures.** The server constructs the patched `.nupkg` and removes invalidated upstream signatures. The CLI verifies and stores the served archive without repacking it. Vendoring uses a folder feed with source mapping; installations requiring signed packages need an appropriate server artifact and signing policy. +## Bazel workspaces: NuGet + +rules_csharp's `nuget.from_lock` tag (in `MODULE.bazel`) downloads every package of a +`packages.lock.json` and fails the build on a file that does not match the lock's `contentHash`. +All the `nuget.from_lock` tags of a workspace make one hub (rules_csharp 0.5.0): + +- It has one list of package sources, which it asks before nuget.org. Each tag in turn puts + its `feeds` in the place of that list, and then the sources of the `nuget.config` it names + as `nuget_config`. So the last tag that has either decides. +- It holds a package to the `contentHash` of the first lock that names it. + +Hosted mode re-pins the `contentHash` of a patched package in the project's locks and names the +Socket source of the patch in the project's `nuget.config`. A hub that reads a re-pinned lock +and not that file downloads the upstream package and refuses it. A hub that reads the file and +has the package from a lock the run did not re-pin downloads the patched package and refuses +that. Either way the next `bazel build` fails with "the downloaded package does not match its +contentHash in packages.lock.json". + +So `scan --mode hosted` and `get --mode hosted` read the tags, and write nothing for a NuGet +package that the hub would not take: no `nuget.config`, no re-pinned lock. The package is +`skipped` (`redirect_unconfirmed`), the run exits 0 as for any other package hosted mode does +not pin, and one warning per package names the tag by its file and line and the way on. + +### The hub does not read the config + +`redirect_bazel_nuget_config_undeclared`. The common case is a tag that names no +`nuget_config`, and the warning names the attribute to put on it: + +```starlark +nuget.from_lock( + nuget_config = "//:nuget.config", + packages_lock_json = "//:packages.lock.json", +) +``` + +With the attribute, the same command names the Socket source in that `nuget.config`, which it +creates where the project has none, and re-pins the lock, and the next build takes the patched +package from the Socket source (rules_csharp 0.5.0 on Bazel 9.3.2, 8.8.2 and 7.7.2). That +build updates `MODULE.bazel.lock`, which records the two files: commit all three with +`MODULE.bazel`. The label is the one of the project's own config, in the package of the lock's +label: the file of NuGet's spellings (`nuget.config`, `NuGet.config`, `NuGet.Config`) that the +project directory holds, as the directory spells it, or `nuget.config` where it holds none. +Where the tag does not name its lock as a label in a plain string, or the config lies outside +the package of that label, the warning names the file and leaves the label to you. + +The other cases, each with its own way on in the warning: + +- **The tag that decides names another file.** Where that file is a NuGet config in a directory + above the project, as with one `nuget.config` for several projects, run the command in that + directory (`--cwd`): hosted mode then writes the file the tag reads. Otherwise put the + attribute on that tag and name in the project's config the package sources of the other file + that the build needs, because the hub takes its sources from one file. +- **The tag that decides has `feeds` and no `nuget_config`.** Put the attribute on it and name + its feeds in the project's config: the sources of a `nuget_config` stand in the place of + `feeds`. +- **A tag has `feeds`, and the config names no package source.** A config without a source + leaves the `feeds` of the tags as they are; the first source in it takes their place. So + hosted mode does not name the Socket source there until the config names those feeds: name + them and run the command again. +- **A tag is not written out:** it stands in a comprehension or a condition, passes `**`, or + has a `nuget_config` that is a constant or an expression. socket-patch evaluates nothing, so + it cannot tell which file the hub reads, and a build that may break outranks a patch that is + missed and explained. Write the tag as a call of its own, with the attribute in a plain + string. + +### Another lock holds the package + +`redirect_bazel_nuget_lock_not_repinned`. A hosted run re-pins the locks of the projects in the +directory it runs in. Where another tag reads a lock outside that run which holds the package +at the same version with another `contentHash`, and the hub reads that lock first, the hub +would hold the patched package to the upstream `contentHash`. Hosted mode then writes nothing +for the package and names the tag and the lock. Run the command in a directory above both +locks (`--cwd`) that holds a solution file or a `nuget.config`, which is what makes hosted mode +look for projects below it: there it re-pins the lock of every project. Where the tags are in +more than one file their order is not known, and such a lock stands against the pin wherever +its tag is; a lock that cannot be read counts as one that holds the package. So does the lock +of a tag that does not name it as a label in a plain string, beside a tag that names a lock of +the project: the warning says to write that label out. + +### What is read, and what is not + +The tags are read from the checkout: the root `MODULE.bazel` and the files it includes, under +whatever name the file gives the `nuget` extension of `rules_csharp`, and the locks the tags +name. Nothing is evaluated and no Bazel is run. + +- **Nothing changes** for a project in no Bazel workspace (the rule is under + [Bazel workspaces: `.bazelignore`](#bazel-workspaces-bazelignore)), for one that restores + into no lock file, and for one whose locks no `nuget.from_lock` call names, which Bazel does + not build. A tag whose `packages_lock_json` is not a label in a plain string may name any + lock, and counts as one that names the project's where no tag names a lock of the project. + Nothing changes either for a package that no lock of the project which a tag reads holds: + the hub does not build it from this project. +- **Hosted mode writes** when every `nuget.from_lock` is a call of its own and the last one + that has a `nuget_config` or `feeds` names the project's config as a label in a plain string + (`//:nuget.config`, `@//:nuget.config`, or with the root module's own name; in another case + too, unless the directory holds both spellings, or one that is not NuGet's on a disk that + does not fold case, beside which hosted mode writes a file of its own). Where the tags are + in more than one file, + every tag that has either must name the config. A tag that reads another lock and names the + config serves, unless its lock holds the package as above. +- **A part of `MODULE.bazel` that socket-patch cannot read** (a `bazel_dep` whose name is an + expression, an `include` it cannot follow) can hide a tag and changes nothing else: where + the tags that are read show that the hub reads the config, hosted mode writes. +- **Not seen:** a tag of another Bazel module, whose sources and locks the hub takes as well; a + tag that an unreadable part of `MODULE.bazel` hides; a lock in another repository; what the + command line or an rc file tells Bazel; and what a directory of `local_package_dirs` holds. + The hub asks those directories before every source, so with the upstream package in one the + build fails on a re-pinned lock (measured on Bazel 9.3.2). The warning says so in a workspace + that has the attribute; where the tags show that the hub reads the config, hosted mode + writes as it does without Bazel. +- **Not covered:** a release of rules_csharp after 0.5.0 that gives each `name` of + `nuget.from_lock` a hub of its own, where the two rules above hold per name and the tags are + still read as one hub; a Bazel workspace below the directory of the run, whose tags are not + read while the locks of its projects are re-pinned with the others; a project below the root + of a workspace that is in no git checkout, because a workspace is looked for only up to the + root of the project's checkout; and a project whose own locks no tag reads while a tag names + its `nuget.config` and reads another lock that holds the package. Hosted mode writes in each + as it does without Bazel. +- A pin that an earlier release wrote in such a workspace stays in the two files: `socket-patch + rollback` takes it out, and the attribute makes the build read it. + ## Gradle Gradle builds are part of the `maven` ecosystem: patches are Maven PURLs and every diff --git a/scripts/loc-budget.json b/scripts/loc-budget.json index 6729c125c..c4de181da 100644 --- a/scripts/loc-budget.json +++ b/scripts/loc-budget.json @@ -4,7 +4,7 @@ "paths": [ "crates/socket-patch-core/src/bazel/" ], - "max": 2486 + "max": 2766 }, "lock-readers": { "paths": [