diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 2596ee4c8..78c3ee3e5 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -102,6 +102,7 @@ Every subcommand accepts the same set of "global" flags via a single shared `Glo | `--maven-config` | — | — | (recorded choice, else `auto`) | enum: `auto` \| `none` | Maven reactor vendoring: write the repository tail (`auto`) or use only the fallback file repository (`none`). The choice persists in the vendor ledger. | | `--vendor-url` | — | `SOCKET_VENDOR_URL` | (active API/proxy base) | string | Base host for the vendoring-service package-reference request | | `--patch-server-url` | — | `SOCKET_PATCH_SERVER_URL` | (server-returned) | string | Override the host of the prebuilt-archive download URL (local-dev / testing) | +| `--patch-mirror-url` | — | `SOCKET_PATCH_MIRROR_URL` | (none) | string | **Experimental** (hidden from `--help`, no stability guarantee). Base URL of a mirror of the patch server (an Artifactory remote repository). Hosted mode writes its references under it and every command reads them there. See [Patch mirror (experimental)](#patch-mirror-experimental) | | `--offline` | — | `SOCKET_OFFLINE` | `false` | bool | **Strict airgap on every command** — never contact the network | | `--strict` | — | `SOCKET_STRICT` | `false` | bool | Keep rejecting file-content mismatches even with `--allow-patch-conflicts`; mismatches already fail by default without consent. `apply --force` overrides this file check, but never grants consent for declared project patches. | | `--allow-patch-conflicts` | — | — | `false` | bool | Explicit, invocation-only consent to attempt Socket patches alongside project patches/replacements, or overwrite modified installed files. No environment or repository-policy binding. Backend safety refusals still apply. See [Existing project patches](#existing-project-patches). | @@ -410,11 +411,11 @@ Pinned by `tests/scan/scan_proxy_fallback_e2e.rs`, `tests/cli/api_client_errors_ **Vendored entries and the rest of the CLI.** Because nothing is in the manifest, vendored patches are invisible to `apply` (nothing to apply in place) but fully visible to `list` (listed from the ledger, labeled `Mode: vendored (recorded in .socket/vendor/state.json)` in human mode, exit 0 on a vendored-only project), `vex` (attested from the embedded records while a lockfile still wires the artifact — see "Manifest-less VEX"), `repair` (health-checked and rebuilt from the ledger), and `scan --prune` (lockfile-driven reconcile). They are exempt from standalone `vendor`'s manifest reconcile (`reconcile_dropped` never touches `detached` entries) and exit via `remove ` (which reverts them), `vendor --revert`, or `rollback`, whose vendored leg reverts every in-scope ledger entry (unscoped and identifier-scoped runs; path-scoped runs reach them only when an installed copy matches). -`scan --mode hosted` swaps the in-place apply for the registry-redirect pipeline: discover → resolve hosted-patch references (grant token + integrity + per-dep registry override) → rewrite ONLY the patched dependencies' lockfile / registry-config entries to point at the hosted packages. A dep counts as **redirected** only when its hosted-artifact URL (or per-dep registry index URL) actually landed in a project file — a granted reference whose rewriter found nothing to edit is neither counted nor attested. **No ledger (v5.0)**: hosted mode writes ONLY the lockfile / registry-config edits — `.socket/vendor/redirect-state.json` is never written (on success or failure), and a pre-v5 one on disk is ignored (never read for planning, never quarantined, left byte-identical). The lockfiles are the only record of a hosted patch: `list`, `vex`, `rollback`, `remove`, `vendor` and `repair` all discover the hosted pins from them (a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin), and commit-ready output is just the lockfile / config changes. Cargo and golang are confirmed only by their rewriter's own report (`confirmed_cargo_uuids` / `confirmed_golang_uuids`): a golang dep counts only when its go.mod `replace M V => patch.socket.dev/gopatch/ ` and both go.sum lines are in place (a re-run restores the zip line `go mod tidy` drops while no package of the module is loaded, so a later import still builds), never because the patch-server origin or leftover go.sum lines appear somewhere. Gradle is confirmed the same way (`confirmed_gradle_uuids`): only when the final files hold the owned script, the index row, the live apply line in every build's settings file and the suffixed version in every lock entry of the GA (see [Gradle builds](#gradle-builds-v50)). A golang module that go.mod does not require and go.sum has no zip (`h1:` of the module tree) line for at the patched version is outside the build graph and is refused with `redirect_golang_not_in_module_graph` (nothing written); with a `go.work` in effect the build graph is the workspace's (every `use` member's go.mod, go.sum and `go.work.sum`), so a member raising the module refuses the pin with `redirect_golang_version_mismatch`, and a `go.work` that overrides, ignores or rejects the replace refuses it with `redirect_golang_workspace_conflict` before anything is written. Run in the directory holding the `go.work`, a module the workspace shares (no root `go.mod`, or the root module is a `use` member and another member lists it or `go.work` already holds the Socket replace) is pinned in `go.work` and its two sum lines go into `go.work.sum`, the members' files untouched; a member's own `replace` of it, which the go.work pin would override, refuses it (`redirect_golang_workspace_conflict`); a `/go.mod`-only line doesn't count, since go writes one for every version MVS reads (#509). In a go < 1.17 graph, where `require` holds only lower bounds, the build selects the highest of the required version and go.sum's zip-line versions, and a patch for a lower version warns `redirect_golang_version_mismatch` (nothing written). Only the exact module `patch.socket.dev/gopatch/` is socket-owned; any other module path is refused with `redirect_golang_untrusted_module_path`. A vendored golang module is taken over like cargo and the npm family: its vendor wiring, committed copy and ledger entry are reverted first (`redirect_takeover_reverted_vendored`). A vendored PyPI package (requirements.txt, Poetry, Pipenv, uv, Hatch, PDM, pylock) is taken over the same way: its vendored wiring is restored to the recorded registry entry, its ledger entry and wheel are removed, and it is redirected in the same commit. The Python rewriters treat any non-registry source as user-authored, so without the revert they refused socket-patch's own vendored source and left the project vendored. A vendored Composer package is taken over the same way: its `composer.lock` entry is restored to the recorded pre-vendor entry, its ledger entry and committed copy are removed, and it is redirected in the same commit, so the lock matches a direct hosted scan and `rollback` can restore it from packagist. Without the revert the rewriter edited the vendored entry in place and kept its patch-uuid `dist.reference`, which `rollback` refused. A vendored Composer entry the takeover cannot revert (its vendored ledger entry is missing) is left vendored with `redirect_composer_vendored_entry`, never repointed in place. A takeover revert that leaves vendored wiring in place is refused with `redirect_vendored_revert_failed`. That covers a drift-skipped record (`vendor_lock_entry_drifted`) and a reverted file that still references the artifact (`vendor_revert_residual_reference`). The ledger entry and artifact are kept, and the package stays vendored and skipped. `--dry-run` predicts the same refusal from the same signals instead of previewing `redirect_would_revert_vendored`. The hosted requirements.txt rewriter only rewrites an existing pin in the root `requirements.txt`, so a vendored requirements.txt package whose wiring is a pin in a `-r` include or a `(transitive)` line vendored mode appended is kept vendored, wet and `--dry-run` alike, with `redirect_requirements_takeover_unreachable` (a top-level `warnings[]` entry, and the `errorCode` of the package's `skipped` event). Its wiring, ledger entry and wheel are kept, so it stays vendored and patched (exit 0). The uv and Poetry rewriters are covered the same way: a vendored uv package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (earlier releases pinned such an entry down to the patch's version, where vendored uv now refuses the patch with `pypi_uv_lock_package_missing`; the revert brings the lock's own version back, and hosted mode only pins the version the lock resolves) is kept vendored with `redirect_uv_takeover_version_unreachable`, and a vendored Poetry package on a Poetry 0.x lock (which hosted mode refuses outright) is kept vendored with `redirect_poetry_lock_unsupported`. **Staged takeover (v5.0)**: the takeover is atomic. Each purl's vendored revert is staged in memory (the run's group commit, the same journaled commit vendored mode uses), the hosted rewrite plans against the reverted project, and a staged purl the rewrite does not pin is retracted: every staged revert is undone, that purl keeps its vendored wiring, ledger entry and artifact byte-identical, and the rest are staged and rewritten again. A retracted purl is a `skipped` event whose `errorCode` is the cause: its existing skip reason (unavailable wheel metadata, …), the rewriter warning that names the package (`redirect_yarn_berry_missing_checksum`, `redirect_pypi_platform_wheel`, …), `redirect_requirements_takeover_unreachable`, the rewrite's lock-level refusal (`redirect_yarn_berry_mixed_line_endings`, `redirect_bun_lock_unsupported`, a Gradle planner refusal, …) or `redirect_takeover_not_pinned`; that warning is reported in `warnings[]`, followed by `redirect_takeover_kept_vendored` naming the package. Exit 0: the package stays vendored and patched, never unpatched in both modes. The reverts, the hosted pins and the vendored ledger then reach the disk in one commit, and the reverted artifacts are deleted only after it: a refusal or write failure before the commit writes nothing, a failed commit puts back the files it replaced, and a commit interrupted after its journal was written is finished by the next command that takes the apply lock. `--dry-run` runs the same steps and drops the staged state instead of committing, so its `verified` events and warnings are the wet run's `applied` events and warnings. A hosted run with no takeover stages its writes the same way but commits them without the journal (it writes nothing under `.socket/`): a file that fails to be replaced puts back the ones already replaced, so a failed run leaves no lock half-redirected. Re-runs over already-rewritten output plan from the current lock text and are idempotent (exit 0, lock unchanged). **Lock (v5.0)**: the hosted engine acquires `<.socket>/apply.lock` before its first wet write (the staged takeover reverts) — not on `--dry-run`, and not when the run would write nothing (zero redirects, all skipped) — so previews and no-op runs never create `.socket/`; contention is `lock_held` and a lock-file I/O fault (a read-only project root, a file squatting on `.socket/`) is `lock_io` — both exit 1, refused BEFORE any project file is written, and rendered like every other lock holder: human `Error (): ` on stderr (+ the `--lock-timeout` hint for a live holder); JSON prints the envelope with `status: "error"` and `error: {code: "lock_held" | "lock_io", message}` (no `redirect` payload: nothing was rewritten). **Takeover symlink pre-check (v5.0)**: a vendored→hosted takeover whose recorded wiring file is a symlink is refused up front with `redirect_symlinked_file_unsupported` — wet and `--dry-run` alike, before any revert — so "nothing was written" holds. **Human mode (v5.0)**: hosted `scan` runs discovery and update detection like the other modes, then rewrites without a prompt (scan never prompts) and prints its result line with one row per package it switched (see "Human output conventions"); `--dry-run` previews through the engine, and a detail fetch that leaves nothing to redirect enters the engine as a no-op (`Redirected 0 packages; rewrote 0 files.`, no lock, no `.socket/`). The detail fetch prints the same progress counter and per-package `Warning: could not fetch details for …` lines as the agent arm. An EMPTY hosted discovery prints `No patches available for installed packages.` and exits 0 without entering the engine; a discovery whose every offer is paid-tier for an org without paid access prints `No downloadable patches: every patch found requires a paid Socket plan.`, and exits 0 without entering the engine (the closing lines count what a paid plan would fix and name the plan page) (parity with the agent/vendored arms). Under `--json` (v5.0, MAJOR) the outcome is recorded into the scan envelope: one event per selected patch, sorted by purl, each with `details.mode: "hosted"` — `applied` for a confirmed pin (`verified` under `--dry-run`), `skipped` with the skip reason as `errorCode` (and its detail as `reason`), or `skipped` / `redirect_unconfirmed` for a patch that was granted but no lockfile entry pinning it could be rewritten (the human output's `Not hosted : …` line; more than three such packages share one line that names the first three) — plus a `redirect: {mode: "hosted", rewrittenFiles}` payload (the files rewritten, or that would be). A `skipped` event does not change `status` or the exit code (the hosted exit policy is an open decision, #704). Rewriter warnings carry stable `redirect_*` codes (e.g. `redirect_npm_no_lockfile`, `redirect_gradle_manual_snippet`, `redirect_golang_unsupported`); new codes are additive (MINOR). v5.0 additive codes: `redirect_composer_no_lockfile` / `redirect_gem_no_gemfile` (composer / gem: neither manifest nor lock present — once per run, after the intake gates), `redirect_pypi_no_lockfile` (pypi: a granted patch reached the PyPI rewriters but the project has no root `requirements.txt`, `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock`, PEP 751 / PEP 723 lock or Hatch configuration — once per run, beside each patch's `redirect_unconfirmed` row; #638), `redirect_pypi_pixi_project` (pypi: the same lane in a pixi project — a root `pixi.toml` or `pixi.lock` and no file a PyPI rewriter reads. pixi installs from its own lock, and mostly from conda channels, so no requirements file or other tool's lock would pin anything; said once per run in place of `redirect_pypi_no_lockfile`, with nothing written, and the detail points at agent mode: `scan --mode agent` after `pixi install`, then `apply` after every later `pixi install`. A pixi project whose own files name another lock tool, a `Pipfile` or a `[tool.uv]`, `[tool.poetry]` or `[tool.pdm]` table, keeps `redirect_pypi_no_lockfile` and that tool's lock step; a build backend alone names no such tool), `redirect_gem_bundle_gemfile_unsupported` (gem: `BUNDLE_GEMFILE` — `BUNDLE_GEMFILE:` in the bundler app config, which outranks the environment variable as in `Bundler::Settings`, else the environment variable — names a manifest other than the project's `Gemfile` / `gems.rb`, so no gem is redirected or attested; a value naming one of those two selects that pair even when the other spelling is present), `redirect_gem_bundle_lockfile_unsupported` (gem: bundler 4's custom lockfile — the `BUNDLE_LOCKFILE` environment variable, else `BUNDLE_LOCKFILE:` in the bundler app config, else in the global config — names a lock other than the loaded pair's own `Gemfile.lock` / `gems.locked`, so no gem is redirected or attested rather than pinning a lock bundler ignores), `redirect_gem_twin_manifest_ambiguous` (gem: a `Gemfile` + `gems.rb` twin under default discovery; bundler 1.x loads the `Gemfile` and bundler ≥ 2 loads `gems.rb`, and a lock's `BUNDLED WITH` records which bundler wrote it, not which one installs it, so neither pair is wired or attested — remove the unused spelling or set `BUNDLE_GEMFILE` to the one in use), `redirect_gem_mirror_overrides_source` (gem: Bundler's all-source, exact patch-source or patch-hostname mirror can route the gem's Socket `source:` to an unpatched upstream gem. Intake reads the app config (`BUNDLE_APP_CONFIG`, where a set-but-empty value selects `/config`, honoring `BUNDLE_IGNORE_CONFIG`) and all `BUNDLE_MIRROR__...` variables visible to the scan; app config overrides the environment per encoded key, then `mirror.all` takes precedence over exact source, which takes precedence over hostname. URI matching follows Bundler's whole-URI case folding, default-port/trailing-slash normalization and single slash key alias, not URL prefixes. An exact-source fallback-timeout key without a mirror URL shadows the hostname mirror and fetches that source directly; a configured URL is conservatively refused even if a timeout could bypass an unreachable mirror at install time. Like `redirect_gem_bundle_gemfile_unsupported`, the gate leaves the Gemfile pair byte-identical and confirms no gem redirect. On an embedded `scan --vex`, rediscovered older hosted gem pins may attest only from verified installed bytes: a missing tree is not excused by the lockfile, and `--vex-no-verify` omits those hosted gems with `mirror_overrides_source` rather than trusting their intercepted source. Agent/vendored evidence, unrelated ecosystems and standalone VEX behavior are unchanged. Details identify the setting form and its app/environment origin without printing mirror values or source URLs, which may contain credentials. Remove the applicable all/source/hostname setting (including any slash alias) from that origin and reuse its existing mirror URL under `mirror.https://rubygems.org` to clear the refusal; an environment setting must be unset in the scan/install environment. User-global Bundler config and mirrors set only in a later install environment are not inspected; keep those mirrors scoped to the upstream source too), `redirect_maven_no_pom` (no `pom.xml` and no Gradle build), `redirect_nuget_lock_unparseable` (a present-but-corrupt `packages.lock.json` — warned once, nothing mutated; an absent lock still proceeds; a leading UTF-8 BOM, which dotnet reads past, is not corrupt and is kept on rewrite), `redirect_nuget_lock_other_version` (the lock also resolves the patched id at another version in some target framework: the exact-id `packageSourceMapping` would route that framework to a feed that serves only the patched version, so the dep is skipped with nothing written; only lock entries at the patched version are ever re-pinned, and `resolved` is never rewritten), `redirect_nuget_mapping_set_aside` (v5.0 #462: another source's `packageSourceMapping` also named the patched id exactly — Visual Studio's mapping UI writes such lists — which ties with the Socket source so NuGet would take the package from whichever feed answers first; that pattern, or its whole `` when it was the only one, is commented out in a `` comment while the patch is wired, and `remove` / `rollback` put it back byte-exact), `redirect_nuget_mapping_conflict` (such a pattern sits in markup that cannot go in a comment; the dep is skipped, nothing written). v5.0 #354: when the rewriter creates the `packageSourceMapping`, its `*` catch-all also names the sources NuGet merges in from the user config and every parent directory's config (honoring ``), since NuGet drops every source no pattern names; a fresh config only seeds `nuget.org` when those inherited configs have it (a parent that cleared it for a mirror keeps that choice); when an inherited config already has a `packageSourceMapping` (NuGet merges those too), no catch-all is written at all — only the Socket pattern — so a source the parent restricts is never widened; the exclusivity set-aside exempts only the source the run wires, so a `socket-patch-*` lookalike key or a stale uuid naming the id is set aside too, `redirect_cargo_lock_pkg_ambiguous` (several same-name+version `[[package]]` blocks and none carries the index `source` — transactional skip), `redirect_cargo_dep_overridden` (the crate does not resolve to crates.io because the user overrides it — its one `Cargo.lock` block carries a git / other-registry `source`, as a `[patch]` git or registry override in the root manifest or any cargo config leaves it, or has no `source` while the root `Cargo.toml` / project `.cargo/config*` holds a `[patch]` path entry for it; with no `Cargo.lock`, the same `[patch.crates-io]` / `[patch."https://github.com/rust-lang/crates.io-index"]` entry for the crate, by key or `package = "…"`, in the root `Cargo.toml` or the project `.cargo/config*` — transactional skip, nothing rewritten: a Socket pin would silently replace the user's fork and the then-unused `[patch]` entry breaks `cargo --locked`). Also additive: `redirect_cargo_directory_source` (#455: a cargo config replaces crates.io with an on-disk source through its `[source.crates-io] replace-with` chain — a directory source, as a `cargo vendor` project has, or a `cargo local-registry` — so the project builds every crates.io crate offline from it and a Socket registry pin would break `cargo build --frozen --offline`. The configs read are the ones cargo reads for a build in the scanned directory: the project's own `.cargo/config*`, every parent directory's (a monorepo root) and `$CARGO_HOME`'s. Every cargo dependency is skipped with nothing written, one warning each naming the crate and the config that set the replacement, and the detail points at agent mode, which patches the copy cargo builds in place. A replacement with a registry mirror is not an offline source and is still redirected). Also additive: `redirect_gem_version_not_locked` (gem: no `GEM` section of the lock lists the crawled `name (version)`, for example a version another project installed into the shared gem home; the gem is skipped with nothing written, so the user's declared constraint and the locked version are never overwritten), `redirect_gem_no_lockfile` (gem: the project has a `Gemfile` / `gems.rb` but no `Gemfile.lock` / `gems.locked`, so nothing records which version it resolves and the crawled version may be another project's copy in the shared gem home; the gem is skipped with nothing written, never pinned over the user's constraint or appended as a new dependency, and the detail asks for `bundle lock` (or `bundle install`) and a re-run). Also v5.0: a registry override of the wrong kind (or none at all) warns the arm's missing-override code for nuget/gem/golang. Refusals stay fail-closed with a diagnosis that names the actual cause: a yarn-berry lock entry resolving through a non-`npm:` protocol keeps `redirect_yarn_berry_unsupported_protocol` with the entry's ACTUAL protocol in the detail — except socket-patch's OWN vendored wiring (a `file:` range into `.socket/vendor/`), which gets the distinct `redirect_yarn_berry_vendored_entry` code whose detail names the retirement path (`remove ` per package, or `vendor --revert` which unwinds every vendored package, then re-run `scan --mode hosted`). Both leave the entry byte-identical; neither changes exit code or status. **yarn berry line endings (v5.0)**: yarn writes a NEW `yarn.lock` with the OS line ending (`os.EOL` — CRLF on Windows) and keeps an existing lock's majority ending on every later write, and a `core.autocrlf` checkout turns an LF lock CRLF on any OS — so a uniformly CRLF lock is rewritten in its own ending: every untouched byte (a leading BOM included) round-trips (and `rollback`'s upstream restore keeps the lock's own ending). A lock that MIXES CRLF and LF (or holds a bare CR) has no single ending to keep — yarn's own `--immutable` check rejects it too (YN0028) — so it is refused untouched with `redirect_yarn_berry_mixed_line_endings` (the detail names `yarn install`, which normalizes it). The root `package.json`, which the rewrite re-renders to add `resolutions`, gets the same gate: a mixed one is refused untouched with the same code — the decision vendored mode takes with `vendor_yarn_berry_mixed_line_endings`, from the same shared berry gate set. This replaces v4's `redirect_yarn_berry_crlf_unsupported`, which refused every CRLF lock and is no longer emitted. A vendored→hosted takeover of a yarn-berry entry is checked against these project gates (mixed `yarn.lock` / `package.json` line endings, unsupported `cacheKey`, a non-zero `.yarnrc.yml` `compressionLevel`) on the project as it is before any revert, because the revert re-renders `package.json` in its majority ending — wet and `--dry-run` alike. A refused purl keeps its vendored wiring, ledger entry and artifact byte-identical, is skipped with the gate's code (reported in `warnings[]`, followed by `redirect_takeover_kept_vendored`) and is never announced as `redirect_takeover_reverted_vendored`. +`scan --mode hosted` swaps the in-place apply for the registry-redirect pipeline: discover → resolve hosted-patch references (grant token + integrity + per-dep registry override) → rewrite ONLY the patched dependencies' lockfile / registry-config entries to point at the hosted packages. A dep counts as **redirected** only when its hosted-artifact URL (or per-dep registry index URL) actually landed in a project file — a granted reference whose rewriter found nothing to edit is neither counted nor attested. **No ledger (v5.0)**: hosted mode writes ONLY the lockfile / registry-config edits — `.socket/vendor/redirect-state.json` is never written (on success or failure), and a pre-v5 one on disk is ignored (never read for planning, never quarantined, left byte-identical). The lockfiles are the only record of a hosted patch: `list`, `vex`, `rollback`, `remove`, `vendor` and `repair` all discover the hosted pins from them (a hosted URL counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin or under the `--patch-mirror-url` / `SOCKET_PATCH_MIRROR_URL` base), and commit-ready output is just the lockfile / config changes. Cargo and golang are confirmed only by their rewriter's own report (`confirmed_cargo_uuids` / `confirmed_golang_uuids`): a golang dep counts only when its go.mod `replace M V => patch.socket.dev/gopatch/ ` and both go.sum lines are in place (a re-run restores the zip line `go mod tidy` drops while no package of the module is loaded, so a later import still builds), never because the patch-server origin or leftover go.sum lines appear somewhere. Gradle is confirmed the same way (`confirmed_gradle_uuids`): only when the final files hold the owned script, the index row, the live apply line in every build's settings file and the suffixed version in every lock entry of the GA (see [Gradle builds](#gradle-builds-v50)). A golang module that go.mod does not require and go.sum has no zip (`h1:` of the module tree) line for at the patched version is outside the build graph and is refused with `redirect_golang_not_in_module_graph` (nothing written); with a `go.work` in effect the build graph is the workspace's (every `use` member's go.mod, go.sum and `go.work.sum`), so a member raising the module refuses the pin with `redirect_golang_version_mismatch`, and a `go.work` that overrides, ignores or rejects the replace refuses it with `redirect_golang_workspace_conflict` before anything is written. Run in the directory holding the `go.work`, a module the workspace shares (no root `go.mod`, or the root module is a `use` member and another member lists it or `go.work` already holds the Socket replace) is pinned in `go.work` and its two sum lines go into `go.work.sum`, the members' files untouched; a member's own `replace` of it, which the go.work pin would override, refuses it (`redirect_golang_workspace_conflict`); a `/go.mod`-only line doesn't count, since go writes one for every version MVS reads (#509). In a go < 1.17 graph, where `require` holds only lower bounds, the build selects the highest of the required version and go.sum's zip-line versions, and a patch for a lower version warns `redirect_golang_version_mismatch` (nothing written). Only the exact module `patch.socket.dev/gopatch/` is socket-owned; any other module path is refused with `redirect_golang_untrusted_module_path`. A vendored golang module is taken over like cargo and the npm family: its vendor wiring, committed copy and ledger entry are reverted first (`redirect_takeover_reverted_vendored`). A vendored PyPI package (requirements.txt, Poetry, Pipenv, uv, Hatch, PDM, pylock) is taken over the same way: its vendored wiring is restored to the recorded registry entry, its ledger entry and wheel are removed, and it is redirected in the same commit. The Python rewriters treat any non-registry source as user-authored, so without the revert they refused socket-patch's own vendored source and left the project vendored. A vendored Composer package is taken over the same way: its `composer.lock` entry is restored to the recorded pre-vendor entry, its ledger entry and committed copy are removed, and it is redirected in the same commit, so the lock matches a direct hosted scan and `rollback` can restore it from packagist. Without the revert the rewriter edited the vendored entry in place and kept its patch-uuid `dist.reference`, which `rollback` refused. A vendored Composer entry the takeover cannot revert (its vendored ledger entry is missing) is left vendored with `redirect_composer_vendored_entry`, never repointed in place. A takeover revert that leaves vendored wiring in place is refused with `redirect_vendored_revert_failed`. That covers a drift-skipped record (`vendor_lock_entry_drifted`) and a reverted file that still references the artifact (`vendor_revert_residual_reference`). The ledger entry and artifact are kept, and the package stays vendored and skipped. `--dry-run` predicts the same refusal from the same signals instead of previewing `redirect_would_revert_vendored`. The hosted requirements.txt rewriter only rewrites an existing pin in the root `requirements.txt`, so a vendored requirements.txt package whose wiring is a pin in a `-r` include or a `(transitive)` line vendored mode appended is kept vendored, wet and `--dry-run` alike, with `redirect_requirements_takeover_unreachable` (a top-level `warnings[]` entry, and the `errorCode` of the package's `skipped` event). Its wiring, ledger entry and wheel are kept, so it stays vendored and patched (exit 0). The uv and Poetry rewriters are covered the same way: a vendored uv package whose recorded pre-vendor `uv.lock` entry is at another version than the patch (earlier releases pinned such an entry down to the patch's version, where vendored uv now refuses the patch with `pypi_uv_lock_package_missing`; the revert brings the lock's own version back, and hosted mode only pins the version the lock resolves) is kept vendored with `redirect_uv_takeover_version_unreachable`, and a vendored Poetry package on a Poetry 0.x lock (which hosted mode refuses outright) is kept vendored with `redirect_poetry_lock_unsupported`. **Staged takeover (v5.0)**: the takeover is atomic. Each purl's vendored revert is staged in memory (the run's group commit, the same journaled commit vendored mode uses), the hosted rewrite plans against the reverted project, and a staged purl the rewrite does not pin is retracted: every staged revert is undone, that purl keeps its vendored wiring, ledger entry and artifact byte-identical, and the rest are staged and rewritten again. A retracted purl is a `skipped` event whose `errorCode` is the cause: its existing skip reason (unavailable wheel metadata, …), the rewriter warning that names the package (`redirect_yarn_berry_missing_checksum`, `redirect_pypi_platform_wheel`, …), `redirect_requirements_takeover_unreachable`, the rewrite's lock-level refusal (`redirect_yarn_berry_mixed_line_endings`, `redirect_bun_lock_unsupported`, a Gradle planner refusal, …) or `redirect_takeover_not_pinned`; that warning is reported in `warnings[]`, followed by `redirect_takeover_kept_vendored` naming the package. Exit 0: the package stays vendored and patched, never unpatched in both modes. The reverts, the hosted pins and the vendored ledger then reach the disk in one commit, and the reverted artifacts are deleted only after it: a refusal or write failure before the commit writes nothing, a failed commit puts back the files it replaced, and a commit interrupted after its journal was written is finished by the next command that takes the apply lock. `--dry-run` runs the same steps and drops the staged state instead of committing, so its `verified` events and warnings are the wet run's `applied` events and warnings. A hosted run with no takeover stages its writes the same way but commits them without the journal (it writes nothing under `.socket/`): a file that fails to be replaced puts back the ones already replaced, so a failed run leaves no lock half-redirected. Re-runs over already-rewritten output plan from the current lock text and are idempotent (exit 0, lock unchanged). **Lock (v5.0)**: the hosted engine acquires `<.socket>/apply.lock` before its first wet write (the staged takeover reverts) — not on `--dry-run`, and not when the run would write nothing (zero redirects, all skipped) — so previews and no-op runs never create `.socket/`; contention is `lock_held` and a lock-file I/O fault (a read-only project root, a file squatting on `.socket/`) is `lock_io` — both exit 1, refused BEFORE any project file is written, and rendered like every other lock holder: human `Error (): ` on stderr (+ the `--lock-timeout` hint for a live holder); JSON prints the envelope with `status: "error"` and `error: {code: "lock_held" | "lock_io", message}` (no `redirect` payload: nothing was rewritten). **Takeover symlink pre-check (v5.0)**: a vendored→hosted takeover whose recorded wiring file is a symlink is refused up front with `redirect_symlinked_file_unsupported` — wet and `--dry-run` alike, before any revert — so "nothing was written" holds. **Human mode (v5.0)**: hosted `scan` runs discovery and update detection like the other modes, then rewrites without a prompt (scan never prompts) and prints its result line with one row per package it switched (see "Human output conventions"); `--dry-run` previews through the engine, and a detail fetch that leaves nothing to redirect enters the engine as a no-op (`Redirected 0 packages; rewrote 0 files.`, no lock, no `.socket/`). The detail fetch prints the same progress counter and per-package `Warning: could not fetch details for …` lines as the agent arm. An EMPTY hosted discovery prints `No patches available for installed packages.` and exits 0 without entering the engine; a discovery whose every offer is paid-tier for an org without paid access prints `No downloadable patches: every patch found requires a paid Socket plan.`, and exits 0 without entering the engine (the closing lines count what a paid plan would fix and name the plan page) (parity with the agent/vendored arms). Under `--json` (v5.0, MAJOR) the outcome is recorded into the scan envelope: one event per selected patch, sorted by purl, each with `details.mode: "hosted"` — `applied` for a confirmed pin (`verified` under `--dry-run`), `skipped` with the skip reason as `errorCode` (and its detail as `reason`), or `skipped` / `redirect_unconfirmed` for a patch that was granted but no lockfile entry pinning it could be rewritten (the human output's `Not hosted : …` line; more than three such packages share one line that names the first three) — plus a `redirect: {mode: "hosted", rewrittenFiles}` payload (the files rewritten, or that would be). A `skipped` event does not change `status` or the exit code (the hosted exit policy is an open decision, #704). Rewriter warnings carry stable `redirect_*` codes (e.g. `redirect_npm_no_lockfile`, `redirect_gradle_manual_snippet`, `redirect_golang_unsupported`); new codes are additive (MINOR). v5.0 additive codes: `redirect_composer_no_lockfile` / `redirect_gem_no_gemfile` (composer / gem: neither manifest nor lock present — once per run, after the intake gates), `redirect_pypi_no_lockfile` (pypi: a granted patch reached the PyPI rewriters but the project has no root `requirements.txt`, `uv.lock`, `poetry.lock`, `pdm.lock`, `Pipfile.lock`, PEP 751 / PEP 723 lock or Hatch configuration — once per run, beside each patch's `redirect_unconfirmed` row; #638), `redirect_pypi_pixi_project` (pypi: the same lane in a pixi project — a root `pixi.toml` or `pixi.lock` and no file a PyPI rewriter reads. pixi installs from its own lock, and mostly from conda channels, so no requirements file or other tool's lock would pin anything; said once per run in place of `redirect_pypi_no_lockfile`, with nothing written, and the detail points at agent mode: `scan --mode agent` after `pixi install`, then `apply` after every later `pixi install`. A pixi project whose own files name another lock tool, a `Pipfile` or a `[tool.uv]`, `[tool.poetry]` or `[tool.pdm]` table, keeps `redirect_pypi_no_lockfile` and that tool's lock step; a build backend alone names no such tool), `redirect_gem_bundle_gemfile_unsupported` (gem: `BUNDLE_GEMFILE` — `BUNDLE_GEMFILE:` in the bundler app config, which outranks the environment variable as in `Bundler::Settings`, else the environment variable — names a manifest other than the project's `Gemfile` / `gems.rb`, so no gem is redirected or attested; a value naming one of those two selects that pair even when the other spelling is present), `redirect_gem_bundle_lockfile_unsupported` (gem: bundler 4's custom lockfile — the `BUNDLE_LOCKFILE` environment variable, else `BUNDLE_LOCKFILE:` in the bundler app config, else in the global config — names a lock other than the loaded pair's own `Gemfile.lock` / `gems.locked`, so no gem is redirected or attested rather than pinning a lock bundler ignores), `redirect_gem_twin_manifest_ambiguous` (gem: a `Gemfile` + `gems.rb` twin under default discovery; bundler 1.x loads the `Gemfile` and bundler ≥ 2 loads `gems.rb`, and a lock's `BUNDLED WITH` records which bundler wrote it, not which one installs it, so neither pair is wired or attested — remove the unused spelling or set `BUNDLE_GEMFILE` to the one in use), `redirect_gem_mirror_overrides_source` (gem: Bundler's all-source, exact patch-source or patch-hostname mirror can route the gem's Socket `source:` to an unpatched upstream gem. Intake reads the app config (`BUNDLE_APP_CONFIG`, where a set-but-empty value selects `/config`, honoring `BUNDLE_IGNORE_CONFIG`) and all `BUNDLE_MIRROR__...` variables visible to the scan; app config overrides the environment per encoded key, then `mirror.all` takes precedence over exact source, which takes precedence over hostname. URI matching follows Bundler's whole-URI case folding, default-port/trailing-slash normalization and single slash key alias, not URL prefixes. An exact-source fallback-timeout key without a mirror URL shadows the hostname mirror and fetches that source directly; a configured URL is conservatively refused even if a timeout could bypass an unreachable mirror at install time. Like `redirect_gem_bundle_gemfile_unsupported`, the gate leaves the Gemfile pair byte-identical and confirms no gem redirect. On an embedded `scan --vex`, rediscovered older hosted gem pins may attest only from verified installed bytes: a missing tree is not excused by the lockfile, and `--vex-no-verify` omits those hosted gems with `mirror_overrides_source` rather than trusting their intercepted source. Agent/vendored evidence, unrelated ecosystems and standalone VEX behavior are unchanged. Details identify the setting form and its app/environment origin without printing mirror values or source URLs, which may contain credentials. Remove the applicable all/source/hostname setting (including any slash alias) from that origin and reuse its existing mirror URL under `mirror.https://rubygems.org` to clear the refusal; an environment setting must be unset in the scan/install environment. User-global Bundler config and mirrors set only in a later install environment are not inspected; keep those mirrors scoped to the upstream source too), `redirect_maven_no_pom` (no `pom.xml` and no Gradle build), `redirect_nuget_lock_unparseable` (a present-but-corrupt `packages.lock.json` — warned once, nothing mutated; an absent lock still proceeds; a leading UTF-8 BOM, which dotnet reads past, is not corrupt and is kept on rewrite), `redirect_nuget_lock_other_version` (the lock also resolves the patched id at another version in some target framework: the exact-id `packageSourceMapping` would route that framework to a feed that serves only the patched version, so the dep is skipped with nothing written; only lock entries at the patched version are ever re-pinned, and `resolved` is never rewritten), `redirect_nuget_mapping_set_aside` (v5.0 #462: another source's `packageSourceMapping` also named the patched id exactly — Visual Studio's mapping UI writes such lists — which ties with the Socket source so NuGet would take the package from whichever feed answers first; that pattern, or its whole `` when it was the only one, is commented out in a `` comment while the patch is wired, and `remove` / `rollback` put it back byte-exact), `redirect_nuget_mapping_conflict` (such a pattern sits in markup that cannot go in a comment; the dep is skipped, nothing written). v5.0 #354: when the rewriter creates the `packageSourceMapping`, its `*` catch-all also names the sources NuGet merges in from the user config and every parent directory's config (honoring ``), since NuGet drops every source no pattern names; a fresh config only seeds `nuget.org` when those inherited configs have it (a parent that cleared it for a mirror keeps that choice); when an inherited config already has a `packageSourceMapping` (NuGet merges those too), no catch-all is written at all — only the Socket pattern — so a source the parent restricts is never widened; the exclusivity set-aside exempts only the source the run wires, so a `socket-patch-*` lookalike key or a stale uuid naming the id is set aside too, `redirect_cargo_lock_pkg_ambiguous` (several same-name+version `[[package]]` blocks and none carries the index `source` — transactional skip), `redirect_cargo_dep_overridden` (the crate does not resolve to crates.io because the user overrides it — its one `Cargo.lock` block carries a git / other-registry `source`, as a `[patch]` git or registry override in the root manifest or any cargo config leaves it, or has no `source` while the root `Cargo.toml` / project `.cargo/config*` holds a `[patch]` path entry for it; with no `Cargo.lock`, the same `[patch.crates-io]` / `[patch."https://github.com/rust-lang/crates.io-index"]` entry for the crate, by key or `package = "…"`, in the root `Cargo.toml` or the project `.cargo/config*` — transactional skip, nothing rewritten: a Socket pin would silently replace the user's fork and the then-unused `[patch]` entry breaks `cargo --locked`). Also additive: `redirect_cargo_directory_source` (#455: a cargo config replaces crates.io with an on-disk source through its `[source.crates-io] replace-with` chain — a directory source, as a `cargo vendor` project has, or a `cargo local-registry` — so the project builds every crates.io crate offline from it and a Socket registry pin would break `cargo build --frozen --offline`. The configs read are the ones cargo reads for a build in the scanned directory: the project's own `.cargo/config*`, every parent directory's (a monorepo root) and `$CARGO_HOME`'s. Every cargo dependency is skipped with nothing written, one warning each naming the crate and the config that set the replacement, and the detail points at agent mode, which patches the copy cargo builds in place. A replacement with a registry mirror is not an offline source and is still redirected). Also additive: `redirect_gem_version_not_locked` (gem: no `GEM` section of the lock lists the crawled `name (version)`, for example a version another project installed into the shared gem home; the gem is skipped with nothing written, so the user's declared constraint and the locked version are never overwritten), `redirect_gem_no_lockfile` (gem: the project has a `Gemfile` / `gems.rb` but no `Gemfile.lock` / `gems.locked`, so nothing records which version it resolves and the crawled version may be another project's copy in the shared gem home; the gem is skipped with nothing written, never pinned over the user's constraint or appended as a new dependency, and the detail asks for `bundle lock` (or `bundle install`) and a re-run). Also v5.0: a registry override of the wrong kind (or none at all) warns the arm's missing-override code for nuget/gem/golang. Refusals stay fail-closed with a diagnosis that names the actual cause: a yarn-berry lock entry resolving through a non-`npm:` protocol keeps `redirect_yarn_berry_unsupported_protocol` with the entry's ACTUAL protocol in the detail — except socket-patch's OWN vendored wiring (a `file:` range into `.socket/vendor/`), which gets the distinct `redirect_yarn_berry_vendored_entry` code whose detail names the retirement path (`remove ` per package, or `vendor --revert` which unwinds every vendored package, then re-run `scan --mode hosted`). Both leave the entry byte-identical; neither changes exit code or status. **yarn berry line endings (v5.0)**: yarn writes a NEW `yarn.lock` with the OS line ending (`os.EOL` — CRLF on Windows) and keeps an existing lock's majority ending on every later write, and a `core.autocrlf` checkout turns an LF lock CRLF on any OS — so a uniformly CRLF lock is rewritten in its own ending: every untouched byte (a leading BOM included) round-trips (and `rollback`'s upstream restore keeps the lock's own ending). A lock that MIXES CRLF and LF (or holds a bare CR) has no single ending to keep — yarn's own `--immutable` check rejects it too (YN0028) — so it is refused untouched with `redirect_yarn_berry_mixed_line_endings` (the detail names `yarn install`, which normalizes it). The root `package.json`, which the rewrite re-renders to add `resolutions`, gets the same gate: a mixed one is refused untouched with the same code — the decision vendored mode takes with `vendor_yarn_berry_mixed_line_endings`, from the same shared berry gate set. This replaces v4's `redirect_yarn_berry_crlf_unsupported`, which refused every CRLF lock and is no longer emitted. A vendored→hosted takeover of a yarn-berry entry is checked against these project gates (mixed `yarn.lock` / `package.json` line endings, unsupported `cacheKey`, a non-zero `.yarnrc.yml` `compressionLevel`) on the project as it is before any revert, because the revert re-renders `package.json` in its majority ending — wet and `--dry-run` alike. A refused purl keeps its vendored wiring, ledger entry and artifact byte-identical, is skipped with the gate's code (reported in `warnings[]`, followed by `redirect_takeover_kept_vendored`) and is never announced as `redirect_takeover_reverted_vendored`. **Attribution gate (v5.0).** A hosted run never leaves wiring that lockfile discovery calls contested: before the rewrite writes any lockfile, the same discovery `vex`, `list`, `rollback`, `remove` and `vendor` read runs over the project as the rewrite would leave it. A candidate whose pin discovery reads but cannot attribute to one package version (a requirements `-r` include resolving the same version from the registry beside a rewired `Pipfile.lock`, #567; a Maven pin in a ``, #260) is left out of the rewrite and reported as a `skipped` event with `errorCode` `redirect_unattributable` (nothing written for it; exit code unchanged). Unchanged: a pin in a file discovery does not read (a pre-2.6 bundler `Gemfile`, locked by the next `bundle install`) keeps the rewriter's verdict, and so does a deliberate partial redirect the run already reports (a bundled or `bun patch`-ed copy left on the registry, a yarn `npm:` alias entry left on the registry with `redirect_yarn_classic_alias_skipped` / `redirect_yarn_berry_alias_skipped` while the package's direct entry is pinned, a dep withheld from the vlt rewrite while a sibling lock takes it), and so does a vendored→hosted takeover: its vendored wiring is reverted in the run's staged overlay before the rewrite plans, and it is redirected when the rewriters pin it (otherwise it is retracted and stays vendored), in a `--dry-run` preview the same. A pin whose own pnpm or yarn lock also installs an unpatched copy of the same `name@version` (a pnpm `file:` directory or tarball dependency; a yarn `file:` / URL / berry `portal:` / `link:` dependency declared under another name) is written too, with a `redirect_copy_unpatched` warning naming that entry, and is not attested. A lockless NuGet / Cargo pin (an exclusive Socket source mapping without `packages.lock.json`, a Cargo registry pin without `Cargo.lock`) is still written, with a `redirect_pin_lockless` warning: no lockfile records its version, so `vex` cannot attest it and `rollback` / `remove` / `vendor` refuse it until the lockfile exists (whether such pins should be written at all is an open decision). The rollout's recorded view uses the same discovery: a uuid a file merely mentions (a stale `package.json` field, an inactive `pdm.lock`, a comment) is not a pin and does not count as already patched. -**Grant gate (additive).** A reference grant is the patch API's answer, and its fields are written into files the project commits. Each granted reference is therefore checked once, before anything is planned for it (no takeover, no artifact fetch, no rewriter sees one that fails): the patch uuid is a canonical lowercase uuid; the artifact URL and a registry override's index URL are `https` URLs of plain URL characters (no whitespace or control character, no quote, backslash, `$`, `{`, `}`, `<`, `>`, `&`, `#`, `;`, `|` or `^`; cargo's `sparse+` prefix is kept); `sha1` is 40 hex digits, `sha256` 64, `sha512` exactly one `sha512-` token, the Go hashes `h1:` hashes, the yarn berry checksum `/`, the cargo and gem checksums 64 hex digits; the package's name and version, and every identifier of a registry override, are in their ecosystem's coordinate grammar (letters, digits and that ecosystem's punctuation; Maven coordinates as the JVM backends accept them). A Maven grant's sha256 digests may also be spelled `sha256-…` / `sha256:…`: the pom, Gradle and sbt planners read those, and a digest that is still not 64 hex digits writes no checksum pin (`redirect_maven_trusted_checksums_invalid_hash`). An absent or empty optional field is not checked. A grant that fails is refused for that package only: a `skipped` event whose `errorCode` is `redirect_grant_malformed` and whose `reason` names the field (never its value), the same code and detail in `warnings[]`, nothing written for it, the rest of the run and the exit code unchanged. A plain-`http` artifact or index URL is refused the same way with `redirect_grant_insecure_url` (every later install would send the grant token in clear text), unless its origin is the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin (scheme, host and port) or a loopback address (`localhost`, `127.0.0.0/8`, `::1`). The in-memory engine applies the same gate; it has no configured origin. +**Grant gate (additive).** A reference grant is the patch API's answer, and its fields are written into files the project commits. Each granted reference is therefore checked once, before anything is planned for it (no takeover, no artifact fetch, no rewriter sees one that fails): the patch uuid is a canonical lowercase uuid; the artifact URL and a registry override's index URL are `https` URLs of plain URL characters (no whitespace or control character, no quote, backslash, `$`, `{`, `}`, `<`, `>`, `&`, `#`, `;`, `|` or `^`; cargo's `sparse+` prefix is kept); `sha1` is 40 hex digits, `sha256` 64, `sha512` exactly one `sha512-` token, the Go hashes `h1:` hashes, the yarn berry checksum `/`, the cargo and gem checksums 64 hex digits; the package's name and version, and every identifier of a registry override, are in their ecosystem's coordinate grammar (letters, digits and that ecosystem's punctuation; Maven coordinates as the JVM backends accept them). A Maven grant's sha256 digests may also be spelled `sha256-…` / `sha256:…`: the pom, Gradle and sbt planners read those, and a digest that is still not 64 hex digits writes no checksum pin (`redirect_maven_trusted_checksums_invalid_hash`). An absent or empty optional field is not checked. A grant that fails is refused for that package only: a `skipped` event whose `errorCode` is `redirect_grant_malformed` and whose `reason` names the field (never its value), the same code and detail in `warnings[]`, nothing written for it, the rest of the run and the exit code unchanged. A plain-`http` artifact or index URL is refused the same way with `redirect_grant_insecure_url` (every later install would send the grant token in clear text), unless its origin is the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin (scheme, host and port), it is under the `--patch-mirror-url` / `SOCKET_PATCH_MIRROR_URL` base, or its host is a loopback address (`localhost`, `127.0.0.0/8`, `::1`). The in-memory engine applies the same gate; it has no configured origin. The rewriter reads a fixed set of candidate files from the project root: the npm-family locks (`package-lock.json`, `npm-shrinkwrap.json`, `pnpm-lock.yaml`, `shrinkwrap.yaml`, `yarn.lock`, plus `.yarnrc.yml` for the berry cache-config gate, `bun.lock` / `bun.lockb`, and `vlt-lock.json` with `vlt.json` and `node_modules/.vlt-lock.json` read only), `requirements.txt` / `uv.lock` / `Pipfile.lock` (pipfile-spec 6; see the Pipenv section below) / `poetry.lock` (every Poetry lock generation from 1.0 on — the 0.12 `[metadata.hashes]` layout is refused because that installer ignores URL sources; a Poetry < 1.4 writer additionally gets `redirect_poetry_stale_install_risk`, see `docs/testing/poetry-compatibility.md`) / `pdm.lock` (PDM lock formats `2` and `4.3`–`4.5.1`; the identity-losing `3.1` / `4.0`–`4.2` formats and unknown future formats are refused with `redirect_pdm_refused`, and a lock-format-`2` writer additionally gets `redirect_pdm_legacy_sync_required`, see `docs/testing/pdm-compatibility.md`; when `uv.lock` or `poetry.lock` sits beside it they drive and `pdm.lock` is left alone), `Cargo.toml` / `Cargo.lock` / `.cargo/config.toml` (plus the legacy extensionless `.cargo/config` — cargo reads that spelling in preference when both exist, so the managed `[registries.…]` block is written into whichever one is present; the hosted redirect is a `sparse+` registry, which cargo loads only from **1.68**, so `rust-toolchain.toml` / `rust-toolchain` are read too (never edited) and a project whose declared floor — the lowest `rust-version` of the root and the member manifests read here (a `rust-version.workspace = true` inheritor reads `[workspace.package]`), else the channel rustup picks from the root (the legacy `rust-toolchain` over `rust-toolchain.toml`; a dated `nightly-`/`beta-YYYY-MM-DD` maps to its release train) — is below 1.68 is refused `redirect_cargo_old_cargo` per crate (nothing written, an earlier run's pins left in place with the `rollback` that drops them named; `--mode vendored` builds on cargo 1.41+), while an undeclared or rolling-channel floor still redirects (toolchain files in ancestor directories are not read); **cargo also reads every workspace-member manifest** — the `[workspace] members` globs minus `exclude` — and every in-root path-dependency manifest, recursively, reached without crossing a symbolic link and never under `.socket/`, and pins the crate in each one that declares it, so those `/Cargo.toml` files can appear in `rewrittenFiles`. A crate is redirected only when every declaration pins and every other `Cargo.lock` package depending on it is a planned member: one a registry or git crate — or a path package outside the root or behind a link — also depends on is refused `redirect_cargo_transitive_dependents` (a pin reaches only the declarations it sits on), a crate no manifest declares keeps `redirect_cargo_toml_dep_not_found` with a transitive-only detail naming `--mode vendored`, a crate every declaration of which requires another version (no requirement accepts the patched version) is refused `redirect_cargo_toml_dep_unrewritable`, and so is a requirement that also matches another locked version of the crate — each a transactional skip, never recorded or attested. With NO `Cargo.lock` there is no resolved graph to ask, so the dependents question is answered from the manifests instead: a crate declared beside any other dependency — anything but a path dependency on a manifest this run also pins, or a `workspace = true` inheritor of a table it scans — or beside a workspace member this run did not read (a `members` glob, or a member outside the project or behind a symbolic link, which member discovery drops) is refused `redirect_cargo_lockless_dependents`, whose detail names the remedies (commit a lockfile, or `--mode vendored`); a project whose only dependency is the patched crate has nothing that could pull it in and still redirects. All-CRLF manifests, locks and configs are rewritten with CRLF kept (mixed endings keep refusing where the grammar does not match), and `remove` / rollback match the recorded fragments across a later CRLF↔LF checkout conversion), `composer.lock`, `nuget.config` / `packages.lock.json` (plus, v5.0 #353/#514, every lock a project under the root restores into: a member project's `packages.lock.json`, a per-project `packages..lock.json`, a literal `NuGetLockFilePath` — each pinned with the root config; a `NuGetLockFilePath` it cannot evaluate warns `redirect_nuget_lock_path_unresolved` and a project tree it cannot list warns `redirect_nuget_lock_unreadable`, both skipping the nuget redirect with nothing written), `Gemfile` / `Gemfile.lock`, `pom.xml` (+ `.mvn/maven.config` / `.mvn/checksums/checksums.sha256` for maven Trusted Checksums merge, and, for a Gradle build, every settings, build, `buildSrc`, included-build, applied and plugin-source script, version catalog and lock file the script graph reaches, plus `gradle/verification-metadata.xml`, `gradle/wrapper/gradle-wrapper.properties` and the owned `.socket/gradle/` files), and the sbt build files (`socket-patch.sbt`, `socket-patch-vendor.sbt`, `build.sbt`, `project/build.properties`, `.sbtopts`, `.jvmopts`; `build.sbt.lock` and the Mill / scala-cli build files `build.mill`, `build.mill.yaml`, `build.sc`, `.mill-version`, `project.scala` for their presence only) — read, never edited; `socket-patch.sbt` is the only sbt file hosted mode writes (see **Hosted sbt** below). **npm-family flavor coverage**: package-lock / npm-shrinkwrap (a package the project patches itself with npm ≥ 12.1's native `npm patch` — a root `patchedDependencies` key, or the lock entry's `patched` record — is left on its registry entry in every npm lock, `redirect_npm_patched_dependency_skipped`), pnpm (root OR any nested `*/pnpm-lock.yaml`), yarn classic (a yarn 2+ install migrates a v1 `yarn.lock` and drops its pins, so a run whose v1 lock carries a hosted pin warns `redirect_yarn_classic_berry_migration_risk` — the hosted twin of the vendored `yarn_classic_berry_migration_risk` — unless the root `package.json`, read as advisory input, declares `"packageManager": "yarn@1…"`), **yarn berry** (the pin yarn writes for a root `resolutions` entry: the root `package.json` — edited only beside a berry `yarn.lock` — gains one `"@npm:": ""` selector per locked range (`redirect_yarn_berry_resolution` edits), and only that `yarn.lock` entry is re-keyed `"@"` with the same `resolution:` + `yarnBerry10c0` checksum (`redirect_yarn_berry_entry`), moved to yarn's key order; never an `npm:` locator, whose fetcher sends npm registry auth to the patch host, nor a tarball locator under an `npm:` key, which hardened mode rejects (YN0078). An older release's `npm:::__archiveUrl=` pin is still recognized and is re-pinned on the next run; rollback rebuilds the key from the selectors and drops them. Refused, nothing written: a user-authored `resolutions` entry for the package `redirect_yarn_berry_resolutions_conflict`, no root manifest `redirect_yarn_berry_manifest_missing`, a builtin `patch:` entry wrapping the same descriptor `redirect_yarn_berry_shared_descriptor`, an artifact URL yarn cannot fetch as a tarball `redirect_yarn_berry_artifact_url_unsupported`; cacheKey `10c0` and `.yarnrc.yml compressionLevel 0` gated by `redirect_yarn_berry_cache_unsupported`), and **bun** (text `bun.lock` lockfileVersion 0 to 3 — 0 is the `--save-text-lockfile` opt-in lock of Bun 1.1.39–1.1.45, 1 the 1.2–1.3 default, 2 the 1.4+ default, 3 what Bun 1.4+ writes when `package.json` has a nested override or a yarn-style path resolution (only the lock's head differs: its `overrides` object nests); all four emit one `packages` grammar, so the registry 4-tuple → URL 3-tuple rewrite is version-independent and the lock's own version line is kept. Any other or missing version, or a `packages` section outside bun's single-line grammar, is refused `redirect_bun_lock_unsupported` — the detail is the shared version gate's text (a newer version: update socket-patch, re-locking would reproduce it; no integer: re-lock with Bun ≥ 1.2), identical to the vendored refusal. A version-0 lock holding `workspace:` packages is refused `redirect_bun_workspace_unsupported` (its 2-tuple workspace grammar cannot keep the hosted tuple through a frozen install); the remedy is to delete `bun.lock` and re-run `bun install` with Bun ≥ 1.2, which writes lockfileVersion 1 (accepted). A plain in-place `bun install` bumps the version only when a workspace depends on another workspace (e.g. root → member — the shape the matrix measured); otherwise Bun 1.2.0 keeps version 0 and Bun 1.2.23+ fail to resolve, so the in-place bump is not the documented remedy. Bun lock version, grammar and workspace compatibility are checked before a vendored takeover, including during dry-run: these refusals preserve the existing lock, artifact and vendor ledger. Version-1 and version-2 workspace locks are rewritten, nested versions included. A granted dep with no rewritable entry warns `redirect_bun_entry_not_found` (or `redirect_bun_non_registry_entry_skipped` when the only same-version entry is a user URL / `file:` tarball, #497), a grant without a sha512 `redirect_bun_missing_sha512`; a CRLF lock keeps `\r\n` on the rewritten line, and a hosted URL left by an earlier grant of the same `name@version` is re-pinned in place. **Digest-less re-saves (Bun 1.1.39–1.3.9)**: every text-lock Bun below 1.3.10 re-saves a URL tuple WITHOUT its `sha512` whenever the lock is re-saved for another reason (`bun add`, `bun install` after a package.json or workspace change), leaving the 2-tuple `["name@", {meta}]` — the spec Bun installs from is intact. The CLI treats that spelling as its own wiring: a repeat hosted run counts the dep as redirected (no `redirect_bun_entry_not_found`) and HEALS the line back to the 3-tuple with the current `sha512`, recording the heal as a further `redirect_bun_lock_package` edit whose `original` is the 2-tuple (a stale URL is re-pinned from either spelling); `rollback`, scoped `rollback ` / `remove ` and the vendored takeover accept the digest-less spelling of a recorded `new` line (same key, spec and meta, only the trailing `"sha512-…"` missing) and restore the recorded original over it, so the chain always unwinds to the pristine registry line. Anything else — another uuid/token, another version, a re-laid meta object — is still drift. **Native `bun.lockb`**: when no text `bun.lock` exists, binary format versions 1, 2 and 3 are read and rewritten directly. Socket Patch does not invoke Bun or convert the project to a text lockfile. Exact matching package records are rewritten to hosted tarballs with the granted integrity, preserving dependency resolution IDs, workspace/dependency topology and unrelated package metadata; binary pointers and the package metadata hash are updated. Per-package `redirect_bun_lockb_package` snapshots support scoped rollback, repeat runs, superseding grants and hosted ↔ vendored takeover. A regular binary lock is discoverable even with no Bun runtime or `node_modules`; a dry run previews the same binary edits without writing them. A malformed, unreadable, unsupported or unverified binary structure is `redirect_bun_lockb_invalid` (exit 0, `redirected: 0`), and it refuses the npm rewrite before any takeover or sibling npm-family lock mutation. A symlinked binary write target is `redirect_symlinked_file_unsupported` (exit 1, including dry-run). `bun.lock` wins when both spellings exist. Binary-only projects do not receive `redirect_npm_no_lockfile`. Measured boundaries and the real-Bun matrix: `docs/testing/bun-compatibility.md`), and **vlt** (`vlt-lock.json` without `lockfileVersion`, `0` or `1`; see the vlt hosted-mode contract below). **Rush monorepos**: when `rush.json` is present the rewriter also reads `common/config/rush/pnpm-lock.yaml` and each `common/config/subspaces//pnpm-lock.yaml` (sorted for determinism) under their repo-relative keys and repoints them in place; editing them emits `redirect_rush_repo_state_stale` when the `repo-state.json` beside a rewritten lock exists (`common/config/rush/repo-state.json` for the common lock, `common/config/subspaces//repo-state.json` for a subspace lock; the `pnpmShrinkwrapHash` desync is refreshed by `rush update`, which the redirect survives), and `redirect_pnpm_trust_lockfile` carries the Rush pnpm >=11 install remedy (see the trust paragraph above). **maven** is fail-closed via version suffixing: a `mavenSuffixedVersion` + `mavenPomSha256` override pins the Socket-only `-socket.` by rewriting the literal `` (`redirect_maven_dep_version`) or adding a `` entry (`redirect_maven_dep_management_added`), plus optional Trusted Checksums (`redirect_maven_trusted_checksums`, conflicts as `redirect_maven_trusted_checksums_conflict`; the `.mvn/maven.config` line that names the checksums directory is spelled `${maven.multiModuleProjectDirectory}/.mvn/checksums`, which no Maven 3 build fails on wherever it is started, or `${session.rootDirectory}/.mvn/checksums` for a project that declares Maven 4 (a wrapper that pins it, or a root pom whose model is newer than 4.0.0: its ``, else the version its namespace names), the only form Maven 4 reads. The six settings hosted mode wrote, standing together in the other spelling, are respelled in that one line by every `scan --mode hosted` that reads the file (`redirect_maven_config`, `rewritten`, `key: "trustedChecksumsDirectory"`; the edit that merges the settings in keeps `key: "trustedChecksums"`), whatever the run redirects: with no Maven patch in the run, in a reactor root, in a root another rewriter owns, with no checksum line left, with a later line that switches the pin off. A scan with nothing to redirect reads `.mvn/maven.config` for that alone, and takes the write lock for it when it can. The respelling is never the reason a run fails: when the lock cannot be had, or the file is a symbolic link (which a write would replace), the line is left as it is and nothing is reported for it. This is what removes the line an earlier release wrote for every project, which stops Maven 3.9.2 to 3.9.8 from starting with `-f` from outside the project. A directory line that stands apart from the other five is the user's and is never rewritten. Since that respelling is decided by the file and by no dep, a Maven reason code that says *the run writes nothing for the dep* means that dep in that run: no pin in a pom, and no line or file under `.mvn/`, the checksum lines a malformed grant hash would drop included. It says nothing of wiring an earlier run left in the poms, nor of that one respelled line. When the respelling takes `${session.rootDirectory}` out of a project that declares no Maven, a Maven 4 build of it stops enforcing the pin (Maven 3.9.4 and later 3.x go on enforcing it): the additive warning `redirect_maven_trusted_checksums_maven3_only` says so, once, with the remedy (declare Maven 4 and scan again). A directory line in the launcher's spelling that the user wrote, in a project that declares Maven 4, names no directory there: it is left as written and reported as `redirect_maven_trusted_checksums_conflict`. Either form is recognised by `vex`, `rollback` and `remove`; when the Maven the project declares does not enforce the pin as the config stands (the wrapper pins a Maven older than 3.9.4, or the project declares Maven 4 and the line is in the spelling Maven 4 does not read), the additive warning `redirect_maven_trusted_checksums_unenforced`, whose detail names the remedy for that case; a grant sha256 that is not 64 hex digits writes no checksum line, `redirect_maven_trusted_checksums_invalid_hash`); a `${property}` version is wired through the property's own value (`redirect_maven_property`) or a literal that records the text it replaced (`redirect_maven_dep_version` with the `${...}` text as `original`), and stays `redirect_maven_dep_unpinned` only when what it reads cannot be told, a reactor root (a `` / `` declaration, a profile's included, or a Maven 4 root that lists none but whose subprojects Maven discovers: model version not `4.0.0`, `pom` packaging, a direct subdirectory holding a `pom.xml`, and `maven.project.discoverSubprojects` not turned off in `.mvn/maven.config`) is planned over every pom of the reactor, each module `pom.xml` that takes an edit appearing in `rewrittenFiles` (see [Hosted Maven reactors and property versions](#hosted-maven-reactors-and-property-versions)); a reactor one of whose poms cannot be read, and a same-GAV grant in a reactor, are refused whole with `redirect_maven_multimodule_unsupported`: the run writes nothing for the dep and it is not counted as redirected (a Gradle build beside it is still planned by the Gradle rewriter, but a dep in a root with a `pom.xml` is confirmed only when the poms pin it too), a non-matching literal skipped (`redirect_maven_dep_version_mismatch`), a `` variant (sources, tests, a native build) left at its version (`redirect_maven_classifier_unsupported`; the grant serves the main jar only, and the dep is skipped, and the run writes nothing for it, when a variant other than `sources` / `javadoc`, live or inside ``, may resolve any version but the suffixed pin — the patched release, another one, a `${property}` a profile moves, or no version in the pom once its classifier-keyed `` (a profile's too) is consulted — since `vex` would never attest that pin: `vex_maven_classifier_unpatched`), a versioned declaration inside `` (a literal or a `${property}`) left as-is (`redirect_maven_profile_dependency_unpatched`): when any version it may resolve — its own, or what its own profile's or any combination of profiles' `` make of it, read as `vex` reads it — is not the suffixed pin, the dep is refused, and the run writes nothing for it, since it overrides the pin whenever that profile is active and `vex` would never attest it (a profile `` entry only when management governs the pin, i.e. no live direct declaration carries a version); a versionless one takes the pin, and the dep is also skipped when a profile holds its only declaration, a `${property}` groupId / artifactId resolved from the pom's own `` before matching (one the pom does not define, which may be the patched GA and whose own version would then win, refuses the dep, and the run writes nothing for it: `redirect_maven_coordinate_unresolved`), and a `${property}` groupId / artifactId some profile's `` (or several profiles' together) redefine so that the declaration a pin would go into names another package, or another declaration names the GA's main jar at another version (or one of its executable classifier copies, a `test-jar` included), refuses the dep the same way (`redirect_maven_profile_coordinates_unpatched`): `vex` would never attest that pin (`vex_maven_profile_unpatched`), and a pom not readable as one left untouched (`redirect_maven_pom_unreadable`). Matches and anchors skip comments, CDATA, ``, ``, ``, `` and `` (the scope `vex` reads), and an existing self-closed `` / `` is expanded in place, never duplicated; an override without a suffixed version falls back to same-GAV repository injection (`redirect_maven_same_gav_fallback`, NOT fail-closed). A pin is written only where Maven selects the patch's base version already (`redirect_maven_version_not_selected`, `redirect_maven_selection_undetermined`): see [Maven version selection](#maven-version-selection-hosted-and-vendored-maven). **gradle** (v5.0) is automated wiring, no longer a pasted snippet: the owned settings script `.socket/gradle/socket-patch.hosted.settings.gradle` with its index `.socket/gradle/hosted-index.tsv`, one apply line per build's settings file, every lock entry of the GA moved to the suffixed version, and the suffixed component in an existing `gradle/verification-metadata.xml`. A refused dep writes nothing and keeps `redirect_gradle_manual_snippet` as its fallback; same-GAV grants are refused (`redirect_gradle_same_gav_unsupported`). Rules, refusals and codes: [Gradle builds](#gradle-builds-v50). @@ -499,6 +500,22 @@ v5.0 removed the v4 spellings `scan --apply` (use `--mode agent`), `scan --vendo **Python stale-install guard**: after a hosted redirect, `scan` / `get` use the Python crawler to inspect every matching installed package, including Poetry's out-of-tree virtualenvs, the project's Hatch environments (Hatch's data dir / `HATCH_DATA_DIR`, `[dirs.env] virtual`, explicit env `path`s) and `--global-prefix`. Each installed copy is judged against the patched wheel the lock now pins: downloaded through its grant URL, verified against the pinned sha256, and compared with the wheel's own `RECORD` (files outside its `.dist-info` and `.data` dirs), but only when the installed `.dist-info/WHEEL` carries the same tags, since another build's bytes say nothing about this one. An agent-mode manifest record of the same patch judges the copy by its `afterHash` map instead. A readable file that differs from the patched state emits `redirect_pypi_stale_install` in JSON `warnings[]` and human stderr. The probe changes no installed files, re-runs on idempotent scans, and downloads a wheel only when a copy is installed. When the wheel cannot judge a copy (an sdist pin, a failed download or sha256 check, an installed copy of another build) and no other variant proved it patched, the scan fetches that patch's record (`…/patches/view/`, for that purl only) and judges the copy by its `afterHash` map; otherwise it fetches no patch record. A copy that record cannot judge either (the fetch fails, or `--offline`) is unjudged: no warning, but the same run's `--vex` withholds the purl (`stale_install_unverified`), even with `--vex-no-verify`. Missing/unreadable files alone do not prove staleness; lock-only checkouts stay quiet. Dry runs skip the probe. Same-run VEX excludes positively stale Python packages (qualifier-insensitive), even with `--vex-no-verify` or a healthy copy in another interpreter; if nothing remains to attest, the command exits 1 with `no_applicable_patches`. Reinstall from the rewritten lock in the affected interpreter and verify with `socket-patch vex`. A stale Hatch environment instead names `hatch env remove ` / `hatch env prune`: Hatch's pip installer (and uv before Hatch 1.16) keeps a same-version release, so only a recreated env picks up the patch. The same Hatch envs are what agent mode patches and `vex` judges for a Hatch project. +### Patch mirror (experimental) + +**Experimental, and outside the semver policy.** This is for Socket Enterprise customers whose builds may only install through their own artifact host. `--patch-mirror-url` is hidden from `--help`. The flag, `SOCKET_PATCH_MIRROR_URL`, `SOCKET_PATCH_MIRROR_AUTHORIZATION`, the `redirect_mirror_*` codes and everything in this section may change or be removed in any release without a semver bump. The one exception is the rule "A run that was not given the mirror leaves its pins alone" below: it holds for every hosted run, mirror or not, and is additive (MINOR). + +A patch mirror is a caching proxy the operator runs in front of the patch server that serves `https://patch.socket.dev/` at `/`. `--patch-mirror-url ` / `SOCKET_PATCH_MIRROR_URL` names that base. It is tested against an Artifactory generic remote repository with npm, pnpm, yarn classic, yarn berry, pip, uv and Bundler; nothing else is tested yet, a Maven, Gradle or sbt build included. + +* **Operator configuration only.** The base is read from the flag or the environment, never from `socket.yml` or any other repository file (the socket.yml trust boundary below): it names the host every later install fetches from and the host whose references `vex` attests. It must be an `http(s)` URL with a host and no userinfo, query or fragment, in the characters a grant URL may hold; trailing slashes are dropped. Anything else is a usage error (exit 2, `Error: --patch-mirror-url (or SOCKET_PATCH_MIRROR_URL) is not a usable patch mirror base URL: `), checked before the project is read. The message never repeats the value. `apply` is the exception: it does not use the mirror and runs unattended from install hooks, so there a base it cannot use is no mirror and no error. +* **What hosted mode writes.** `scan` / `get --mode hosted` move each granted reference onto the mirror before anything else reads it: the artifact URL, and the registry index URL of a gem or Maven-layout reference, keep their path (grant token and patch uuid included) and take the base in place of scheme and authority. The integrity pins are unchanged, and a re-run is idempotent. A Go reference is not moved: its pin is the module path `patch.socket.dev/gopatch/`, which names no host to fetch from. A Cargo or NuGet reference is not written at all: `skipped` / `redirect_mirror_unsupported` (their registry index is rendered by the patch server with its own absolute download URLs, which a mirror cannot rewrite; the detail names `--mode vendored`). A reference with no path to serve, or whose mirrored URL the grant gate refuses, is `skipped` / `redirect_mirror_url_refused`. +* **A project already pinned on the patch server.** npm, pnpm, yarn classic, `requirements.txt` and uv locks are re-pinned onto the mirror by the same run. The yarn berry, Bun, Bundler, Poetry and PDM rewriters own an earlier pin only on the host they are writing, so they leave a direct pin as it is: the patch is `skipped` (`redirect_unconfirmed`) and a `redirect_mirror_pin_not_moved` warning names the file, the host and the remedy (`rollback `, then the same command again). +* **The run's own downloads** (a wheel's metadata, a tarball's `package.json`, a gem for the stale-install refresh) read the patch server, not the mirror, which may want credentials the run does not have. +* **The mirror is checked before anything is pinned.** One `HEAD` for each mirrored reference the project's files do not already hold (a re-run that pins nothing new sends none), in candidate order, after the unconfigured-host guard and before the rewrite. The request carries no Socket credential, carries `SOCKET_PATCH_MIRROR_AUTHORIZATION` as its `Authorization` header when that is set, and follows no redirect. 2xx: that pin proceeds. 401 / 403 / 407, or a 3xx: nothing more is asked, every pin the mirror has not already refused proceeds, and one `redirect_mirror_unverified` warning says they were not checked. A transport failure, or no answer within the per-attempt timeout: nothing more is asked, and that reference and every one not yet asked about are `skipped` / `redirect_mirror_unavailable`. Any other answer: that reference is `skipped` / `redirect_mirror_unavailable`. Nothing is written for a skipped reference, and a warning names its package; when the mirror served none of the references it was asked about there is one warning instead, about the mirror. Exit 0 either way, files byte-identical for the skipped references, no `.npmrc` setting when nothing was pinned. No retry. A dry run checks too. +* **What every command reads.** The mirror's base counts as a patch host for `list`, `vex`, `rollback`, `remove`, `vendor`, `repair` and `scan` ("Patch hosts" under Manifest-less VEX): a mirrored reference is the patch server's own path under that base. A URL elsewhere on the mirror's host, in another repository of it, is not one: unlike a `--patch-server-url` origin, which holds its whole host, the mirror holds only what is under its base. Without the setting those references are on a host the run was not given and are not hosted pins. What a command downloads for itself while it restores a pin (a yarn berry checksum) comes from the patch server, never the mirror. +* **A source of the project's own on the mirror's host.** The yarn berry, Bun, Poetry and PDM rewriters take over an earlier hosted pin in place when it shares the new one's origin and file name. Under a mirror it must also sit under the mirror's base, so a fork of the same name and version in another repository of that host stays the project's own source and is refused as any other foreign source is. +* **A run that was not given the mirror leaves its pins alone.** Before the rewrite, a candidate whose patch uuid one of the project's candidate files already pins through a URL in the hosted shape on an origin that is neither Socket's patch server, a configured origin, nor an origin of this run's own grants is `skipped` / `redirect_pin_on_unconfigured_host`, with a warning naming the file, the host and the setting. Nothing is written for it. This holds with or without a mirror configured: a hosted run never moves a pin off a host it does not know onto the patch server. +* **In-memory engine.** `hosted-bundle` and the embedding addon have no mirror setting: they write the patch server's own URLs, and they too leave a pin on a host their grants do not name alone (`redirect_pin_on_unconfigured_host`). + ### socket.yml patch policy (v5.0) A repository can **narrow** what `scan` patches with a `patches` block in its root `socket.yml` (the Socket scanner's config file; `version: 2` keeps every other consumer working — they strip or ignore the block). Usage guide: [repository patch policy](../../docs/configuration.md#repository-patch-policy). @@ -691,7 +708,7 @@ Every VEX statement's impact string records which patch-application mode persist **No product to detect.** The product is read from the git `origin` remote, then from the name and version in `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `composer.json`, `pom.xml`, the root's single `.csproj` or its single `.gemspec`. An application with none of these (a Bundler or pip project in a directory that has no git remote) has no product, and the run stops with `product_undetected` (`vex`: exit 2; embedded `--vex`: the host command's exit 1). No product is assumed: a directory name is not an identity (one checkout is `app` in a container and `workspace` in CI, and unrelated projects share those names), so a document issued under it could be read as a statement about another project. The message says what detection read, names any of those files that exists without a usable name and version, and ends with the run's own flag and a value built from the directory name, ready to paste: `… Pass a name that identifies this project, for example: --product pkg:generic/my-app` (`--vex-product …` on an embedded run). A run started in the project directory says `the current directory` instead of `.`. -**Patch hosts (manifest-less VEX).** A hosted lockfile reference counts only when it points at Socket's patch server or the operator's `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin. A hosted URL on any OTHER host — a staging patch server used without `--patch-server-url`, or a look-alike host — is not a hosted pin at all (v5.0: there is no redirect ledger to vouch for it), so `vex` does not attest it, and `list`, `rollback`, `remove`, `vendor` and `repair` do not see it either. Pass `--patch-server-url` for a non-production patch server. A pre-v5 redirect ledger still on disk does not change this: a record whose lockfile or recorded wiring resolves the package from a hosted-shaped URL on an origin this run was not given (or one carrying userinfo) is never attested from the ledger, whatever is installed and even under `--no-verify` / `--vex-no-verify`: `vex` reports it as `redirect_unwired`, with a `vex_claim_unwired` note naming the cause. (A patch that `apply`'s manifest also records is still verified as an agent-mode patch, on that record.) `scan` does not treat such wiring as gone: its takeover and hosted-wiring advisories still see it. +**Patch hosts (manifest-less VEX).** A hosted lockfile reference counts only when it points at Socket's patch server or the operator's `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin or under the `--patch-mirror-url` / `SOCKET_PATCH_MIRROR_URL` base. A hosted URL on any OTHER host — a staging patch server used without `--patch-server-url`, or a look-alike host — is not a hosted pin at all (v5.0: there is no redirect ledger to vouch for it), so `vex` does not attest it, and `list`, `rollback`, `remove`, `vendor` and `repair` do not see it either. Pass `--patch-server-url` for a non-production patch server. A pre-v5 redirect ledger still on disk does not change this: a record whose lockfile or recorded wiring resolves the package from a hosted-shaped URL on an origin this run was not given (or one carrying userinfo) is never attested from the ledger, whatever is installed and even under `--no-verify` / `--vex-no-verify`: `vex` reports it as `redirect_unwired`, with a `vex_claim_unwired` note naming the cause. (A patch that `apply`'s manifest also records is still verified as an agent-mode patch, on that record.) `scan` does not treat such wiring as gone: its takeover and hosted-wiring advisories still see it. ### Manifest-less VEX (lockfile discovery) @@ -725,7 +742,7 @@ Discovery is read-only, never touches the network, and never fails the run: a ma Recognition rules that hold for every ecosystem: -* **Patch hosts.** A hosted reference counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin, with no userinfo. The URL must be one every client reads alike: a backslash, whitespace or a control character anywhere in it (after the lock format's own `\/` unescape and the trim of surrounding whitespace) makes it no reference, because URL parsers disagree about the host of a URL such as `https://patch.socket.dev\@evil.example/…`. Its path must be one the patch server serves a patch under: `patch//…`, `patch-registry//…` (`` one of `npm`, `pypi`, `cargo`, `golang`, `gem`, `composer`, `maven`, `nuget`) or `gopatch/…`, first in the path on Socket's own server and under any path prefix on a `--patch-server-url` deployment, with no percent-encoded `/` or `\` (`%2f`, `%5c`) anywhere in it. The uuid is the LAST canonical-uuid segment below that level, because grant tokens may themselves be uuid-shaped. A URL on a patch host that fails these rules is not a hosted pin (`vex` does not attest it; `list`, `rollback`, `remove` and `scan` do not manage it, and a hosted `scan` does not write one), but a uuid in its path still counts as *mentioned* for the liveness gates below, so no ledger record is revived from it. The Go module prefix is fixed. `socket-patch-` registry / repository / source names count only through a pin. For a URL on any other host, see **Patch hosts** above. +* **Patch hosts.** A hosted reference counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin or under the `--patch-mirror-url` / `SOCKET_PATCH_MIRROR_URL` base, with no userinfo. The URL must be one every client reads alike: a backslash, whitespace or a control character anywhere in it (after the lock format's own `\/` unescape and the trim of surrounding whitespace) makes it no reference, because URL parsers disagree about the host of a URL such as `https://patch.socket.dev\@evil.example/…`. Its path must be one the patch server serves a patch under: `patch//…`, `patch-registry//…` (`` one of `npm`, `pypi`, `cargo`, `golang`, `gem`, `composer`, `maven`, `nuget`) or `gopatch/…`, first in the path on Socket's own server and under any path prefix on a `--patch-server-url` deployment, with no percent-encoded `/` or `\` (`%2f`, `%5c`) anywhere in it. The uuid is the LAST canonical-uuid segment below that level, because grant tokens may themselves be uuid-shaped. A URL on a patch host that fails these rules is not a hosted pin (`vex` does not attest it; `list`, `rollback`, `remove` and `scan` do not manage it, and a hosted `scan` does not write one), but a uuid in its path still counts as *mentioned* for the liveness gates below, so no ledger record is revived from it. The Go module prefix is fixed. `socket-patch-` registry / repository / source names count only through a pin. For a URL on any other host, see **Patch hosts** above. * **Pins, not definitions.** A registry, index or source *definition* alone (cargo `[registries]`, nuget ``, pom ``, uv index tables, `.npmrc`) never makes a reference, because it survives a reverted pin. Sections the package manager ignores are not read: npm's v2 `dependencies` mirror, a `.cargo/config.toml` shadowed by `.cargo/config`. A Socket pin inside a maven `` is diagnosed, never a reference. * **Contested locks.** When one lock wires a package to a patch and another lock resolves the same `name@version` from a non-Socket source, the build's bytes depend on which package manager runs. The reference is then dropped with a `patched_ref_unattributable` diagnostic naming both files. This applies across npm / pnpm / yarn / bun and across uv / pylock / poetry / pdm / Pipfile.lock / requirements. A PEP 723 script lock is a lock like the others: `uv run` installs its script's dependencies from it, so a script lock that resolves the release from the index contests a project lock's reference and the reverse (both writers rewire script locks, so a re-run clears it). A Python lock also contests itself: another entry of the **same** `uv.lock`, script lock, `pylock*.toml`, `poetry.lock`, `pdm.lock` or `Pipfile.lock` that resolves the wired release from a non-Socket source (a uv marker fork, a Poetry multiple-constraint dependency, a PEP 751 marker split, a PDM extras variant, the other `Pipfile.lock` category) is installed wherever its markers, extras or category select it, so the reference is withheld until `scan` / `vendor` rewires every entry. Requirements and `pyproject.toml` / `hatch.toml` declarations are not lock entries, and a Socket reference there is withheld (`patched_ref_unattributable`, naming the other requirement) while another requirement of the same package can install the same release where the reference does not apply: beside a reference that carries an environment marker, any requirement of the set that is not an exact pin of another release; beside a reference in an extra, a dependency group or a Hatch environment, any requirement of the package in `[project].dependencies` (which `pip install .` reads), or an exact pin of the release in any table not installed together with the reference. A reference and a compatible requirement that pip resolves together (one requirement set, or `[project].dependencies` with the project's extras) resolve to the reference, so an unmarked reference is not contested there. The root `requirements.txt` and its in-root `-r` includes count as one lock here: pip reads them as one requirement set, where a direct reference wins over a compatible `==` pin of the same version in another file of the set (#1086). A PyPI version matches under PEP 440 equality, so a requirements line pinned in any spelling that can only install the wired release (`==1.2.3.0`, `==v1.2.3`, `===1.2.3`) contests it like `==1.2.3` does; a range or wildcard (`~=1.2.3`, `==1.2.*`) does not (#1602). A **bundled** npm copy (`inBundle: true`, or v1 `bundled: true`, beneath a dependency that bundles it; the root project's own bundles install from the lock, #1618) of the same `name@version` contests the reference too, in the same lock, in the other npm lock of a shrinkwrap/package-lock pair, or in any other lock. npm unpacks it from the parent package's tarball, so no rewire reaches it and it stays unpatched. Bun and vlt unpack bundled copies the same way (#469, #471). pnpm unpacks bundled copies too, but its lock cannot tie one to a reference (see **Unattested references** below). For Bun, that is a `bun.lock` entry whose meta is `{ "bundled": true }`, or a `bun.lockb` record that a dependency edge with the `bundled` behavior bit reaches, including one Bun shares with a regular install. Such an entry is never a reference, and it contests the reference the same way. For vlt, the lock records no node for a bundled copy, so the copy is found in the installed store: a real package directory inside a store package's own `node_modules`. yarn classic's `yarn.lock` records nothing about bundled copies either (#758), so for a lock with a Socket wiring they are found in the installed `node_modules`: each package a parent's `bundleDependencies` / `bundledDependencies` names inside the parent's own `node_modules`, the dependencies of those beside them, and everything nested under them. Such a copy contests the reference in `yarn.lock` (the diagnostic names the parent's lock entry and the copy's directory); a lock-only `vex` with no `node_modules` can't see it. Hosted and vendored scans skip these copies with `redirect_bun_bundled_instance_skipped` / `redirect_vlt_bundled_instance_skipped` / `vendor_bundled_instance_skipped`, and a yarn classic hosted scan names one with `redirect_copy_unpatched`. When a bundled copy is the only instance, vendoring refuses with `vendor_lock_entry_not_rewritable`. A reference withheld only because such an unreachable copy installs beside it (a bundled npm / Bun / vlt copy, an npm copy beneath a `hasShrinkwrap` package (#753) or one npm installs from a git / url / `file:` spec, or a yarn classic git or `file:` directory block of the same `name@version`) is still the rewriter's own wiring of exactly one package version: it is never attested, but `rollback`, `remove`, `list` and the hosted → vendored takeover treat it as an ordinary hosted pin and restore its upstream registry entry (#828), rather than refusing it with `hosted_wiring_contested`. Another entry of the **same** npm, Bun or yarn lock that resolves the wired `name@version` from a non-Socket source (for example a workspace member added after the rewire, then `npm install` / `bun install` / `yarn install`; for yarn berry, a registry locator such as `left-pad@npm:1.3.0` beside the hosted `…::__archiveUrl=` one) contests the reference too (#588). The package manager installs both entries, and that copy stays unpatched. Re-running `scan` / `vendor` rewires every copy. * **Unattested references.** Some evidence shows a build may run a copy no wiring reaches, but cannot be tied to the reference's exact `name@version` or cannot say the build runs it. The reference then stays a reference: `list`, `rollback` and `remove` find it, and the ledgers' liveness gates (`vendor --check`, `scan`) keep treating the wiring as live, because re-running `scan` / `vendor` could never clear the evidence. Only `vex` omits it, as a run warning and as the `failed[].reason`. Thirteen cases besides Gradle's (`vex_gradle_lock_above_base`) and NuGet's (`vex_nuget_lock_other_version`, see the table above); one of them, a vendored pin in a lock the package manager is told not to read (`vex_lock_not_read`), has its own bullet below; another, **hosted sbt pins a build bypasses** (`vex_sbt_pin_bypassed`), is under **Hosted sbt** above: **pnpm bundled copies** (`vex_pnpm_bundled_copy`): a `packages:` entry's `bundledDependencies:` names the copies pnpm unpacks from that package's own tarball, but pnpm never locks them, so the bundled version is not in the lock. A pnpm reference whose package name a `bundledDependencies` list in the same lock names is omitted whatever its version (a missed attestation when the bundled copy is another version, never a false one), and `bundledDependencies: true` (or a value that cannot be read) omits every reference of that lock. It reaches no other lock. **deno.lock** (`vex_deno_lock_copy`): `deno install` installs a `package.json` project's npm dependencies from `deno.lock` and never reads `package-lock.json` / `pnpm-lock.yaml` / `yarn.lock`, so an npm-family reference whose `name@version` the `deno.lock` npm section also locks is omitted (#406). Whether Deno or another package manager populates `node_modules` (`nodeModulesDir: "manual"` allows either) is not in the files, so this holds whatever `nodeModulesDir` says. **Maven reactors** (`vex_maven_reactor_root`): a hosted patch in a reactor (a root `pom.xml` that declares `` / ``, or whose subprojects Maven 4 discovers) is omitted while a module that resolves the patched version, or may, is reached by no pin and no declaration of the patch, or a declaration may still build the unpatched version: a module's own literal `` overrides a pin above it (#261). The note names the module. A reactor one of whose poms cannot be read is omitted the same way, whatever its root pins. See [Hosted Maven reactors and property versions](#hosted-maven-reactors-and-property-versions). **Maven classifier copies** (`vex_maven_classifier_unpatched`): a `` dependency of a hosted-pinned GA (or one whose `` implies a classifier or carries code, such as `test-jar`) is its own coordinate; the classifier-less pin does not retarget it and the grant serves only the main jar, so Maven fetches it from the next repository. When a classifier copy other than `sources` / `javadoc` (`tests`, a native or JDK-specific build), live or inside ``, resolves any version other than the suffixed pin — its own literal ``, a `` entry with the same classifier (a profile's overriding the live one), or no version in the root pom — the pin is omitted. A pre-v5 single-pom vendored repository holds only the main jar, so any such copy omits its reference. `sources` / `javadoc` copies never run on a classpath and do not withhold the statement; hosted scan only warns about them (`redirect_maven_classifier_unsupported`) and skips the dep, writing nothing, when an executable copy may resolve anything but the suffixed pin (so it never writes a pin this rule withholds). **Maven profile declarations** (`vex_maven_profile_unpatched`): a `` declaration of a pinned GA's main jar in the root `pom.xml` at any version other than the pin (the base release, another release, a `${property}` the pom does not resolve) overrides the pin whenever that profile is active, so the reference is omitted. A profile `` entry always counts; a profile `` entry counts only when management governs the pin (no direct `` outside ``), since a direct literal version beats management. A declaration without a `` takes the live pin, and a `pom` (or `java-source` / `javadoc`) typed one carries no code; neither withholds the statement. Activation is not evaluated: every profile counts, as in vendored mode, which rewrites profile declarations too; hosted scan leaves them as written and writes no pin they bypass (`redirect_maven_profile_dependency_unpatched`). A pin spelled through a property counts the same way: a `${property}` `` is interpolated through every property it names, the project's own `${project.version}` included (itself often spelled through `${revision}`), at most 16 levels deep (a cycle or an undefined name never resolves to a pin), and when any profile's `` — or any combination of profiles' together, each nested reference read with that combination active, as the vendored planner weighs them too — redefine one of those properties so that it resolves anything other than the pin (or nothing), the reference is omitted too; a `.mvn/maven.config` user property wins over every profile's. A profile declaration reads its own profile's `` first. When more than 8 profiles may change such a version, or a profile's `` cannot be parsed, the reference is omitted (fail closed); the rest of the pom is still read. One pom's `${property}` texts are also given a fixed amount of interpolation in all (about four million property lookups, which a pom reaches only with thousands of different texts that each take hundreds of lookups under many profile combinations); a `${property}` read after that counts as unresolved, the same fail-closed way. The statement names the profiles that move the pin and the version they move it to. Coordinates count the same way: when a profile's `` (or several profiles' together, or the profile holding the declaration) redefine a property a `${property}` `` / `` names, so that the declaration carrying the pin names another package (the pin then no longer applies to the patched GA) or another declaration names the patched GA (its main jar, or an executable classifier copy) at a version other than the pin, or as a classifier copy with no version of its own, the reference is omitted; a profile value the pom cannot resolve may name any package, a user property wins over every profile's, and the same 8-profile and unreadable-profile limits fail closed. Each such reading is weighed like a declaration of its own: a retargeted classifier copy (a `test-jar` is the `tests` copy; an explicit `` names the copy whatever the type) by the classifier rule (`vex_maven_classifier_unpatched`, so a pre-v5 vendored repository, which serves only the main jar, is bypassed even at the vendored version), any other non-`jar` type not at all, and a reading that moves the coordinates and the version onto the pin together does not withhold the statement. Hosted scan refuses such a dep (`redirect_maven_profile_coordinates_unpatched`), and vendored mode leaves the declaration as written and does not pin its root (`coordinates_overridden`), except a retargeted classifier copy, which it reports as `classifier_declared` with the root still pinned. The same holds for a pre-v5 single-pom vendored repository, which serves only the original version. **Maven parent poms** (`vex_maven_parent_declares`): a pin that only the root pom's `` carries, for a package the root pom does not declare in its own `` (what hosted scan adds for a dependency with no literal ``), governs the declaration the build inherits. Maven merges a parent pom's `` into the child, and management fills in only a dependency that declares no version (Maven 3.9.16: with the parent declaring the package at a literal version the build resolves the upstream jar and never asks for the patched version). The parent chain is followed through the checkout (``, default `../pom.xml`, when the pom there is the parent the reference names). When a parent pom, one above it, or a profile of either declares the package with a version, the reference is omitted, naming that pom; declare the package in the root pom with no `` (its own declaration replaces the inherited one and the pin governs it) or drop the version from the parent's declaration. The nearest parent declaring it with no version, or a chain read to a pom with no `` that never declares it, leaves the pin attested. A parent that cannot be read (`` or no pom at the relative path, so Maven resolves it from a repository; a pom outside the git checkout; one that does not parse) may declare it either way, so the reference then attests only from the installed copy of the patched version (`-socket.` in the local repository, hash-verified) and is `package_not_found` until a build has resolved it. A pin on the root pom's own declaration, or one governing a declaration the root pom makes with no version, is not weighed against any parent. **Maven user properties** (`vex_maven_config_override`): Maven resolves a `${property}` `` from `.mvn/maven.config`'s `-D` user properties before the pom's `` (`${project.*}` / `${pom.*}` always come from the model). A pin the pom spells through a property that `.mvn/maven.config` redefines resolves that other version, so the reference is omitted. A Socket version that only a `-D` user property supplies is not `pom.xml` wiring (hosted scan never writes one, and `rollback` / `remove` never edit `.mvn/maven.config`): it is no reference at all, only a diagnostic. **pnpm branch locks** (`vex_pnpm_git_branch_lockfile`): with `gitBranchLockfile` on, pnpm installs a git branch from its own `pnpm-lock..yaml` (the **pnpm per-branch locks** state above), which no wiring reaches. `vex` does not decide whether the setting is on: the presence of any branch lock (at the root, or in a workspace member that installs from its own lock under `sharedWorkspaceLockfile: false`, read from the project's files and never from the environment) omits every reference of the root pnpm lock (`pnpm-lock.yaml`, or `shrinkwrap.yaml` without it) and of each member's own lock, hosted and vendored alike, whatever `gitBranchLockfile` or pnpm's `lockfile` setting says (#556; deterministic and fail-closed). This is checked only when a root or member pnpm lock wires a patch. Hosted mode refuses every pnpm pin when the setting is on and vendored mode refuses the project, so this also covers wiring written before the branch lock appeared. The run warning names the branch lock in the marked lock's own directory. Rush locks are not affected. **Self-patched npm and Bun packages** (`vex_npm_patched_dependency`): a reference in `package-lock.json` / `npm-shrinkwrap.json` / `bun.lock` / `bun.lockb` to a package the project patches itself is omitted, by the hosted rewriters' own per-dependency rule (`redirect_npm_patched_dependency_skipped`, #711; `redirect_bun_patched_dependency_skipped`, #367): a root `package.json` `patchedDependencies` key for its `name@version`, its bare name, or `name@` of any tarball (npm and Bun alike); for npm also a non-null `patched` record on any present npm lock's entry for that `name@version`; for `bun.lock` also the keys it mirrors (the run warning then names `bun.lock` as the key's source). npm applies the user's diff on top of whatever tarball the lock names and Bun applies it only to the resolution it is keyed on, so both modes leave such a package unwired; this covers pins an earlier release wrote, and an npm one fails `npm ci` with `EPATCHFAILED` until `socket-patch rollback ` restores the registry entry (the remedy says so). A key naming the pinned tarball itself (a patch made on top of the pin) is no conflict for the hosted rewriters, which keep the pin, but the reference is omitted all the same: the user's diff runs over the Socket bytes, never the Socket patch alone; a reference with neither a hosted url nor a vendored tarball is matched by its `name@version` and counts every tarball key. Only a reference that survives the lock contests carries this reason. **Packages patched with pnpm, Yarn or patch-package** (`vex_npm_patched_dependency` too): a reference of any npm-family lock (pnpm, Rush, yarn classic and berry, vlt, npm, Bun; hosted or vendored) to a `name@version` the project patches itself is omitted, read from the project, its declared workspace members and (on disk) the ancestor workspace settings up to the git repository root, by the consent gate's own walk: pnpm's `patchedDependencies` (`package.json`'s `pnpm.patchedDependencies`, `pnpm-workspace.yaml`'s `patchedDependencies`, and the `patchedDependencies` map or a `(patch_hash=…)` key suffix of the package's own in the reference's own pnpm lock or the root one; a patched peer's hash inside a dependent's key does not count), a yarn berry `patch:` descriptor outside Yarn's builtin compatibility patches (`package.json`'s dependency maps and `resolutions`, or a `patch:` locator `yarn.lock` resolves, including one wrapping the hosted tarball), or a patch-package file (`patches/+.patch`, or under a `--patch-dir` a `package.json` script names; it counts for every version of ``, since patch-package applies it to whichever version is installed). A key's version may be exact, a range or absent (every version), and metadata that cannot be read (an unparseable `package.json`, a `--patch-dir` the script computes) omits every npm reference. This is the `user_patch_conflict` consent gate's evidence (see [Existing project patches](#existing-project-patches)) minus native npm / Bun and the other ecosystems, and the evidence the in-memory hosted engine withholds a package by. The package manager (or the postinstall) applies the user's diff on top of whatever tarball the lock names, so the install never runs the Socket patch alone: the reference is omitted even when an `--allow-patch-conflicts` run wired it, for as long as the patch is declared. Yarn classic and vlt have no patch mechanism of their own (vlt's `vlt.json` `modifiers` rewrite specs, not contents), so patch-package is what is read for them. npm's and Bun's own top-level `patchedDependencies` are judged by the rule above, not here. **Go workspaces** (`go_workspace_override`): a hosted or vendored Go `replace` that the workspace go resolves (an ancestor's `go.work`, one `GOWORK` names, members outside the project included) overrides, ignores or rejects, by the same check hosted and vendored modes run before writing one (`redirect_golang_workspace_conflict`, `vendor_go_workspace_conflict`), is omitted under the reason `vex`'s own go.mod gate reports. So is a replace in the root `go.work` while go builds another one. **Unpinned Cargo declarations** (`vex_cargo_unpinned_declaration`): cargo resolves each source on its own, so when one declaration of a crate pins it to `socket-patch-` but another — the root `Cargo.toml`'s or a workspace member's, read as hosted scan reads them — is not pinned, with a version requirement that may select the locked patched version, that declaration builds a separate, unpatched copy, possibly of the very version the pin covers. Not pinned means crates.io or another registry that no Socket `[patch.]` registry entry redirects, a `registry-index`, a path or git — the last three even beside `registry = "socket-patch-"`, which they override (hosted scan's own pinned check). The `Cargo.lock` reference is omitted, naming the unpinned declaration's manifest. Hosted scan pins every declaration, so this is a partial revert or a hand edit; the warning says whether re-running `scan` pins it again (a crates.io declaration whose requirement matches no other locked version of the crate) or cannot (`redirect_cargo_toml_dep_unrewritable`: roll the patch back or change the declaration). A declaration whose requirement excludes the patched version (cargo resolves it to another version), and a `[workspace.dependencies]` entry no manifest inherits, do not withhold the statement. When the lock already holds a same-version copy from another source, the reference is withheld as an unpatched lock copy instead. A declaration pinned to ANOTHER Socket patch, which the lock does not resolve the crate from, adds that patch's `Cargo.toml` reference, so the package is gated as `wiring_conflict`. **Vendored Cargo copies a `[patch]` does not reach** (`vex_cargo_patch_bypassed`): a `[patch.]` entry replaces the crate only in declarations from ``, and `vendor` writes `[patch.crates-io]`. So a vendored reference is omitted when a live declaration of the crate (the root `Cargo.toml`'s or a workspace member's) names another registry, a `registry-index` or a git repository, with a version requirement that may select the patched version, unless a `[patch]` table for its source points at the same copy (`[patch.]` for a registry dependency, `[patch.""]` for a `registry-index` one, `[patch.""]` for a git one; URLs compare as cargo canonicalizes them: the host's case, a lock's `registry+` prefix, one trailing `/` and a `.git` suffix ignored, a GitHub URL case-insensitive, a `sparse+` index another source than the git one, and the crates.io git index is crates.io); cargo builds that declaration's own, unpatched copy. Workspace members are read for this whenever the root manifest or the project cargo config wires a Socket copy through `[patch]`, whatever the lock says. It is also omitted when `Cargo.lock` locks the crate at exactly the patched version from any source (a registry, crates.io included, or git) beside the vendored copy; the warning names that lock entry. A path declaration (the user's own fork) does not withhold the statement. Only `[patch]` entries cargo honors count: a manifest entry the project config or a crates.io URL table replaces covers nothing. The warning ends with the remedy for its evidence: for a declaration, re-running `vendor` does not help (declare the crate from crates.io there, point a `[patch]` table for its source at the copy, or roll the patch back); for a stale lock entry a `[patch]` table at the copy redirects (crates.io under `[patch.crates-io]`, another registry or a git repository under its own table; a registry's table by its index URL, or by its name when the project's cargo config defines that registry), re-lock (`cargo update -p "#@"`, or re-run `vendor` for crates.io); for another lock entry, `cargo tree -i "#@"` names the dependency that declares it. The commands name the entry by its source, because the bare `@` also fits the copy the lock holds at `+socket.` and cargo refuses it as ambiguous. A source with a character a shell would read inside the quotes is not printed: the command then takes the bare crate name, which cargo answers with the specs it accepts. @@ -1972,7 +1989,7 @@ Restore the system but keep the local patch state for a later re-apply: manifest v5.0 replaces v4's per-purl reverts and whole-ledger reverse replay (`revert_remaining_redirect_edits`) with ONE mechanism, the **upstream restore** (core `patch/redirect/upstream/`), shared by `rollback`, `remove` and the hosted → vendored takeover: -* **Scope.** The hosted pins are what lockfile discovery finds — `(purl, patch uuid, files wiring it)`, recognized only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin. A scoped rollback (paths / identifiers / `--ecosystems`) restores exactly the pins in scope; each pin restores or refuses on its own (there is no whole-ledger replay, and a pre-v5 ledger's edits are never replayed). A pin discovery cannot see is out of reach: a lockless cargo `registry = "socket-patch-"` pin, a nuget exact-id mapping with no `packages.lock.json`, a gem wired only in the `Gemfile` (pre-bundler-2.6 mixed state) — restore those files from version control. +* **Scope.** The hosted pins are what lockfile discovery finds — `(purl, patch uuid, files wiring it)`, recognized only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin or under the `--patch-mirror-url` / `SOCKET_PATCH_MIRROR_URL` base. A scoped rollback (paths / identifiers / `--ecosystems`) restores exactly the pins in scope; each pin restores or refuses on its own (there is no whole-ledger replay, and a pre-v5 ledger's edits are never replayed). A pin discovery cannot see is out of reach: a lockless cargo `registry = "socket-patch-"` pin, a nuget exact-id mapping with no `packages.lock.json`, a gem wired only in the `Gemfile` (pre-bundler-2.6 mixed state) — restore those files from version control. * **What a restore does.** Every file wiring the pin is rewritten back to the DEFAULT UPSTREAM registry entry for `name@version`, re-resolving whatever the entry pins (tarball URL, integrity, checksum, hashes) from the public registry; only the hosted entries change and every other byte stays the file's own. A pin is **all-or-nothing**: refused in one of its files, it is restored in none of them, so no pin is left half hosted. Nothing reaches disk until every pin has resolved, and `--dry-run` resolves exactly like a wet run — registry lookups included — and skips only the write. Per format: * **npm family** — `package-lock.json` / `npm-shrinkwrap.json`, `yarn.lock` (classic and berry), `pnpm-lock.yaml` / `shrinkwrap.yaml`, `bun.lock`: resolution + integrity (+ shasum where recorded) from the npm registry's version document (`SOCKET_NPM_REGISTRY`); a yarn berry lock whose settings name another registry reads that registry's document instead — the registry yarn itself resolves the package against (#1017): a scoped package's `npmScopes..npmRegistryServer`, else `YARN_NPM_REGISTRY_SERVER`, else `npmRegistryServer`, each from the closest `.yarnrc.yml` (`YARN_RC_FILENAME`) that sets it among the lock's directory, every parent directory and then the home directory, with a `${VAR:-default}` / `${VAR-default}` reference to an unset variable read as its default, as yarn reads it (a reference to a variable that is set is not expanded — the rc file must not choose which of the process's variables lands in a requested URL — and, like a reference yarn cannot expand or an `npmScopes` written as a flow mapping, falls back to the default registry with `upstream_registry_fallback`), so a mirror's off-path `dist.tarball` keeps its `::__archiveUrl=` binding (a berry entry the hosted pin keyed by its tarball URL also takes the version document's `bin` back, in place of the served tarball's own spelling the pin wrote, as yarn writes the `npm:` entry, #1131), and a pnpm lock whose sibling settings name a registry reads that registry's document — `.npmrc` `registry` (or, for a scoped name, `@scope:registry`); on pnpm 10 a pnpm-workspace.yaml `registries` map instead when present; on pnpm 11+ (or an unknown major) pnpm-workspace.yaml `registries."@scope"` / `registry` / `registries.default` too, ahead of the matching `.npmrc` key — so a mirror's `tarball:` comes back as pnpm recorded it and a URL conventional under that registry stays derived (falling back to the default registry, with `upstream_registry_fallback`, when the mirror can't be read; a value holding an unexpanded `${VAR}` is read as unset). pnpm 12.0–12.12 (named by the same install record, package.json pin or rush.json `pnpmVersion` as tier 2 below) compare a scoped package's URL with the default registry's rather than its scope registry's, so they record `tarball:` for every package from a scope registry other than the default one, and the restore gives it back, both in the entry and when reading the lock's evidence below (#1619); pnpm 12.13+ derive it from the scope registry like pnpm <= 11. Whether a restored pnpm entry gets its `tarball:` back follows `lockfileIncludeTarballUrl` as the pnpm that wrote the lock read it (#902), from the strongest evidence available: (1) the lock's own unpinned registry resolutions (a bare one proves it off; a URL pnpm could have derived proves it on; pnpm 11+'s env lockfile document does not count); else (2) the settings file the installed pnpm major reads (`node_modules/.modules.yaml` `packageManager`, else package.json `packageManager`; a pre-9 lock or shrinkwrap means pnpm <= 8): `.npmrc` `lockfile-include-tarball-url` on pnpm <= 9, pnpm-workspace.yaml `lockfileIncludeTarballUrl` on pnpm >= 11 (also assumed for a lock carrying an env lockfile document), the workspace file then `.npmrc` on pnpm 10; else (3) pnpm 10's reading. A Rush lock (`common/config/rush/pnpm-lock.yaml` or a subspace lock, with `rush.json` at the Rush root) takes its pnpm major from rush.json `pnpmVersion` instead of tier 2's install record and package.json pin. A 9.0 lock may come from pnpm 9, 10 or 11+, so when tier 3's reading differs from pnpm 9's (`.npmrc` only) or pnpm >= 11's (pnpm-workspace.yaml only) — e.g. `.npmrc` on with the workspace file silent, or the workspace file setting it with `.npmrc` silent or disagreeing — the restore follows pnpm 10 but warns `upstream_pnpm_tarball_setting_guessed` (once per lock, naming the entries); a URL pnpm records anyway (not derivable from the registry) never warns. A `bun.lock` 4-tuple's registry slot is rebuilt the way Bun writes it (#992): `""` for a package from registry.npmjs.org, otherwise the full tarball URL — Bun 1.1.39–1.3.6 read `""` as npmjs whatever the project configures. The registry is the one Bun resolves the package against: a scope's `.npmrc` `@scope:registry` or `bunfig.toml` `[install.scopes]` entry, else `BUN_CONFIG_REGISTRY` / `NPM_CONFIG_REGISTRY`, then the `.npmrc` `registry` or `bunfig.toml` `[install] registry`. Bun reads these from the files beside the lock and from the user's own (#1276): `$XDG_CONFIG_HOME/.npmrc` when it exists, else `~/.npmrc` (never `NPM_CONFIG_USERCONFIG`), and the global bunfig `$XDG_CONFIG_HOME/.bunfig.toml` when `XDG_CONFIG_HOME` is set, else `~/.bunfig.toml`. A key set beside the lock wins over the user's own file of the same kind. Between the two kinds, Bun ≤ 1.3 takes any `.npmrc` over any bunfig and Bun ≥ 1.4 any bunfig over any `.npmrc`. A lockfileVersion-2 `bun.lock` is Bun ≥ 1.4's (Bun 1.3 ignores it); when the kinds disagree on a package of a lockfileVersion-0/1 `bun.lock` (which Bun 1.4 keeps as is) or a `bun.lockb`, the restore refuses that pin rather than guess. Only the conventional token variables (`NPM_TOKEN`, `NODE_AUTH_TOKEN`, `BUN_AUTH_TOKEN`) expand in the project's files; the user's own files expand any variable, as Bun does. Its version document's `dist.tarball` fills the slot, and when it can't be read (`upstream_registry_fallback`) the default registry's conventional URL is re-based on it. The `bun.lockb` takeover restore records the same URL. Side settings: a project `.npmrc` that is exactly `allow-remote=all\n` is deleted once no root npm lock entry is hosted, otherwise a remaining top-level `allow-remote=all` warns `npm_allow_remote_left`; a `pnpm-workspace.yaml` that is exactly the scaffold hosted mode creates (in either line ending) is deleted once neither `pnpm-lock.yaml` nor, under `sharedWorkspaceLockfile: false`, any member's own `pnpm-lock.yaml` is hosted any more (#1268; a keyless file hosted mode spliced the scaffold into gets the spliced lines removed, #1096), otherwise a remaining `trustLockfile: true` warns `pnpm_trust_lockfile_left`. **`bun.lockb` (binary)**: `rollback` and `remove` refuse it (the checkout remedy). The hosted → vendored takeover and the eject DO restore it, since the vendor ledger then records the rebuilt record as its pre-vendor original: the native codec turns each hosted remote-tarball record back into Bun's npm registry record for `name@version` (the registry's `dist.tarball` + `dist.integrity`, the package metadata hash re-derived, the hosted URL string dropped from the string pool). The hosted rewrite keeps the registry record's inactive bytes (padding, semver) in the tarball record, so a lock it wrote comes back byte for byte — early writers' uninitialized padding included; a record without them (an older socket-patch or a Bun re-save) is rebuilt the way Bun writes one, and refused for a prerelease/build version. A lock the hosted rewrite had to normalize is marked in the root package's resolution value bytes (which no Bun reader reads): a binary format 1 lock it promoted to format 2 is demoted back to its exact format-1 bytes (verified by promoting it again, otherwise refused), and a lock whose workspace dependency behaviors it normalized is refused with the `git checkout -- bun.lockb` remedy. * **vlt** — `vlt-lock.json`: slot [2] from the registry's `dist.integrity`, slot [3] per the lock's own convention (see the vlt hosted-mode contract); every hosted instance of the pin together. @@ -2125,6 +2142,7 @@ Empty string means unset at every layer: exported-but-empty flag-bound vars are | `SOCKET_VENDOR_SOURCE` | `--vendor-source` | `service` | `auto` is a compatibility alias for `service`; `build` is rejected. | | `SOCKET_VENDOR_URL` | `--vendor-url` | (active API/proxy base) | Vendoring-service package-reference host. | | `SOCKET_PATCH_SERVER_URL` | `--patch-server-url` | (server-returned) | Rewrites the prebuilt-archive download host. | +| `SOCKET_PATCH_MIRROR_URL` | `--patch-mirror-url` | (none) | **Experimental** (no stability guarantee). Base URL of a patch-server mirror; see "Patch mirror (experimental)". Empty = unset. | | `SOCKET_OFFLINE` | `--offline` | `false` | — | | `SOCKET_STRICT` | `--strict` | `false` | Mismatch policy for the in-place apply paths; see "Global arguments". | | `SOCKET_GLOBAL` | `--global` / `-g` | `false` | — | @@ -2164,6 +2182,7 @@ Empty string means unset at every layer: exported-but-empty flag-bound vars are |---|---|---| | `SOCKET_NO_CONFIG` | `false` | Truthy (`1`/`true`/`yes`/`on`): disable the socket-cli persisted-config fallback layer entirely — pure flag+env behavior. Also the test-hermeticity switch (the workspace `.cargo/config.toml` exports it as `1` for every cargo-run process). | | `SOCKET_NO_API_TOKEN` | `false` | Truthy: ignore **ambient** API tokens (the `SOCKET_API_TOKEN` env var and the socket-cli config token); only an explicit `--api-token` flag authenticates. Peer alias: `SOCKET_CLI_NO_API_TOKEN`. | +| `SOCKET_PATCH_MIRROR_AUTHORIZATION` | (none) | **Experimental** (no stability guarantee; see "Patch mirror (experimental)"). The operator's `Authorization` header value for the patch mirror (`Bearer …`, `Basic …`). Sent only with the hosted run's check of the mirror; never written to a file, never passed to a child process, and there is no flag for it. | | `SOCKET_NO_UPDATE_CHECK` | `false` | Truthy: disable the passive update notice entirely (see "Passive update notice"). Explicit `--update` still works. Also a test-hermeticity switch (the workspace `.cargo/config.toml` exports it as `1` for every cargo-run process). No `SOCKET_CLI_*` alias (socket-cli has no equivalent today). | | `SOCKET_PLAIN` | (unset) | Overrides the detection of an AI coding agent's terminal (see "Human output conventions", **Agents**). Truthy (`1`/`true`/`yes`/`on`/`y`/`t`): the plain output a pipe gets, on a terminal too. Any other value: the full terminal output, whatever agent variable is set. Presentation only: no result, exit code or `--json` document depends on it. | @@ -2513,6 +2532,11 @@ command's integration tests check its output. | `vendor_lock_checksums_unsupported` / `vendor_stale_lock_checksum` | `failed` | vendor (gem): an ambiguous/platform CHECKSUMS entry, or a v1-wired lock whose stale token blocks the hot path (run `vendor --revert` + re-vendor). | | `platform_gem_unsupported` | `failed` | vendor / scan / get `--mode vendored` (gem): a precompiled platform gem that cannot be vendored. `Gemfile.lock` pins the gem as platform gems (`nokogiri (1.18.10-x86_64-linux-gnu)`) and also holds the plain ` ()` entry of the `ruby` platform, which no prebuilt tree serves (the detail names it; `bundle lock --remove-platform ruby` drops it); or it pins the gem as platform gems and its `BUNDLED WITH` is absent or names a Bundler older than 2.3, the oldest the vendored layout is verified on (the detail names the version needed; `bundle update --bundler` re-locks); or the installed copy is a platform gem and the lock names no platform for the gem; or the patch record is for one platform's build (`?platform=`). Nothing is written. A locked platform the patch service has no prebuild for is not this failure: see `vendor_prebuilt_platform_pending`. | | `redirect_unattributable` | `skipped` (hosted event) | scan/get `--mode hosted`: the rewriters would pin the patch, but lockfile discovery over the result reads that pin as contested (another lock or requirements file resolves the same version elsewhere, or the pin is not one the package manager consumes), so `vex`, `rollback`, `remove` and `vendor` would refuse it. The candidate is left out of the rewrite, so nothing is written for it; the detail carries discovery's findings. Exit code unchanged. | +| `redirect_mirror_unsupported` / `redirect_mirror_url_refused` | `skipped` (hosted event) + `warnings[]` | **Experimental** (patch mirror). scan/get `--mode hosted` with `--patch-mirror-url`: the patch's ecosystem cannot be served through a mirror (`redirect_mirror_unsupported`: Cargo, NuGet; use `--mode vendored`), or its reference could not be moved onto the mirror (`redirect_mirror_url_refused`). Nothing is written for it. See "Patch mirror (experimental)". | +| `redirect_mirror_unavailable` | `skipped` (hosted event) + `warnings[]` | **Experimental** (patch mirror). scan/get `--mode hosted` with `--patch-mirror-url`: the mirror did not serve the reference the run asked it for, or could not be reached. That reference is skipped and nothing is written for it. One warning per package, or a single warning about the mirror when it served none of the references it was asked about. | +| `redirect_mirror_unverified` | `warnings[]` (warning, one per run) | **Experimental** (patch mirror). scan/get `--mode hosted` with `--patch-mirror-url`: the mirror answered 401 / 403 / 407 or a redirect to the run's check, so the references were written without confirming the mirror serves them. `SOCKET_PATCH_MIRROR_AUTHORIZATION` lets the next run check. | +| `redirect_mirror_pin_not_moved` | `warnings[]` (warning) | **Experimental** (patch mirror). scan/get `--mode hosted` with `--patch-mirror-url`: a yarn berry, Bun, Bundler, Poetry or PDM file pins the patch directly on the patch server, and that pin cannot be moved onto the mirror in place. It is left as it is (the patch is `skipped` / `redirect_unconfirmed`); `rollback ` and a re-run pin it through the mirror. | +| `redirect_pin_on_unconfigured_host` | `skipped` (hosted event) + `warnings[]` | scan/get `--mode hosted`: a project file already pins the patch through a URL in the hosted shape on a host this run was not given as a patch mirror or patch server. The pin is left as it is; set `--patch-mirror-url` to that mirror's base, or `--patch-server-url` when the host is another patch server. | | `redirect_grant_malformed` / `redirect_grant_insecure_url` | `skipped` (hosted event) + `warnings[]` | scan/get `--mode hosted` and the in-memory engine: the patch API's reference for the package has a field that is not in the shape the project's files take verbatim (`redirect_grant_malformed`; the detail names the field), or names a plain-`http` artifact or registry index URL on an origin that is neither `--patch-server-url` nor a loopback address (`redirect_grant_insecure_url`; the detail names the host). Refused before anything is planned: nothing is written for the package and it stays unpatched. Exit code unchanged. See the grant gate under "Scan modes (v5.0)". | | `redirect_copy_unpatched` | `warnings[]` (warning) | scan/get `--mode hosted` (pnpm, yarn): the pin was written, but its own lock also installs an unpatched copy of the same `name@version`, which no rewrite reaches: a pnpm `file:` directory or tarball dependency (#935), or a yarn `file:` tarball / directory, URL tarball or berry `portal:` / `link:` dependency declared under another name (`lp2@file:…`, #939, #1236; not repeated for a copy a yarn classic skip warning already names), or a yarn classic copy installed from a parent's `bundleDependencies` (#758). The detail names the lock entry (for a bundled copy, the parent's entry and the copy's directory). Neither the run's own `--vex` nor a later `vex` attests the package (`vex` warns `patched_ref_unattributable`), but discovery still manages the pin, so `rollback` / `remove` undo it. | | `redirect_pin_lockless` | `warnings[]` (warning) | scan/get `--mode hosted` (nuget, cargo): the pin was written without a lockfile that records its version, so `vex` cannot attest it and `rollback` / `remove` / `vendor` refuse it as unattributable. The detail names the lockfile to create (`dotnet restore --use-lock-file`, `cargo generate-lockfile`) before re-running the hosted scan. | @@ -3047,7 +3071,7 @@ Versioning lives in **`Cargo.toml`** at the workspace root (`version = "..."`) a | Add a new visible alias to an existing subcommand | **MINOR** | | Fix a bug without changing any of the above | **PATCH** | -A subcommand or flag that `--help` does not show and this document does not describe is outside the contract: it can change or be removed in any release. +A subcommand or flag that `--help` does not show and this document does not describe is outside the contract: it can change or be removed in any release. So is anything this document marks **Experimental**: its flags, variables, codes and behavior can change or be removed in any release. After bumping `Cargo.toml`, run: diff --git a/crates/socket-patch-cli/src/args.rs b/crates/socket-patch-cli/src/args.rs index cdda3eb5a..919bc0cbe 100644 --- a/crates/socket-patch-cli/src/args.rs +++ b/crates/socket-patch-cli/src/args.rs @@ -172,6 +172,26 @@ pub struct GlobalArgs { )] pub patch_server_url: Option, + /// Base URL of a mirror of Socket's patch server: a caching proxy in + /// front of https://patch.socket.dev, such as an Artifactory remote + /// repository. Hosted mode writes its patch references under this base + /// instead of the patch server's own host (same path), and every + /// command accepts lockfile URLs on this origin as hosted patch + /// references. Set it the same way wherever socket-patch runs for the + /// project. + // + // EXPERIMENTAL (hidden from `--help`): for organizations that install + // through their own artifact host. The flag, its variables and its + // warning codes carry no stability guarantee yet; see "Patch mirror + // (experimental)" in CLI_CONTRACT.md. + #[arg( + help_heading = GLOBAL_OPTIONS, + long = "patch-mirror-url", + env = "SOCKET_PATCH_MIRROR_URL", + hide = true + )] + pub patch_mirror_url: Option, + /// Disable network access; fail if remote data is needed. #[arg( help_heading = GLOBAL_OPTIONS, @@ -626,6 +646,42 @@ impl GlobalArgs { .map_or_else(|| self.telemetry_auth(), TelemetryAuth::for_client) } + /// Refuse a `--patch-mirror-url` that cannot prefix a committed URL, + /// once, before any command reads the project. The message says what + /// is wrong and never repeats the value: a base given with credentials + /// in it must not be echoed into a terminal or a CI log. + pub fn validate_patch_mirror(&self) -> Result<(), String> { + let Some(raw) = self + .patch_mirror_url + .as_deref() + .filter(|url| !url.trim().is_empty()) + else { + return Ok(()); + }; + socket_patch_core::patch::redirect::mirror::PatchMirror::parse(raw) + .map(|_| ()) + .map_err(|why| { + format!( + "--patch-mirror-url (or SOCKET_PATCH_MIRROR_URL) is not a usable patch \ + mirror base URL: {why}" + ) + }) + } + + /// The patch-server mirror this run was given (`--patch-mirror-url`). + /// [`Self::validate_patch_mirror`] refused a base that does not parse + /// before the command started; a struct literal with one has no mirror. + pub(crate) fn patch_mirror( + &self, + ) -> Option { + self.patch_mirror_url + .as_deref() + .filter(|url| !url.trim().is_empty()) + .and_then(|url| { + socket_patch_core::patch::redirect::mirror::PatchMirror::parse(url).ok() + }) + } + /// The vendoring-service config every vendor entry point (`vendor`, /// `scan`/`get --mode vendored`) builds from the same flags, so they /// commit byte-identical artifacts for the same patch. Clap validated @@ -728,6 +784,7 @@ pub const GLOBAL_ARG_ENV_VARS: &[&str] = &[ "SOCKET_VENDOR_SOURCE", "SOCKET_VENDOR_URL", "SOCKET_PATCH_SERVER_URL", + "SOCKET_PATCH_MIRROR_URL", "SOCKET_OFFLINE", "SOCKET_STRICT", "SOCKET_GLOBAL", @@ -800,6 +857,7 @@ impl Default for GlobalArgs { maven_config: None, vendor_url: None, patch_server_url: None, + patch_mirror_url: None, offline: false, strict: false, allow_patch_conflicts: false, diff --git a/crates/socket-patch-cli/src/commands/agent_download.rs b/crates/socket-patch-cli/src/commands/agent_download.rs index 52efd79eb..dd455eb5d 100644 --- a/crates/socket-patch-cli/src/commands/agent_download.rs +++ b/crates/socket-patch-cli/src/commands/agent_download.rs @@ -398,6 +398,9 @@ pub struct DownloadParams { /// `--patch-server-url`: the extra origin whose URLs count as hosted /// when lockfile discovery reads the project's hosted pins. pub patch_server_url: Option, + /// `--patch-mirror-url`: the patch-server mirror whose origin counts as + /// hosted the same way. + pub patch_mirror_url: Option, } impl DownloadParams { @@ -1202,8 +1205,10 @@ pub(crate) async fn vendored_download_refusals( prior: Option<&crate::ecosystem_dispatch::NpmCrawlSnapshot>, ) -> VendorRefusals { let cwd = params.cwd.as_path(); - let origins = - crate::commands::hosted_unwind::patch_server_origins_of(params.patch_server_url.as_deref()); + let origins = crate::commands::hosted_unwind::patch_server_origins_of( + params.patch_server_url.as_deref(), + params.patch_mirror_url.as_deref(), + ); let pins = hosted_pins(cwd, origins.clone()).await; let options = params.crawler_options(); let installed = crate::commands::vendor::installed_lookup(&options, prior); diff --git a/crates/socket-patch-cli/src/commands/get.rs b/crates/socket-patch-cli/src/commands/get.rs index 021a1ecd2..a1da229eb 100644 --- a/crates/socket-patch-cli/src/commands/get.rs +++ b/crates/socket-patch-cli/src/commands/get.rs @@ -2321,6 +2321,7 @@ fn get_download_params(args: &GetArgs, save_only: bool, persist_blobs: bool) -> ecosystems: args.common.ecosystems.clone(), persist_blobs, patch_server_url: args.common.patch_server_url.clone(), + patch_mirror_url: args.common.patch_mirror_url.clone(), } } @@ -4209,6 +4210,7 @@ mod tests { ecosystems: None, persist_blobs: false, patch_server_url: None, + patch_mirror_url: None, } } @@ -4960,6 +4962,7 @@ mod tests { // The vendor-detached posture this fn exists for. persist_blobs: false, patch_server_url: None, + patch_mirror_url: None, } } diff --git a/crates/socket-patch-cli/src/commands/hosted_unwind.rs b/crates/socket-patch-cli/src/commands/hosted_unwind.rs index cfe31abfd..393573524 100644 --- a/crates/socket-patch-cli/src/commands/hosted_unwind.rs +++ b/crates/socket-patch-cli/src/commands/hosted_unwind.rs @@ -47,19 +47,53 @@ impl HostedLegOutcome { } /// The patch-server origins that count as hosted, besides Socket's own: -/// the operator's `--patch-server-url` (discovery's allowlist). +/// the operator's `--patch-server-url` and the origin of their +/// `--patch-mirror-url` (discovery's allowlist). pub(crate) fn patch_server_origins(common: &GlobalArgs) -> Vec { - patch_server_origins_of(common.patch_server_url.as_deref()) + patch_server_origins_of( + common.patch_server_url.as_deref(), + common.patch_mirror_url.as_deref(), + ) } -/// [`patch_server_origins`] from the `--patch-server-url` value itself, for -/// callers that carry it outside a [`GlobalArgs`]. -pub(crate) fn patch_server_origins_of(patch_server_url: Option<&str>) -> Vec { - patch_server_url - .into_iter() - .filter(|url| !url.trim().is_empty()) - .map(str::to_string) - .collect() +/// [`patch_server_origins`] from the `--patch-server-url` and +/// `--patch-mirror-url` values themselves, for callers that carry them +/// outside a [`GlobalArgs`]. +/// +/// * `--patch-server-url` is listed as given, and holds its whole origin. +/// * The mirror is listed as a mirror's base +/// ([`socket_patch_core::origin::mirror_entry`]), which holds only what +/// is under that base: a mirrored reference is the patch server's own +/// path under it, and the rest of the mirror's host is not the mirror. A +/// base that does not parse ([`PatchMirror::parse`]) is not listed. +/// * With a mirror, the FIRST entry is always the patch server (the +/// operator's, else Socket's own): an upstream restore downloads from the +/// first entry, and what it downloads is not a patch the mirror was +/// checked for, nor something the run holds the mirror's credentials for. +pub(crate) fn patch_server_origins_of( + patch_server_url: Option<&str>, + patch_mirror_url: Option<&str>, +) -> Vec { + use socket_patch_core::patch::redirect::mirror::PatchMirror; + fn given(url: Option<&str>) -> Option<&str> { + url.map(str::trim).filter(|url| !url.is_empty()) + } + let server = given(patch_server_url).map(str::to_string); + let mirror = given(patch_mirror_url) + .and_then(|url| PatchMirror::parse(url).ok()) + .map(|mirror| socket_patch_core::origin::mirror_entry(mirror.base())); + match mirror { + None => server.into_iter().collect(), + Some(mirror) => vec![ + server.unwrap_or_else(|| { + format!( + "https://{}", + socket_patch_core::origin::SOCKET_PATCH_SERVER_HOST + ) + }), + mirror, + ], + } } /// The gem leg of the unwind's installed-tree care (the "Gem stale-install @@ -291,3 +325,37 @@ pub(crate) async fn run_hosted_leg( .extend(crate::commands::vlt_heal::rollback_heal(common, &unwound).await); out } + +#[cfg(test)] +mod tests { + use super::patch_server_origins_of; + + /// The list every reader recognizes hosted pins against: the patch + /// server as given, the mirror as a mirror's base, and with a mirror + /// the patch server first, because an upstream restore downloads from + /// the first entry and must never ask the mirror. + #[test] + fn the_patch_server_comes_first_and_the_mirror_is_its_base() { + let mirror = "https://repo.example/artifactory/socket-patch"; + let listed = format!("mirror+{mirror}"); + assert!(patch_server_origins_of(None, None).is_empty()); + assert!(patch_server_origins_of(Some(" "), Some("")).is_empty()); + assert_eq!( + patch_server_origins_of(Some("http://127.0.0.1:4545"), None), + ["http://127.0.0.1:4545"] + ); + assert_eq!( + patch_server_origins_of(None, Some(&format!("{mirror}/"))), + ["https://patch.socket.dev", listed.as_str()] + ); + assert_eq!( + patch_server_origins_of(Some("http://127.0.0.1:4545"), Some(mirror)), + ["http://127.0.0.1:4545", listed.as_str()] + ); + // A base that does not parse is no mirror: it is not listed, and + // its userinfo never becomes an accepted origin. + assert!( + patch_server_origins_of(None, Some("https://user:secret@repo.example/x")).is_empty() + ); + } +} diff --git a/crates/socket-patch-cli/src/commands/scan/hosted.rs b/crates/socket-patch-cli/src/commands/scan/hosted.rs index bc2de9139..a3b1cdcc4 100644 --- a/crates/socket-patch-cli/src/commands/scan/hosted.rs +++ b/crates/socket-patch-cli/src/commands/scan/hosted.rs @@ -75,6 +75,253 @@ pub(super) fn prune_ignored_warning() -> RunWarning { ) } +/// The operator's `Authorization` header value for the patch mirror +/// (`Bearer `, `Basic `), read from the environment only. It +/// is sent with the one request that checks the mirror before a run pins +/// anything to it, and nowhere else: it is never written to a file, and +/// the package managers the run starts do not inherit it +/// (`CHILD_ENV_SCRUB`). +pub(crate) const PATCH_MIRROR_AUTHORIZATION_ENV: &str = "SOCKET_PATCH_MIRROR_AUTHORIZATION"; + +/// Skip reason and warning code of the patches a run withheld because its +/// patch mirror (`--patch-mirror-url`) did not serve the artifact it was +/// asked for, or could not be reached. Nothing is written for them. +pub(crate) const MIRROR_UNAVAILABLE: &str = "redirect_mirror_unavailable"; + +/// Warning code of a run whose patch mirror asked for credentials the run +/// was not given: the pins are written, unchecked. +pub(crate) const MIRROR_UNVERIFIED: &str = "redirect_mirror_unverified"; + +/// Warning code of a patch the project pins directly on the patch server, +/// in a lock format whose pin a run cannot move onto the patch mirror in +/// place: the pin is left as it is. +pub(crate) const MIRROR_PIN_NOT_MOVED: &str = "redirect_mirror_pin_not_moved"; + +/// One [`MIRROR_PIN_NOT_MOVED`] warning for each mirrored candidate the +/// rewrite did not pin while one of the project's files still pins its +/// patch directly on the patch server. npm, pnpm, yarn classic, +/// requirements.txt and uv locks are re-pinned onto the mirror by their +/// rewriters; the others own an earlier pin only on the host they are +/// writing, so they leave a direct pin alone as a source the user chose. +/// Without this the run would report such a patch with only the +/// rewriter's own, unrelated-looking reason. +fn mirror_pins_not_moved( + mirror_sources: &socket_patch_core::hosted::engine::MirrorSources, + candidates: &[socket_patch_core::hosted::engine::Candidate], + confirmed: &[(String, String)], + files: &std::collections::BTreeMap, +) -> Vec { + // By origin: an earlier pin of the patch may carry another grant token + // in its path. + let served: Vec = mirror_sources + .served_urls() + .filter_map(socket_patch_core::origin::bare_origin) + .collect::>() + .into_iter() + .collect(); + let mirrored: std::collections::BTreeSet<&str> = mirror_sources.mirrored_urls().collect(); + candidates + .iter() + .filter(|c| mirrored.contains(c.dep.artifact_url.as_str())) + .filter(|c| !confirmed.iter().any(|(_, uuid)| uuid == &c.dep.patch_uuid)) + .filter_map(|c| { + let (file, host) = files.iter().find_map(|(path, text)| { + socket_patch_core::vex::discover::hosted_host_in_text( + text, + &c.dep.patch_uuid, + &served, + ) + .map(|host| (path, host)) + })?; + Some(socket_patch_core::hosted::engine::warning( + MIRROR_PIN_NOT_MOVED, + format!( + "{}: {file} pins it directly on {host}, and a pin in that file cannot be \ + moved onto the patch mirror {} in place, so it was left as it is; run \ + `socket-patch rollback {}` and then this command again", + c.purl, + mirror_sources.base(), + c.purl + ), + )) + }) + .collect() +} + +/// Ask the patch mirror about every mirrored pin this run would newly +/// write, before anything is written +/// ([`socket_patch_core::api::client::MirrorProber::probe`]): every +/// later install fetches from the mirror, so a mistyped base, a repository +/// that does not proxy the patch server, or a patch the mirror cannot fetch +/// must not reach a lockfile. A pin the project's files already hold is not +/// asked about again, so a re-run that changes nothing sends nothing. +/// +/// * The mirror serves the artifact: the pin goes ahead. +/// * It asks for credentials, or redirects: nothing more is asked (every +/// answer would be the same), the pins it has not already refused are +/// written, and one [`MIRROR_UNVERIFIED`] warning says they were not +/// checked. Package +/// managers hold the mirror's credentials, not this CLI, so most private +/// mirrors answer this way unless the operator sets +/// [`PATCH_MIRROR_AUTHORIZATION_ENV`]. +/// * It cannot be reached, or does not answer in time: nothing more is +/// asked (each further request would wait as long for the same), and +/// that candidate and every one not yet asked about are withheld with +/// [`MIRROR_UNAVAILABLE`]. +/// * Any other answer: that candidate is withheld with [`MIRROR_UNAVAILABLE`] +/// and nothing is written for it. A lockfile pinned to a mirror that +/// does not serve the patch breaks every install, where an unpatched one +/// only stays as it was. When the mirror served none of the artifacts it +/// was asked about, one warning says so of the mirror; otherwise each +/// withheld package gets its own. +/// +/// Candidates the mirror does not carry (Go modules) are untouched. +async fn check_mirror( + api_client: &socket_patch_core::api::client::ApiClient, + mirror_sources: &socket_patch_core::hosted::engine::MirrorSources, + files: &std::collections::BTreeMap, + candidates: &mut Vec, + skipped: &mut Vec, + warnings: &mut Vec, + status: &mut crate::ui::StatusLine, +) { + use socket_patch_core::api::client::MirrorProbe; + use socket_patch_core::hosted::engine::{self, SkippedPatch}; + let base = mirror_sources.base(); + let mirrored: std::collections::BTreeSet<&str> = mirror_sources.mirrored_urls().collect(); + // The pins to ask about, one per artifact URL: mirrored, and not + // already in the project's files in either spelling a lock uses. + let already_pinned = |url: &str| { + // As a lock spells it: plain, wholly percent-encoded (yarn berry), + // or with every slash escaped (an older composer.lock). + let encoded = socket_patch_core::utils::uri::encode_uri_component(url); + let escaped = url.replace('/', "\\/"); + files.values().any(|text| { + text.contains(url) || text.contains(encoded.as_str()) || text.contains(escaped.as_str()) + }) + }; + let mut asked = std::collections::BTreeSet::new(); + let new_pins: Vec = candidates + .iter() + .map(|c| &c.dep) + .filter(|dep| mirrored.contains(dep.artifact_url.as_str())) + .filter(|dep| { + let index = dep + .registry_override + .as_ref() + .map(|ov| ov.index_url.trim_end_matches('/')); + !already_pinned(&dep.artifact_url) && !index.is_some_and(already_pinned) + }) + .filter(|dep| asked.insert(dep.artifact_url.clone())) + .map(|dep| dep.artifact_url.clone()) + .collect(); + if new_pins.is_empty() { + return; + } + let authorization = std::env::var(PATCH_MIRROR_AUTHORIZATION_ENV).ok(); + let mut refused: std::collections::BTreeMap = std::collections::BTreeMap::new(); + let mut unverified = None; + // Every pin from `from` on is withheld for `reason`: the mirror gave no + // answer, and each further request would wait as long for the same. + let withhold_from = + |from: usize, reason: &str, refused: &mut std::collections::BTreeMap| { + let reason = socket_patch_core::utils::redact::redact_urls_in(reason).into_owned(); + for url in &new_pins[from..] { + refused.insert(url.clone(), reason.clone()); + } + }; + let prober = match api_client.mirror_prober(authorization.as_deref()) { + Ok(prober) => Some(prober), + Err(reason) => { + withhold_from(0, &reason, &mut refused); + None + } + }; + for (at, url) in new_pins.iter().enumerate() { + let Some(prober) = prober.as_ref() else { + break; + }; + status.set(format!( + "Checking the patch mirror ({}/{})...", + at + 1, + new_pins.len() + )); + match prober.probe(url).await { + MirrorProbe::Serves => {} + MirrorProbe::Unreachable(reason) => { + withhold_from(at, &reason, &mut refused); + break; + } + MirrorProbe::NeedsCredentials(status) => { + unverified = Some(format!( + "answered HTTP {status}, so socket-patch could not check that it serves the \ + patches: it has no credentials for the mirror, or the ones it was given may \ + not read through it" + )); + break; + } + MirrorProbe::Redirected(status) => { + unverified = Some(format!( + "answered HTTP {status}, a redirect the check does not follow, so \ + socket-patch could not check that it serves the patches" + )); + break; + } + MirrorProbe::Refused(reason) => { + let reason = socket_patch_core::utils::redact::redact_urls_in(&reason).into_owned(); + refused.insert(url.clone(), reason); + } + } + } + if let Some(why) = unverified { + warnings.push(engine::warning( + MIRROR_UNVERIFIED, + format!( + "the patch mirror {base} {why}. The references it had not already refused were \ + written without that check. Run an install to confirm, or set \ + {PATCH_MIRROR_AUTHORIZATION_ENV} to an `Authorization` header value the mirror \ + accepts so the next run can check" + ), + )); + } + if refused.is_empty() { + return; + } + // Nothing the mirror was asked about was served: say it of the mirror, + // once. Otherwise the mirror works and each refused patch is its own + // case (a patch the mirror cannot fetch, a cached miss). + let whole_mirror = refused.len() == new_pins.len(); + if whole_mirror { + let reason = refused.values().next().cloned().unwrap_or_default(); + warnings.push(engine::warning( + MIRROR_UNAVAILABLE, + format!( + "the patch mirror {base} does not serve the patches ({reason}), so nothing was \ + pinned to it; check that it proxies Socket's patch server at that base and \ + that --patch-mirror-url names it exactly" + ), + )); + } + candidates.retain(|c| { + let Some(reason) = refused.get(c.dep.artifact_url.as_str()) else { + return true; + }; + let detail = format!( + "{}: the patch mirror {base} does not serve its patch ({reason}), so nothing was \ + pinned for it", + c.purl + ); + if !whole_mirror { + warnings.push(engine::warning(MIRROR_UNAVAILABLE, detail.clone())); + } + skipped.push(SkippedPatch { + detail: Some(detail), + ..SkippedPatch::new(&c.purl, &c.dep.patch_uuid, MIRROR_UNAVAILABLE) + }); + false + }); +} + /// Warning code inside `vex.warnings` of a hosted run: the patches this /// run pinned are attested from their patch records, not hash-verified /// (their bytes are fetched at install time). @@ -708,6 +955,7 @@ async fn refresh_stale_gems( std::collections::BTreeSet, >, overrides: &[DepOverride], + mirror_sources: &socket_patch_core::hosted::engine::MirrorSources, views: &mut ViewFallback<'_>, ) -> GemRefreshed { use socket_patch_core::patch::gem_refresh::{Source, Wanted}; @@ -767,14 +1015,15 @@ async fn refresh_stale_gems( .iter() .filter(|o| o.ecosystem == "gem" && o.name == name && o.version == version) { + // Downloaded from where the patch server serves them: a + // mirrored reference names the operator's mirror, which may + // want credentials this run does not have. if let Some(ro) = &o.registry_override { - urls.push(format!( - "{}/gems/{file}", - ro.index_url.trim_end_matches('/') - )); + let url = format!("{}/gems/{file}", ro.index_url.trim_end_matches('/')); + urls.push(mirror_sources.fetch_url(&url).into_owned()); } if o.artifact_url.ends_with(&format!("/{file}")) { - urls.push(o.artifact_url.clone()); + urls.push(mirror_sources.fetch_url(&o.artifact_url).into_owned()); } } urls.dedup(); @@ -1197,6 +1446,20 @@ async fn run_redirect_selected_untracked( ); } } + // `--patch-mirror-url`: move the grants onto the operator's mirror + // before anything reads them, so the takeover, the withholds and the + // rewriters all see the URL the project's files will hold. The run's + // own downloads below still read the patch server (`mirror_sources`). + let mut mirror_sources = engine::MirrorSources::default(); + if let Some(mirror) = common.patch_mirror() { + (candidates, mirror_sources) = engine::mirror_candidates( + std::mem::take(&mut candidates), + &mirror, + &crate::commands::hosted_unwind::patch_server_origins(common), + &mut skipped, + &mut grant_warnings, + ); + } // The rollout rows key on the selection's purl spelling; the server's // reference may spell a candidate's `purl` differently. Uuids are unique // across the selection. @@ -1444,6 +1707,47 @@ async fn run_redirect_selected_untracked( CandidateFiles::default() }; + // A patch the project already pins through a host this run was not + // given (a patch mirror left out of this run's settings) is left + // alone, never moved onto the patch server behind the project's back. + // The run's own origins are the configured ones, the origins of its + // grants, and where the mirrored grants are served from (a direct pin + // an earlier run wrote moves onto the mirror). + { + let mut accepted = crate::commands::hosted_unwind::patch_server_origins(common); + let own = socket_patch_core::patch::redirect::upstream::foreign_dep_origins( + candidates.iter().map(|c| &c.dep), + &accepted, + ); + accepted.extend(own); + // By origin: a direct pin an earlier run wrote may carry another + // grant token in its path. + accepted.extend( + mirror_sources + .served_urls() + .filter_map(socket_patch_core::origin::bare_origin), + ); + engine::withhold_foreign_host_pins( + &read.files, + &accepted, + &mut candidates, + &mut skipped, + &mut grant_warnings, + ); + } + if !mirror_sources.is_empty() { + check_mirror( + api_client, + &mirror_sources, + &read.files, + &mut candidates, + &mut skipped, + &mut grant_warnings, + &mut status, + ) + .await; + } + let mut python_metadata = std::collections::BTreeMap::new(); let mut unavailable_python_artifacts = std::collections::BTreeSet::new(); // `pipenv --version` (see `pipenv_major` below), started before the @@ -1483,7 +1787,11 @@ async fn run_redirect_selected_untracked( }; // Lazily built: a dep's attempt starts only once it is pulled here. let mut unstarted = wheel_deps.iter().map(|&(dep, sha256)| { - try_fetch_hosted_wheel_metadata_once(api_client, &dep.artifact_url, sha256) + try_fetch_hosted_wheel_metadata_once( + api_client, + mirror_sources.fetch_artifact_url(&dep.artifact_url), + sha256, + ) }); let mut in_flight: futures_util::stream::FuturesOrdered<_> = unstarted .by_ref() @@ -1513,7 +1821,13 @@ async fn run_redirect_selected_untracked( } }, }; - match finish_hosted_wheel_metadata(api_client, &dep.artifact_url, sha256, attempt).await + match finish_hosted_wheel_metadata( + api_client, + mirror_sources.fetch_artifact_url(&dep.artifact_url), + sha256, + attempt, + ) + .await { Ok(Some(metadata)) => { python_metadata.insert(dep.artifact_url.clone(), metadata); @@ -1535,7 +1849,7 @@ async fn run_redirect_selected_untracked( status.set(format!("Fetching hosted package manifest for {}", dep.name)); match socket_patch_core::hosted::npm_manifest::fetch_hosted_npm_manifest( api_client, - &dep.artifact_url, + mirror_sources.fetch_artifact_url(&dep.artifact_url), dep.integrity.sha512.as_deref(), ) .await @@ -1577,7 +1891,9 @@ async fn run_redirect_selected_untracked( for (url, sha512, dep) in classic_targets { status.set(format!("Fetching hosted tarball for {}", dep.name)); match socket_patch_core::hosted::npm_manifest::fetch_hosted_classic_artifact( - api_client, &url, &sha512, + api_client, + mirror_sources.fetch_artifact_url(&url), + &sha512, ) .await { @@ -1785,6 +2101,12 @@ async fn run_redirect_selected_untracked( // Candidates the rewrite left out because discovery would not attribute // their pin (`engine::rewrite`). skipped.extend(done.unattributed.iter().cloned()); + grant_warnings.extend(mirror_pins_not_moved( + &mirror_sources, + &candidates, + &done.confirmed, + &done.files, + )); let takeover::Finished { warnings: takeover_pre_warnings, migrated: takeover_migrated, @@ -1942,6 +2264,7 @@ async fn run_redirect_selected_untracked( &local_records, &gem_artifact_shas, &done.overrides, + &mirror_sources, &mut views, ) .await; @@ -1960,6 +2283,7 @@ async fn run_redirect_selected_untracked( .or_else(|| done.files.get("Pipfile.lock")) .map(String::as_str), &done.overrides, + &mirror_sources, &local_records, &mut views, ) @@ -3026,6 +3350,21 @@ fn describe_skip_reason(reason: &str) -> String { detail)" .into() } + "redirect_mirror_unsupported" => { + "its registry cannot be served through the patch mirror; use --mode vendored for it (see the warning)" + .into() + } + "redirect_mirror_url_refused" => { + "its reference could not be moved onto the patch mirror (see the warning)".into() + } + "redirect_pin_on_unconfigured_host" => { + "the project already pins it through a host this run was not given, so it was \ + left as it is (see the warning)" + .into() + } + "redirect_mirror_unavailable" => { + "the patch mirror does not serve it, so nothing was pinned (see the warning)".into() + } other => format!("server status `{other}`"), } } diff --git a/crates/socket-patch-cli/src/commands/scan/hosted/python.rs b/crates/socket-patch-cli/src/commands/scan/hosted/python.rs index 9f734042b..3f3071adf 100644 --- a/crates/socket-patch-cli/src/commands/scan/hosted/python.rs +++ b/crates/socket-patch-cli/src/commands/scan/hosted/python.rs @@ -22,7 +22,9 @@ enum Evidence<'a> { /// The patched wheel the lock now pins, fetched on the first installed /// copy found (`Some(None)` once that fetch failed). Wheel { - dep: &'a DepOverride, + /// Where the patch server serves the wheel: the lock's own URL, + /// unless the lock pins it through a patch mirror. + fetch_url: &'a str, sha256: &'a str, fetched: Option>, }, @@ -41,13 +43,12 @@ impl Evidence<'_> { match self { Evidence::Local(record) => Some(judge_installed_record(site, record).await), Evidence::Wheel { - dep, + fetch_url, sha256, fetched, } => { if fetched.is_none() { - *fetched = - Some(fetch_patched_wheel_record(api, &dep.artifact_url, sha256).await); + *fetched = Some(fetch_patched_wheel_record(api, fetch_url, sha256).await); } match fetched { Some(Some(record)) => judge_installed_wheel(site, record).await, @@ -96,6 +97,9 @@ pub(super) async fn stale_install_warnings( pipenv_lock: Option<&str>, // The overrides the rewrite pinned: each patch's wheel URL and sha256. overrides: &[DepOverride], + // Where a wheel pinned through a patch mirror is served from: the + // run's own downloads read the patch server. + mirror_sources: &socket_patch_core::hosted::engine::MirrorSources, // The agent-mode manifest's records, looked up by uuid. local_records: &BTreeMap, views: &mut ViewFallback<'_>, @@ -126,7 +130,7 @@ pub(super) async fn stale_install_warnings( .is_some_and(|path| path.ends_with(".whl")) })?; Some(Evidence::Wheel { - dep, + fetch_url: mirror_sources.fetch_artifact_url(&dep.artifact_url), sha256: dep.integrity.sha256.as_deref()?, fetched: None, }) @@ -382,6 +386,7 @@ mod tests { &BTreeSet::new(), None, &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -407,6 +412,7 @@ mod tests { &BTreeSet::new(), None, &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -455,6 +461,7 @@ mod tests { &BTreeSet::new(), None, &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -477,6 +484,7 @@ mod tests { &BTreeSet::new(), None, &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -510,6 +518,7 @@ mod tests { &BTreeSet::new(), None, &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -552,6 +561,7 @@ mod tests { &BTreeSet::new(), None, &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -599,6 +609,7 @@ mod tests { &pipenv, Some(text), &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -648,6 +659,7 @@ mod tests { &pipenv, Some(docs), &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -723,6 +735,7 @@ mod tests { &pipenv, Some(lock), &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -763,6 +776,7 @@ mod tests { &pipenv, Some(lock), &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -793,6 +807,7 @@ mod tests { &pipenv, Some(lock), &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -821,6 +836,7 @@ mod tests { &pipenv, Some(lock), &[], + &socket_patch_core::hosted::engine::MirrorSources::default(), &ledger, &mut ViewFallback::offline(), ) @@ -971,6 +987,7 @@ mod tests { &BTreeSet::new(), None, &self.overrides, + &socket_patch_core::hosted::engine::MirrorSources::default(), &BTreeMap::new(), &mut views, ) diff --git a/crates/socket-patch-cli/src/commands/scan/mod.rs b/crates/socket-patch-cli/src/commands/scan/mod.rs index 50e7adabd..faa9e34eb 100644 --- a/crates/socket-patch-cli/src/commands/scan/mod.rs +++ b/crates/socket-patch-cli/src/commands/scan/mod.rs @@ -945,6 +945,7 @@ fn download_params(args: &ScanArgs, save_only: bool, json: bool, silent: bool) - ecosystems: args.common.ecosystems.clone(), persist_blobs: args.mode != Some(ScanMode::Vendored), patch_server_url: args.common.patch_server_url.clone(), + patch_mirror_url: args.common.patch_mirror_url.clone(), } } diff --git a/crates/socket-patch-cli/src/commands/vlt_heal.rs b/crates/socket-patch-cli/src/commands/vlt_heal.rs index 112df679a..9b891aeab 100644 --- a/crates/socket-patch-cli/src/commands/vlt_heal.rs +++ b/crates/socket-patch-cli/src/commands/vlt_heal.rs @@ -31,15 +31,23 @@ pub(crate) const REINSTALL_REQUIRED: &str = "redirect_vlt_reinstall_required"; pub(crate) const BUNDLED_INSTANCE_SKIPPED: &str = "redirect_vlt_bundled_instance_skipped"; /// The origins the vlt heal treats as Socket-owned in the final lock: the -/// patch server plus `--api-url`. Unlike discovery's +/// patch server, the patch mirror (`--patch-mirror-url`, where the pins a +/// mirrored run wrote resolve from) and `--api-url`. Unlike discovery's /// [`crate::commands::hosted_unwind::patch_server_origins`], it also counts the /// `--api-url` origin, so a /// setup serving the hosted tarballs from the API origin is healed too. /// Discovery-facing code must use the `hosted_unwind` helper, not this one. fn vlt_heal_origins(common: &crate::args::GlobalArgs) -> Vec { + // The mirror is listed as a mirror's base, as discovery lists it: it + // holds only what is under that base. (Built here, not through + // `hosted_unwind`, which imports this module.) + let mirror = common + .patch_mirror() + .map(|mirror| socket_patch_core::origin::mirror_entry(mirror.base())); common .patch_server_url .iter() + .chain(mirror.iter()) .chain(common.api_url.iter()) .filter(|url| !url.trim().is_empty()) .cloned() diff --git a/crates/socket-patch-cli/src/main.rs b/crates/socket-patch-cli/src/main.rs index dae704f9a..954ba7e97 100644 --- a/crates/socket-patch-cli/src/main.rs +++ b/crates/socket-patch-cli/src/main.rs @@ -130,6 +130,19 @@ async fn run(own_args: Vec, hot_patch_command: Option Vec<(&'static str, Option<&'static str>, fn(&GlobalArg ("--patch-server-url", Some("http://localhost:4026"), |c| { assert_eq!(c.patch_server_url.as_deref(), Some("http://localhost:4026")) }), + ( + "--patch-mirror-url", + Some("https://repo.example/artifactory/socket-patch"), + |c| { + assert_eq!( + c.patch_mirror_url.as_deref(), + Some("https://repo.example/artifactory/socket-patch") + ) + }, + ), ("--offline", None, |c| assert!(c.offline)), ("--strict", None, |c| assert!(c.strict)), ("--allow-patch-conflicts", None, |c| { @@ -254,14 +264,15 @@ fn global_flag_cases_cover_every_global_field() { vendor_source: _, vendor_url: _, patch_server_url: _, + patch_mirror_url: _, maven_config: _, } = common; - // 30 fields ↔ 30 long-flag cases. Bump both this count and add a case when + // 31 fields ↔ 31 long-flag cases. Bump both this count and add a case when // the destructure above forces you to add a field. assert_eq!( global_flag_cases().len(), - 30, + 31, "every GlobalArgs field needs a long-flag case in global_flag_cases()", ); } @@ -455,6 +466,10 @@ fn env_vars_populate_global_args() { ("SOCKET_VENDOR_SOURCE", "service"), ("SOCKET_VENDOR_URL", "https://env-vendor.example.com"), ("SOCKET_PATCH_SERVER_URL", "http://localhost:4026"), + ( + "SOCKET_PATCH_MIRROR_URL", + "https://repo.example/artifactory/socket-patch", + ), ("SOCKET_OFFLINE", "true"), ("SOCKET_STRICT", "true"), ("SOCKET_GLOBAL", "true"), @@ -496,6 +511,10 @@ fn env_vars_populate_global_args() { args.common.patch_server_url.as_deref(), Some("http://localhost:4026") ); + assert_eq!( + args.common.patch_mirror_url.as_deref(), + Some("https://repo.example/artifactory/socket-patch") + ); assert!(args.common.offline); assert!(args.common.strict); assert!(args.common.global); @@ -679,6 +698,7 @@ fn empty_nonbool_env_vars_do_not_crash_the_binary() { "SOCKET_VENDOR_SOURCE", "SOCKET_VENDOR_URL", "SOCKET_PATCH_SERVER_URL", + "SOCKET_PATCH_MIRROR_URL", ] { cmd.env(var, ""); } diff --git a/crates/socket-patch-cli/tests/common/clap_parse.rs b/crates/socket-patch-cli/tests/common/clap_parse.rs index 6a1cc66ba..7e4f772f6 100644 --- a/crates/socket-patch-cli/tests/common/clap_parse.rs +++ b/crates/socket-patch-cli/tests/common/clap_parse.rs @@ -63,6 +63,7 @@ pub fn global_defaults() -> Globals { maven_config: None, vendor_url: None, patch_server_url: None, + patch_mirror_url: None, offline: false, strict: false, allow_patch_conflicts: false, diff --git a/crates/socket-patch-cli/tests/e2e_artifactory_mirror.rs b/crates/socket-patch-cli/tests/e2e_artifactory_mirror.rs new file mode 100644 index 000000000..9128cfedb --- /dev/null +++ b/crates/socket-patch-cli/tests/e2e_artifactory_mirror.rs @@ -0,0 +1,1164 @@ +//! Live proof of the experimental `--patch-mirror-url` against a real +//! Artifactory. +//! +//! Nothing here is mocked. The suite needs an Artifactory instance with a +//! **generic remote repository** whose URL is `https://patch.socket.dev`, +//! and drives the real binary, Socket's production patch API (free +//! patches, no token) and the real package managers against it. It is +//! `#[ignore]`d and skips without its settings, so this repository's CI +//! only compiles it; Socket runs it against a real Artifactory outside +//! this repository. Each install-proof leg: +//! +//! 1. locks a pinned upstream package with the real package manager; +//! 2. runs `scan --mode hosted --patch-mirror-url ` and checks +//! that the pin names the Artifactory repository, not the patch server; +//! 3. installs from the rewritten lock alone, with cold caches, and checks +//! the installed bytes carry the patch. +//! +//! The patched artifact can only have come through Artifactory: the lock +//! names no other host for it. +//! +//! ## Configuration +//! +//! | Variable | Meaning | +//! | --- | --- | +//! | `SOCKET_PATCH_ARTIFACTORY_E2E_BASE` | Base URL of a repository that needs credentials, such as `http://127.0.0.1:8081/artifactory/socket-patch`. Without it every leg skips. | +//! | `SOCKET_PATCH_ARTIFACTORY_E2E_USER`, `..._PASSWORD` | A user with Read and Deploy/Cache on that repository. | +//! | `SOCKET_PATCH_ARTIFACTORY_E2E_ANON_BASE` | Base URL of a repository readable without credentials (the yarn legs need one: yarn sends no credentials for a URL pin). Without it those legs skip. | +//! | `SOCKET_PATCH_ARTIFACTORY_E2E_STRICT=1` | Fail instead of skipping a leg whose settings or toolchain are missing. | +//! +//! ```sh +//! cargo test --locked -p socket-patch-cli --test e2e_artifactory_mirror -- --ignored +//! ``` + +#[path = "common/cache_env.rs"] +mod cache_env; +#[path = "common/envelope.rs"] +mod envelope; +#[path = "common/hermetic.rs"] +mod hermetic; + +use std::path::{Path, PathBuf}; +use std::process::{Command, Output}; + +use envelope::warning_codes; + +const NPM_NAME: &str = "minimist"; +const NPM_VERSION: &str = "1.2.2"; +const PYPI_NAME: &str = "urllib3"; +const PYPI_VERSION: &str = "1.26.18"; +const GEM_NAME: &str = "activestorage"; +const GEM_VERSION: &str = "6.0.3"; +/// The banner every Socket patch adds to the files it changes. +const PATCH_MARKER: &str = "Socket Community Patch"; +const PATCH_HOST: &str = "patch.socket.dev"; + +// --------------------------------------------------------------------------- +// Settings + skip policy +// --------------------------------------------------------------------------- + +fn setting(name: &str) -> Option { + std::env::var(name) + .ok() + .map(|v| v.trim().trim_end_matches('/').to_string()) + .filter(|v| !v.is_empty()) +} + +fn strict() -> bool { + setting("SOCKET_PATCH_ARTIFACTORY_E2E_STRICT").is_some_and(|v| v == "1" || v == "true") +} + +macro_rules! soft_skip { + ($leg:expr, $($arg:tt)*) => {{ + let why = format!($($arg)*); + if strict() { + panic!("STRICT: {} cannot run: {why}", $leg); + } + println!("SKIP {}: {why}", $leg); + return; + }}; +} + +/// The repository that needs credentials, with the credentials. +struct Private { + base: String, + user: String, + password: String, +} + +impl Private { + fn from_env() -> Option { + Some(Private { + base: setting("SOCKET_PATCH_ARTIFACTORY_E2E_BASE")?, + user: setting("SOCKET_PATCH_ARTIFACTORY_E2E_USER")?, + password: std::env::var("SOCKET_PATCH_ARTIFACTORY_E2E_PASSWORD").ok()?, + }) + } + + /// The `Authorization` header value the CLI's own check sends. + fn authorization(&self) -> String { + use base64::Engine as _; + format!( + "Basic {}", + base64::engine::general_purpose::STANDARD + .encode(format!("{}:{}", self.user, self.password)) + ) + } + + /// `(scheme://host[:port], host[:port], host)` of the base. + fn origin(&self) -> (String, String, String) { + let (scheme, rest) = self.base.split_once("://").expect("base is a URL"); + let authority = rest.split('/').next().unwrap_or(rest).to_string(); + let host = authority + .split(':') + .next() + .unwrap_or(&authority) + .to_string(); + (format!("{scheme}://{authority}"), authority, host) + } + + /// `(context root URL, repository key)`: the base is + /// `/`. + fn root_and_repo(&self) -> (String, String) { + let (root, repo) = self.base.rsplit_once('/').expect("base has a repository"); + (root.to_string(), repo.to_string()) + } + + /// A netrc file for the repository's host: how pip and uv are handed + /// credentials for a URL that is not their index. + fn netrc(&self, dir: &Path) -> PathBuf { + let path = dir.join("netrc"); + std::fs::write( + &path, + format!( + "machine {}\nlogin {}\npassword {}\n", + self.origin().2, + self.user, + self.password + ), + ) + .expect("write netrc"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt as _; + std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600)).unwrap(); + } + path + } + + /// The `.npmrc` line that hands npm and pnpm the credentials for every + /// path on the repository's host. + fn npmrc_auth(&self) -> String { + use base64::Engine as _; + format!( + "//{}/:_auth={}\n", + self.origin().1, + base64::engine::general_purpose::STANDARD + .encode(format!("{}:{}", self.user, self.password)) + ) + } +} + +// --------------------------------------------------------------------------- +// Processes +// --------------------------------------------------------------------------- + +fn dump(out: &Output) -> String { + format!( + "exit={:?}\nstdout:\n{}\nstderr:\n{}", + out.status.code(), + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr) + ) +} + +fn read(path: &Path) -> String { + std::fs::read_to_string(path).unwrap_or_else(|e| panic!("read {}: {e}", path.display())) +} + +/// The binary against production, with no ambient Socket settings: no +/// token (the free public proxy), no mirror, no crawler inputs. +fn cli(cwd: &Path) -> Command { + let mut cmd = hermetic::binary_command(); + hermetic::scrub_crawler_inputs(&mut cmd); + cmd.env_remove("SOCKET_PATCH_MIRROR_URL") + .env_remove("SOCKET_PATCH_MIRROR_AUTHORIZATION") + .current_dir(cwd); + cmd +} + +/// `scan --mode hosted --json --yes --patch-mirror-url `; the parsed +/// envelope of a run that exited 0. +fn scan_mirrored(cwd: &Path, base: &str, authorization: Option<&str>) -> serde_json::Value { + let mut cmd = cli(cwd); + cmd.args([ + "scan", + "--mode", + "hosted", + "--json", + "--yes", + "--patch-mirror-url", + base, + ]); + if let Some(value) = authorization { + cmd.env("SOCKET_PATCH_MIRROR_AUTHORIZATION", value); + } + let out = cmd.output().expect("run socket-patch"); + assert!( + out.status.success(), + "scan --mode hosted failed:\n{}", + dump(&out) + ); + serde_json::from_slice(&out.stdout) + .unwrap_or_else(|e| panic!("scan did not emit JSON ({e}):\n{}", dump(&out))) +} + +/// The purls a hosted run pinned. +fn pinned(env: &serde_json::Value) -> Vec { + env["events"] + .as_array() + .map(|events| { + events + .iter() + .filter(|e| e["action"] == "applied" && e["details"]["mode"] == "hosted") + .filter_map(|e| e["purl"].as_str().map(str::to_string)) + .collect() + }) + .unwrap_or_default() +} + +fn has_command(program: &str) -> bool { + let mut probe = pm_command(program); + probe + .arg("--version") + .stdout(std::process::Stdio::null()) + .stderr(std::process::Stdio::null()); + cache_env::isolate(&mut probe); + probe.status().map(|s| s.success()).unwrap_or(false) +} + +/// A package manager with no ambient Socket settings and no ambient +/// registry override. +fn pm_command(program: &str) -> Command { + let mut cmd = hermetic::tool_command(program); + hermetic::scrub_socket_vars(&mut cmd); + hermetic::scrub_extra(&mut cmd, &[hermetic::Extra::Venv, hermetic::Extra::Bundle]); + for key in [ + "NPM_CONFIG_REGISTRY", + "npm_config_registry", + "YARN_NPM_REGISTRY_SERVER", + "YARN_NODE_LINKER", + "PIP_INDEX_URL", + "UV_INDEX_URL", + "NETRC", + ] { + cmd.env_remove(key); + } + cmd +} + +fn tool(cwd: &Path, program: &str, args: &[&str], env: &[(&str, &str)]) -> Output { + let mut cmd = pm_command(program); + cmd.args(args).current_dir(cwd); + cache_env::isolate(&mut cmd); + for (k, v) in env { + cmd.env(k, v); + } + cmd.output() + .unwrap_or_else(|e| panic!("failed to spawn `{program}`: {e}")) +} + +fn assert_pristine(path: &Path, leg: &str) { + let body = read(path); + assert!( + !body.contains(PATCH_MARKER), + "{leg}: the upstream install at {} already carries the patch; every later assertion \ + would be vacuous", + path.display() + ); +} + +fn assert_patched(path: &Path, leg: &str) { + assert!(path.exists(), "{leg}: {} was not installed", path.display()); + assert!( + read(path).contains(PATCH_MARKER), + "{leg}: {} was installed from the mirrored lock but does not carry the patch", + path.display() + ); +} + +/// The lock pins the mirror and no longer names the patch server. +fn assert_mirrored_pin(lock: &str, base: &str, leg: &str) { + assert!( + lock.contains(&format!("{base}/patch")), + "{leg}: the lock does not pin the Artifactory repository {base}:\n{lock}" + ); + assert!( + !lock.contains(PATCH_HOST), + "{leg}: the lock still names {PATCH_HOST}:\n{lock}" + ); +} + +struct Fixture { + _tmp: tempfile::TempDir, + proj: PathBuf, + scratch: PathBuf, +} + +fn fixture() -> Fixture { + let tmp = tempfile::tempdir().expect("tempdir"); + let proj = tmp.path().join("proj"); + let scratch = tmp.path().join("scratch"); + std::fs::create_dir_all(&proj).unwrap(); + std::fs::create_dir_all(&scratch).unwrap(); + Fixture { + _tmp: tmp, + proj, + scratch, + } +} + +fn write_npm_manifest(proj: &Path, extra: &str) { + std::fs::write( + proj.join("package.json"), + format!( + r#"{{"name":"artifactory-e2e","version":"0.0.0","private":true{extra},"dependencies":{{"{NPM_NAME}":"{NPM_VERSION}"}}}}"# + ), + ) + .expect("write package.json"); +} + +fn minimist_entry(proj: &Path) -> PathBuf { + proj.join("node_modules").join(NPM_NAME).join("index.js") +} + +// --------------------------------------------------------------------------- +// The mirror check +// --------------------------------------------------------------------------- + +/// What the run does with a real Artifactory's answers before it pins +/// anything: a repository it can read is pinned without a warning, one that +/// wants credentials the run does not have is pinned with a warning, and a +/// repository key that does not exist gets nothing pinned to it. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and the npm registry. Run with --ignored."] +fn the_mirror_is_checked_before_anything_is_pinned() { + const LEG: &str = "the_mirror_is_checked_before_anything_is_pinned"; + let Some(private) = Private::from_env() else { + soft_skip!( + LEG, + "SOCKET_PATCH_ARTIFACTORY_E2E_BASE / _USER / _PASSWORD are not set" + ); + }; + if !has_command("npm") { + soft_skip!(LEG, "`npm` is not on PATH"); + } + let fx = fixture(); + write_npm_manifest(&fx.proj, ""); + let cache = fx.scratch.join("npm-cache").display().to_string(); + let lock_only = tool( + &fx.proj, + "npm", + &[ + "install", + "--package-lock-only", + "--no-audit", + "--no-fund", + "--ignore-scripts", + ], + &[("npm_config_cache", &cache)], + ); + if !lock_only.status.success() { + soft_skip!( + LEG, + "the upstream lock could not be made:\n{}", + dump(&lock_only) + ); + } + let lock_path = fx.proj.join("package-lock.json"); + let pristine = read(&lock_path); + let reset = || { + std::fs::write(&lock_path, &pristine).unwrap(); + let _ = std::fs::remove_file(fx.proj.join(".npmrc")); + }; + + // A repository key that does not exist: nothing is written. + let (root, _) = private.root_and_repo(); + let missing = format!("{root}/socket-patch-e2e-no-such-repository"); + let env = scan_mirrored(&fx.proj, &missing, Some(&private.authorization())); + assert!(pinned(&env).is_empty(), "{LEG}: {env:#}"); + assert!( + warning_codes(&env).contains(&"redirect_mirror_unavailable".to_string()), + "{LEG}: {env:#}" + ); + assert_eq!( + read(&lock_path), + pristine, + "{LEG}: the lock must be untouched" + ); + assert!( + !fx.proj.join(".npmrc").exists(), + "{LEG}: no .npmrc is written" + ); + + // The real repository, checked with the operator's credential. + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert!( + !warning_codes(&env).iter().any(|c| c.contains("mirror")), + "{LEG}: a repository the run could read raises no mirror warning: {env:#}" + ); + assert_mirrored_pin(&read(&lock_path), &private.base, LEG); + let checked = read(&lock_path); + + // A re-run finds its own pin and changes nothing. + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert_eq!( + read(&lock_path), + checked, + "{LEG}: a re-run must leave the lock as it is" + ); + + // The same repository with no credential: Artifactory answers 401, so + // the pin is written unchecked and the run says so. Same lock. + reset(); + let env = scan_mirrored(&fx.proj, &private.base, None); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert!( + warning_codes(&env).contains(&"redirect_mirror_unverified".to_string()), + "{LEG}: {env:#}" + ); + assert_eq!( + read(&lock_path), + checked, + "{LEG}: the unchecked pin is the same pin" + ); + + // A run that was not given the mirror leaves the pin alone. + let bare_scan = || { + let mut bare = cli(&fx.proj); + bare.args(["scan", "--mode", "hosted", "--json", "--yes"]); + let out = bare.output().expect("run socket-patch"); + assert!(out.status.success(), "{}", dump(&out)); + serde_json::from_slice::(&out.stdout).expect("scan emits JSON") + }; + let env = bare_scan(); + assert!( + warning_codes(&env).contains(&"redirect_pin_on_unconfigured_host".to_string()), + "{LEG}: {env:#}" + ); + assert_eq!( + read(&lock_path), + checked, + "{LEG}: the mirrored pin is untouched" + ); + + // A project pinned straight at the patch server moves onto the mirror + // when a run is given one. + reset(); + let env = bare_scan(); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert!( + read(&lock_path).contains(PATCH_HOST), + "{LEG}: a run without a mirror pins the patch server" + ); + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert_eq!( + read(&lock_path), + checked, + "{LEG}: the direct pin must move onto the mirror" + ); +} + +// --------------------------------------------------------------------------- +// npm family +// --------------------------------------------------------------------------- + +/// npm through a repository that needs credentials: the install fails +/// without them, succeeds with the host's credentials in `.npmrc`, and the +/// artifact ends up in Artifactory's cache. `list`, `vex` and `rollback` +/// then read the mirrored pin. +#[tokio::test] +#[ignore = "live: a real Artifactory, the production patch API and the npm registry. Run with --ignored."] +async fn npm_installs_the_patch_through_artifactory() { + const LEG: &str = "npm_installs_the_patch_through_artifactory"; + let Some(private) = Private::from_env() else { + soft_skip!( + LEG, + "SOCKET_PATCH_ARTIFACTORY_E2E_BASE / _USER / _PASSWORD are not set" + ); + }; + if !has_command("npm") { + soft_skip!(LEG, "`npm` is not on PATH"); + } + let fx = fixture(); + write_npm_manifest(&fx.proj, ""); + let cache = |name: &str| fx.scratch.join(name).display().to_string(); + let install = tool( + &fx.proj, + "npm", + &["install", "--no-audit", "--no-fund", "--ignore-scripts"], + &[("npm_config_cache", &cache("upstream-cache"))], + ); + if !install.status.success() { + soft_skip!(LEG, "upstream `npm install` failed:\n{}", dump(&install)); + } + assert_pristine(&minimist_entry(&fx.proj), LEG); + let upstream_lock = read(&fx.proj.join("package-lock.json")); + + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!( + pinned(&env), + [format!("pkg:npm/{NPM_NAME}@{NPM_VERSION}")], + "{LEG}: {env:#}" + ); + let lock = read(&fx.proj.join("package-lock.json")); + assert_mirrored_pin(&lock, &private.base, LEG); + + // Without credentials Artifactory refuses the artifact. + std::fs::remove_dir_all(fx.proj.join("node_modules")).unwrap(); + let ci = ["ci", "--no-audit", "--no-fund", "--ignore-scripts"]; + let refused = tool( + &fx.proj, + "npm", + &ci, + &[("npm_config_cache", &cache("cold-a"))], + ); + assert!( + !refused.status.success() && dump(&refused).contains("E401"), + "{LEG}: an install with no credentials must be refused by Artifactory:\n{}", + dump(&refused) + ); + + // With the host's credentials in the user's npm config it installs the + // patched bytes, from a cold cache. + let userconfig = fx.scratch.join("user.npmrc"); + std::fs::write(&userconfig, private.npmrc_auth()).unwrap(); + let userconfig = userconfig.display().to_string(); + let installed = tool( + &fx.proj, + "npm", + &ci, + &[ + ("npm_config_cache", &cache("cold-b")), + ("npm_config_userconfig", &userconfig), + ], + ); + assert!( + installed.status.success(), + "{LEG}: `npm ci` through Artifactory failed:\n{}", + dump(&installed) + ); + assert_patched(&minimist_entry(&fx.proj), LEG); + + // Artifactory now holds the artifact in the repository's cache. + let mirrored = lock + .split('"') + .find(|s| s.starts_with(&format!("{}/patch/npm/", private.base))) + .expect("the mirrored URL is in the lock") + .to_string(); + let (root, repo) = private.root_and_repo(); + let path = mirrored.strip_prefix(&private.base).unwrap(); + let storage = format!("{root}/api/storage/{repo}-cache{path}"); + let info = reqwest::Client::new() + .get(&storage) + .basic_auth(&private.user, Some(&private.password)) + .send() + .await + .expect("ask Artifactory for the cached artifact"); + assert!( + info.status().is_success(), + "{LEG}: Artifactory does not hold the artifact in {repo}-cache (HTTP {})", + info.status() + ); + let info: serde_json::Value = info.json().await.expect("storage info is JSON"); + assert_eq!(info["repo"], format!("{repo}-cache"), "{LEG}: {info:#}"); + + // `list` and a verifying `vex` read the pin when given the mirror. + let mut list = cli(&fx.proj); + list.args(["list", "--patch-mirror-url", &private.base]); + let out = list.output().unwrap(); + assert!( + String::from_utf8_lossy(&out.stdout).contains("Mode: hosted"), + "{LEG}: {}", + dump(&out) + ); + let mut vex = cli(&fx.proj); + vex.args([ + "vex", + "--product", + "pkg:npm/artifactory-e2e@0.0.0", + "--patch-mirror-url", + &private.base, + ]); + let out = vex.output().unwrap(); + let doc = String::from_utf8_lossy(&out.stdout); + assert!( + out.status.success() && doc.contains("not_affected") && doc.contains("(redirected)"), + "{LEG}: vex must attest the installed, mirrored patch:\n{}", + dump(&out) + ); + + // `rollback` restores what npm wrote, re-resolved from the registry. + let mut rollback = cli(&fx.proj); + rollback.args([ + "rollback", + "--json", + "--yes", + "--patch-mirror-url", + &private.base, + ]); + let out = rollback.output().unwrap(); + assert!( + out.status.success(), + "{LEG}: rollback failed:\n{}", + dump(&out) + ); + assert_eq!( + read(&fx.proj.join("package-lock.json")), + upstream_lock, + "{LEG}: rollback must restore npm's own lock" + ); + assert!( + !fx.proj.join(".npmrc").exists(), + "{LEG}: rollback removes its .npmrc" + ); +} + +/// pnpm sends a token to a URL pin only when the token's scope covers the +/// pin's path, so the host's credentials go in `.npmrc` under the host. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and the npm registry. Run with --ignored."] +fn pnpm_installs_the_patch_through_artifactory() { + const LEG: &str = "pnpm_installs_the_patch_through_artifactory"; + let Some(private) = Private::from_env() else { + soft_skip!( + LEG, + "SOCKET_PATCH_ARTIFACTORY_E2E_BASE / _USER / _PASSWORD are not set" + ); + }; + if !has_command("pnpm") { + soft_skip!(LEG, "`pnpm` is not on PATH"); + } + let fx = fixture(); + write_npm_manifest(&fx.proj, ""); + let store = |name: &str| fx.scratch.join(name).display().to_string(); + let install = tool( + &fx.proj, + "pnpm", + &[ + "install", + "--ignore-scripts", + "--store-dir", + &store("upstream-store"), + ], + &[], + ); + if !install.status.success() { + soft_skip!(LEG, "upstream `pnpm install` failed:\n{}", dump(&install)); + } + assert_pristine(&minimist_entry(&fx.proj), LEG); + + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert_mirrored_pin(&read(&fx.proj.join("pnpm-lock.yaml")), &private.base, LEG); + + std::fs::remove_dir_all(fx.proj.join("node_modules")).unwrap(); + let frozen = [ + "install", + "--frozen-lockfile", + "--ignore-scripts", + "--store-dir", + ]; + let cold_a = store("cold-a"); + let refused = tool( + &fx.proj, + "pnpm", + &[&frozen[..], &[cold_a.as_str()]].concat(), + &[], + ); + assert!( + !refused.status.success(), + "{LEG}: an install with no credentials must be refused:\n{}", + dump(&refused) + ); + + let npmrc = fx.proj.join(".npmrc"); + let existing = std::fs::read_to_string(&npmrc).unwrap_or_default(); + std::fs::write(&npmrc, format!("{existing}{}", private.npmrc_auth())).unwrap(); + let _ = std::fs::remove_dir_all(fx.proj.join("node_modules")); + let cold_b = store("cold-b"); + let installed = tool( + &fx.proj, + "pnpm", + &[&frozen[..], &[cold_b.as_str()]].concat(), + &[], + ); + assert!( + installed.status.success(), + "{LEG}: `pnpm install --frozen-lockfile` through Artifactory failed:\n{}", + dump(&installed) + ); + assert_patched(&minimist_entry(&fx.proj), LEG); +} + +/// yarn classic sends no credentials for a URL pin outside its registry's +/// path, so it reads a repository that allows anonymous access. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and the npm registry. Run with --ignored."] +fn yarn_classic_installs_the_patch_through_an_anonymous_repository() { + const LEG: &str = "yarn_classic_installs_the_patch_through_an_anonymous_repository"; + let Some(base) = setting("SOCKET_PATCH_ARTIFACTORY_E2E_ANON_BASE") else { + soft_skip!(LEG, "SOCKET_PATCH_ARTIFACTORY_E2E_ANON_BASE is not set"); + }; + let version = tool(Path::new("."), "yarn", &["--version"], &[]); + if !version.status.success() || !String::from_utf8_lossy(&version.stdout).starts_with("1.") { + soft_skip!(LEG, "yarn 1.x is not on PATH"); + } + let fx = fixture(); + write_npm_manifest(&fx.proj, ""); + let cache = |name: &str| fx.scratch.join(name).display().to_string(); + let install = tool( + &fx.proj, + "yarn", + &[ + "install", + "--ignore-scripts", + "--cache-folder", + &cache("upstream-cache"), + ], + &[], + ); + if !install.status.success() { + soft_skip!(LEG, "upstream `yarn install` failed:\n{}", dump(&install)); + } + assert_pristine(&minimist_entry(&fx.proj), LEG); + + let env = scan_mirrored(&fx.proj, &base, None); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert!( + !warning_codes(&env).iter().any(|c| c.contains("mirror")), + "{LEG}: an anonymous repository is checked without credentials: {env:#}" + ); + assert_mirrored_pin(&read(&fx.proj.join("yarn.lock")), &base, LEG); + + std::fs::remove_dir_all(fx.proj.join("node_modules")).unwrap(); + let installed = tool( + &fx.proj, + "yarn", + &[ + "install", + "--frozen-lockfile", + "--ignore-scripts", + "--cache-folder", + &cache("cold"), + ], + &[], + ); + assert!( + installed.status.success(), + "{LEG}: `yarn install --frozen-lockfile` through Artifactory failed:\n{}", + dump(&installed) + ); + assert_patched(&minimist_entry(&fx.proj), LEG); +} + +/// yarn berry fetches a URL pin with no credentials at all, so it too +/// reads a repository that allows anonymous access. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and the npm registry. Run with --ignored."] +fn yarn_berry_installs_the_patch_through_an_anonymous_repository() { + const LEG: &str = "yarn_berry_installs_the_patch_through_an_anonymous_repository"; + let Some(base) = setting("SOCKET_PATCH_ARTIFACTORY_E2E_ANON_BASE") else { + soft_skip!(LEG, "SOCKET_PATCH_ARTIFACTORY_E2E_ANON_BASE is not set"); + }; + if !has_command("corepack") { + soft_skip!(LEG, "`corepack` is not on PATH"); + } + let fx = fixture(); + write_npm_manifest(&fx.proj, r#","packageManager":"yarn@4.9.2""#); + // A plain-http repository needs yarn's own opt-in; an https one does not. + let host = base + .split_once("://") + .map(|(_, rest)| rest.split(['/', ':']).next().unwrap_or(rest).to_string()) + .unwrap_or_default(); + std::fs::write( + fx.proj.join(".yarnrc.yml"), + format!( + "nodeLinker: node-modules\nenableGlobalCache: false\nenableTelemetry: false\n\ + unsafeHttpWhitelist: [\"{host}\"]\n" + ), + ) + .unwrap(); + let env_vars = [("COREPACK_ENABLE_DOWNLOAD_PROMPT", "0")]; + // This install writes the lock. Under `CI` yarn installs immutably by + // default and would refuse to create it. + let install = tool( + &fx.proj, + "corepack", + &["yarn", "install"], + &[env_vars[0], ("YARN_ENABLE_IMMUTABLE_INSTALLS", "false")], + ); + if !install.status.success() { + soft_skip!(LEG, "upstream `yarn install` failed:\n{}", dump(&install)); + } + assert_pristine(&minimist_entry(&fx.proj), LEG); + + let env = scan_mirrored(&fx.proj, &base, None); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert_mirrored_pin(&read(&fx.proj.join("yarn.lock")), &base, LEG); + + std::fs::remove_dir_all(fx.proj.join("node_modules")).unwrap(); + let _ = std::fs::remove_dir_all(fx.proj.join(".yarn").join("cache")); + let installed = tool( + &fx.proj, + "corepack", + &["yarn", "install", "--immutable"], + &env_vars, + ); + assert!( + installed.status.success(), + "{LEG}: `yarn install --immutable` through Artifactory failed:\n{}", + dump(&installed) + ); + assert_patched(&minimist_entry(&fx.proj), LEG); +} + +// --------------------------------------------------------------------------- +// PyPI +// --------------------------------------------------------------------------- + +/// A file of the installed `urllib3` that carries the patch. +fn urllib3_patched(venv: &Path) -> bool { + let Ok(libs) = std::fs::read_dir(venv.join("lib")) else { + return false; + }; + libs.flatten().any(|lib| { + let package = lib.path().join("site-packages").join(PYPI_NAME); + walk(&package) + .iter() + .any(|f| std::fs::read_to_string(f).is_ok_and(|body| body.contains(PATCH_MARKER))) + }) +} + +fn walk(dir: &Path) -> Vec { + let mut out = Vec::new(); + for entry in std::fs::read_dir(dir).into_iter().flatten().flatten() { + let path = entry.path(); + if path.is_dir() { + out.extend(walk(&path)); + } else { + out.push(path); + } + } + out +} + +/// pip installs a direct URL reference with the credentials a netrc file +/// holds for the repository's host. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and PyPI. Run with --ignored."] +fn pip_installs_the_patch_through_artifactory() { + const LEG: &str = "pip_installs_the_patch_through_artifactory"; + let Some(private) = Private::from_env() else { + soft_skip!( + LEG, + "SOCKET_PATCH_ARTIFACTORY_E2E_BASE / _USER / _PASSWORD are not set" + ); + }; + if !has_command("python3") { + soft_skip!(LEG, "`python3` is not on PATH"); + } + let fx = fixture(); + std::fs::write( + fx.proj.join("requirements.txt"), + format!("{PYPI_NAME}=={PYPI_VERSION}\n"), + ) + .unwrap(); + let venv = fx.proj.join(".venv"); + let made = tool(&fx.proj, "python3", &["-m", "venv", ".venv"], &[]); + if !made.status.success() { + soft_skip!(LEG, "`python3 -m venv` failed:\n{}", dump(&made)); + } + let python = venv.join("bin").join("python").display().to_string(); + let pip = |extra_env: &[(&str, &str)]| { + tool( + &fx.proj, + &python, + &[ + "-m", + "pip", + "install", + "--no-input", + "--no-cache-dir", + "--disable-pip-version-check", + "--force-reinstall", + "--no-deps", + "-r", + "requirements.txt", + ], + extra_env, + ) + }; + let upstream = pip(&[]); + if !upstream.status.success() { + soft_skip!(LEG, "upstream `pip install` failed:\n{}", dump(&upstream)); + } + assert!( + !urllib3_patched(&venv), + "{LEG}: the upstream install is already patched" + ); + + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!( + pinned(&env), + [format!("pkg:pypi/{PYPI_NAME}@{PYPI_VERSION}")], + "{LEG}: {env:#}" + ); + assert_mirrored_pin(&read(&fx.proj.join("requirements.txt")), &private.base, LEG); + + let refused = pip(&[]); + assert!( + !refused.status.success(), + "{LEG}: an install with no credentials must be refused:\n{}", + dump(&refused) + ); + let netrc = private.netrc(&fx.scratch).display().to_string(); + let installed = pip(&[("NETRC", &netrc)]); + assert!( + installed.status.success(), + "{LEG}: `pip install` through Artifactory failed:\n{}", + dump(&installed) + ); + assert!( + urllib3_patched(&venv), + "{LEG}: the installed urllib3 is not patched" + ); +} + +/// uv installs a `uv.lock` URL source with the credentials a netrc file +/// holds for the repository's host. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and PyPI. Run with --ignored."] +fn uv_installs_the_patch_through_artifactory() { + const LEG: &str = "uv_installs_the_patch_through_artifactory"; + let Some(private) = Private::from_env() else { + soft_skip!( + LEG, + "SOCKET_PATCH_ARTIFACTORY_E2E_BASE / _USER / _PASSWORD are not set" + ); + }; + if !has_command("uv") { + soft_skip!(LEG, "`uv` is not on PATH"); + } + let fx = fixture(); + std::fs::write( + fx.proj.join("pyproject.toml"), + format!( + "[project]\nname = \"artifactory-e2e\"\nversion = \"0.0.0\"\n\ + requires-python = \">=3.9\"\ndependencies = [\"{PYPI_NAME}=={PYPI_VERSION}\"]\n" + ), + ) + .unwrap(); + let cache = |name: &str| fx.scratch.join(name).display().to_string(); + let locked = tool( + &fx.proj, + "uv", + &["lock"], + &[("UV_CACHE_DIR", &cache("upstream"))], + ); + if !locked.status.success() { + soft_skip!(LEG, "`uv lock` failed:\n{}", dump(&locked)); + } + + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert_eq!(pinned(&env).len(), 1, "{LEG}: {env:#}"); + assert_mirrored_pin(&read(&fx.proj.join("uv.lock")), &private.base, LEG); + + let netrc = private.netrc(&fx.scratch).display().to_string(); + let synced = tool( + &fx.proj, + "uv", + &["sync", "--frozen"], + &[("UV_CACHE_DIR", &cache("cold")), ("NETRC", &netrc)], + ); + assert!( + synced.status.success(), + "{LEG}: `uv sync --frozen` through Artifactory failed:\n{}", + dump(&synced) + ); + assert!( + urllib3_patched(&fx.proj.join(".venv")), + "{LEG}: the installed urllib3 is not patched" + ); +} + +// --------------------------------------------------------------------------- +// RubyGems +// --------------------------------------------------------------------------- + +/// The sha256 the lock's `CHECKSUMS` section pins for `name (version)`. +fn lock_checksum(lock: &str, name: &str, version: &str) -> Option { + let lead = format!(" {name} ({version}) sha256="); + lock.lines() + .find_map(|line| line.strip_prefix(&lead)) + .map(|sha| sha.trim().to_string()) +} + +fn sha256_hex(bytes: &[u8]) -> String { + use sha2::Digest as _; + hex::encode(sha2::Sha256::digest(bytes)) +} + +/// Bundler reads each patch's compact index and `.gem` through the +/// repository, with the credentials it holds for the repository's host, +/// and the gems it fetches are the ones the lock's checksums pin. +#[test] +#[ignore = "live: a real Artifactory, the production patch API and rubygems.org. Run with --ignored."] +fn bundler_fetches_the_patched_gems_through_artifactory() { + const LEG: &str = "bundler_fetches_the_patched_gems_through_artifactory"; + let Some(private) = Private::from_env() else { + soft_skip!( + LEG, + "SOCKET_PATCH_ARTIFACTORY_E2E_BASE / _USER / _PASSWORD are not set" + ); + }; + if !has_command("bundle") { + soft_skip!(LEG, "`bundle` is not on PATH"); + } + let fx = fixture(); + std::fs::write( + fx.proj.join("Gemfile"), + format!("source \"https://rubygems.org\"\n\ngem \"{GEM_NAME}\", \"{GEM_VERSION}\"\n"), + ) + .unwrap(); + let app_config = fx.proj.join(".bundle").display().to_string(); + let home = |name: &str| fx.scratch.join(name).display().to_string(); + let bundle = |args: &[&str], user_home: &str, extra: &[(&str, &str)]| { + let mut env = vec![ + ("BUNDLE_APP_CONFIG", app_config.as_str()), + ("BUNDLE_USER_HOME", user_home), + ("GEM_HOME", user_home), + // Lockfile checksums are opt-in before Bundler 4. + ("BUNDLE_LOCKFILE_CHECKSUMS", "true"), + ]; + env.extend_from_slice(extra); + tool(&fx.proj, "bundle", args, &env) + }; + let locked = bundle(&["lock"], &home("upstream"), &[]); + if !locked.status.success() { + soft_skip!(LEG, "`bundle lock` failed:\n{}", dump(&locked)); + } + let upstream_checksum = + lock_checksum(&read(&fx.proj.join("Gemfile.lock")), GEM_NAME, GEM_VERSION); + if upstream_checksum.is_none() { + soft_skip!( + LEG, + "this Bundler writes no lockfile checksums (it needs 2.6 or later)" + ); + } + + // A project already pinned straight at the patch server: Bundler's pin + // cannot be moved onto the mirror in place, so the run leaves it, says + // so, and the remedy it names (roll back, run again) works. + let mut direct = cli(&fx.proj); + direct.args(["scan", "--mode", "hosted", "--json", "--yes"]); + let out = direct.output().expect("run socket-patch"); + assert!(out.status.success(), "{LEG}: {}", dump(&out)); + let direct_lock = read(&fx.proj.join("Gemfile.lock")); + assert!( + direct_lock.contains(PATCH_HOST), + "{LEG}: a run without a mirror pins the patch server:\n{direct_lock}" + ); + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert!( + warning_codes(&env).contains(&"redirect_mirror_pin_not_moved".to_string()), + "{LEG}: {env:#}" + ); + assert_eq!( + read(&fx.proj.join("Gemfile.lock")), + direct_lock, + "{LEG}: the direct pin is left as it is" + ); + let mut rollback = cli(&fx.proj); + rollback.args(["rollback", "--json", "--yes"]); + let out = rollback.output().expect("run socket-patch"); + assert!( + out.status.success(), + "{LEG}: rollback failed:\n{}", + dump(&out) + ); + assert!( + !read(&fx.proj.join("Gemfile.lock")).contains(PATCH_HOST), + "{LEG}: rollback must unwind the direct pins" + ); + + let env = scan_mirrored(&fx.proj, &private.base, Some(&private.authorization())); + assert!( + pinned(&env).contains(&format!("pkg:gem/{GEM_NAME}@{GEM_VERSION}")), + "{LEG}: {env:#}" + ); + let lock = read(&fx.proj.join("Gemfile.lock")); + assert!( + lock.contains(&format!("remote: {}/patch-registry/gem/", private.base)), + "{LEG}: the lock does not resolve the gem from the Artifactory repository:\n{lock}" + ); + assert!( + !lock.contains(PATCH_HOST), + "{LEG}: the lock still names {PATCH_HOST}:\n{lock}" + ); + assert!( + read(&fx.proj.join("Gemfile")).contains(&format!("{}/patch-registry/gem/", private.base)), + "{LEG}: the Gemfile does not name the Artifactory repository" + ); + let patched_checksum = lock_checksum(&lock, GEM_NAME, GEM_VERSION).expect("checksum pinned"); + assert_ne!( + Some(&patched_checksum), + upstream_checksum.as_ref(), + "{LEG}: the lock must pin the patched gem's checksum" + ); + + let fetch = ["cache", "--no-install"]; + let refused = bundle(&fetch, &home("cold-a"), &[("BUNDLE_FROZEN", "true")]); + assert!( + !refused.status.success(), + "{LEG}: a fetch with no credentials must be refused:\n{}", + dump(&refused) + ); + + // Bundler keys credentials by host name. + let credentials = format!("{}:{}", private.user, private.password); + let configured = bundle( + &[ + "config", + "set", + "--local", + &private.origin().2, + &credentials, + ], + &home("cold-b"), + &[], + ); + assert!(configured.status.success(), "{LEG}: {}", dump(&configured)); + let _ = std::fs::remove_dir_all(fx.proj.join("vendor")); + let fetched = bundle(&fetch, &home("cold-b"), &[("BUNDLE_FROZEN", "true")]); + assert!( + fetched.status.success(), + "{LEG}: `bundle cache` through Artifactory failed:\n{}", + dump(&fetched) + ); + let gem = fx + .proj + .join("vendor") + .join("cache") + .join(format!("{GEM_NAME}-{GEM_VERSION}.gem")); + let bytes = + std::fs::read(&gem).unwrap_or_else(|e| panic!("{LEG}: read {}: {e}", gem.display())); + assert_eq!( + sha256_hex(&bytes), + patched_checksum, + "{LEG}: the gem Bundler fetched through Artifactory is not the patched gem the lock pins" + ); +} diff --git a/crates/socket-patch-cli/tests/get/get_fixture.rs b/crates/socket-patch-cli/tests/get/get_fixture.rs index 113cb476b..a4563372c 100644 --- a/crates/socket-patch-cli/tests/get/get_fixture.rs +++ b/crates/socket-patch-cli/tests/get/get_fixture.rs @@ -298,6 +298,7 @@ pub fn engine_params(root: &Path) -> DownloadParams { ecosystems: None, persist_blobs: true, patch_server_url: None, + patch_mirror_url: None, all_releases: true, } } diff --git a/crates/socket-patch-cli/tests/in_process_agent_patch_replace.rs b/crates/socket-patch-cli/tests/in_process_agent_patch_replace.rs index 855f69920..b4b77827d 100644 --- a/crates/socket-patch-cli/tests/in_process_agent_patch_replace.rs +++ b/crates/socket-patch-cli/tests/in_process_agent_patch_replace.rs @@ -559,6 +559,7 @@ async fn engine_get( ecosystems: None, persist_blobs: true, patch_server_url: None, + patch_mirror_url: None, all_releases: true, }; let selected: Vec = selected diff --git a/crates/socket-patch-cli/tests/in_process_get_update_count.rs b/crates/socket-patch-cli/tests/in_process_get_update_count.rs index 5e125d80c..d5dc3850e 100644 --- a/crates/socket-patch-cli/tests/in_process_get_update_count.rs +++ b/crates/socket-patch-cli/tests/in_process_get_update_count.rs @@ -96,6 +96,7 @@ fn params(root: &Path) -> DownloadParams { ecosystems: None, persist_blobs: true, patch_server_url: None, + patch_mirror_url: None, // Skip release-narrowing; npm has no variants anyway. all_releases: true, } diff --git a/crates/socket-patch-cli/tests/patch_mirror.rs b/crates/socket-patch-cli/tests/patch_mirror.rs new file mode 100644 index 000000000..5d80035d2 --- /dev/null +++ b/crates/socket-patch-cli/tests/patch_mirror.rs @@ -0,0 +1,775 @@ +//! `--patch-mirror-url`: the parts of hosted mode through a patch-server +//! mirror that a live mirror cannot show. +//! +//! The proof that a mirrored project installs is +//! `e2e_artifactory_mirror.rs`, which drives a real Artifactory, the +//! production patch API and the real package managers. This file holds +//! only what needs both ends of a conversation observed, or an answer a +//! healthy mirror never gives: +//! +//! * the run's own downloads go to the patch server, never to the mirror; +//! * the operator's mirror credential reaches the mirror and nothing else; +//! * what the run does with each answer to its check of the mirror (a +//! redirect, a miss for one patch, a miss for every patch, no answer); +//! * a base that cannot prefix a committed URL is refused before the +//! project is read; +//! * the everyday shape (pin the mirror, re-run, a run without the mirror +//! leaves the pin alone), so the merge queue sees a regression without +//! an Artifactory. +//! +//! Two wiremock servers stand in: one is the Socket API and the patch +//! server (the grants name its own origin), the other the mirror. + +#[path = "common/envelope.rs"] +mod envelope; +#[path = "common/hermetic.rs"] +mod hermetic; + +use std::path::Path; + +use envelope::warning_codes; + +use serial_test::serial; +use wiremock::matchers::{header, method, path, path_regex}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +const ORG: &str = "test-org"; +const VERSION: &str = "1.0.0"; +const TOKEN: &str = "22222222-2222-4222-8222-222222222222"; +/// Where the mock mirror sits on its server: an Artifactory repository path. +const MIRROR_PREFIX: &str = "/artifactory/socket-patch"; + +/// One patched package of a fixture project. +#[derive(Clone, Copy)] +struct Pkg { + name: &'static str, + uuid: &'static str, +} + +const FIRST: Pkg = Pkg { + name: "mirror-fixture", + uuid: "11111111-1111-4111-8111-111111111111", +}; +const SECOND: Pkg = Pkg { + name: "other-fixture", + uuid: "33333333-3333-4333-8333-333333333333", +}; + +impl Pkg { + fn purl(&self) -> String { + format!("pkg:npm/{}@{VERSION}", self.name) + } + + /// The path the patch server serves the patched tarball at, and so the + /// path under the mirror's base. + fn served_path(&self) -> String { + format!( + "/patch/npm/{0}/{VERSION}/{TOKEN}/{1}/{0}-{VERSION}.tgz", + self.name, self.uuid + ) + } + + /// The patched tarball: a `package.json` and a marked `index.js`. + fn tarball(&self) -> Vec { + let mut builder = tar::Builder::new(flate2::write::GzEncoder::new( + Vec::new(), + flate2::Compression::default(), + )); + for (name, body) in [ + ( + "package/package.json", + format!(r#"{{"name":"{}","version":"{VERSION}"}}"#, self.name), + ), + ( + "package/index.js", + "/* patched */\nmodule.exports = 1;\n".to_string(), + ), + ] { + let mut header = tar::Header::new_gnu(); + header.set_size(body.len() as u64); + header.set_mode(0o644); + header.set_cksum(); + builder + .append_data(&mut header, name, body.as_bytes()) + .unwrap(); + } + builder.into_inner().unwrap().finish().unwrap() + } + + fn sha512(&self) -> String { + use base64::Engine as _; + use sha2::Digest as _; + format!( + "sha512-{}", + base64::engine::general_purpose::STANDARD.encode(sha2::Sha512::digest(self.tarball())) + ) + } +} + +fn mirror_base(mirror: &MockServer) -> String { + format!("{}{MIRROR_PREFIX}", mirror.uri()) +} + +fn mirrored_url(mirror: &MockServer, pkg: Pkg) -> String { + format!("{}{}", mirror_base(mirror), pkg.served_path()) +} + +/// The Socket API and the patch server on one origin: discovery, the +/// reference grants (which name this server), and each patched tarball at +/// its served path. +async fn mock_socket(server: &MockServer, pkgs: &[Pkg]) { + let packages: Vec = pkgs + .iter() + .map(|pkg| { + serde_json::json!({ + "purl": pkg.purl(), + "patches": [{ + "uuid": pkg.uuid, "purl": pkg.purl(), "tier": "free", + "cveIds": [], "ghsaIds": [], "severity": "high", + "title": "mirror fixture" + }] + }) + }) + .collect(); + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/batch"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "packages": packages, + "canAccessPaidPatches": false, + }))) + .mount(server) + .await; + let mut results = serde_json::Map::new(); + for pkg in pkgs { + let served = format!("{}{}", server.uri(), pkg.served_path()); + Mock::given(method("GET")) + .and(path_regex(format!( + "^/v0/orgs/{ORG}/patches/by-package/.*{}.*$", + pkg.name + ))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "patches": [{ + "uuid": pkg.uuid, "purl": pkg.purl(), + "publishedAt": "2024-01-01T00:00:00Z", + "description": "x", "license": "MIT", "tier": "free", + "vulnerabilities": {} + }], + "canAccessPaidPatches": false, + }))) + .mount(server) + .await; + Mock::given(method("GET")) + .and(path(pkg.served_path())) + .respond_with( + ResponseTemplate::new(200).set_body_raw(pkg.tarball(), "application/octet-stream"), + ) + .mount(server) + .await; + results.insert( + pkg.uuid.to_string(), + serde_json::json!({ + "status": "granted", + "url": served, + "purl": pkg.purl(), + "artifacts": [{ + "kind": "tarball", + "url": served, + "integrity": { "sha512": pkg.sha512() } + }], + "registryOverride": null + }), + ); + } + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/package"))) + .respond_with( + ResponseTemplate::new(200).set_body_json(serde_json::json!({ "results": results })), + ) + .mount(server) + .await; +} + +/// The mirror answering the run's check of `pkg` with `status`. +async fn mirror_answers(mirror: &MockServer, pkg: Pkg, status: u16) { + Mock::given(method("HEAD")) + .and(path(format!("{MIRROR_PREFIX}{}", pkg.served_path()))) + .respond_with(ResponseTemplate::new(status)) + .mount(mirror) + .await; +} + +/// An npm project that locks `pkgs` from the registry. +fn write_npm_project(root: &Path, pkgs: &[Pkg]) { + let deps: Vec = pkgs + .iter() + .map(|pkg| format!(r#""{}": "{VERSION}""#, pkg.name)) + .collect(); + let deps = deps.join(", "); + std::fs::write( + root.join("package.json"), + format!(r#"{{ "name": "consumer", "version": "0.0.0", "dependencies": {{ {deps} }} }}"#), + ) + .unwrap(); + let entries: Vec = pkgs + .iter() + .map(|pkg| { + format!( + r#" "node_modules/{0}": {{ + "version": "{VERSION}", + "resolved": "https://registry.npmjs.org/{0}/-/{0}-{VERSION}.tgz", + "integrity": "sha512-UPSTREAM{0}==" + }}"#, + pkg.name + ) + }) + .collect(); + std::fs::write( + root.join("package-lock.json"), + format!( + r#"{{ + "name": "consumer", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": {{ + "": {{ "name": "consumer", "version": "0.0.0", "dependencies": {{ {deps} }} }}, +{} + }} +}} +"#, + entries.join(",\n") + ), + ) + .unwrap(); +} + +fn lock(root: &Path) -> String { + std::fs::read_to_string(root.join("package-lock.json")).unwrap() +} + +/// The binary, hermetic, with no ambient mirror settings. +fn cli() -> std::process::Command { + let mut cmd = hermetic::binary_command(); + cmd.env_remove("SOCKET_PATCH_MIRROR_URL") + .env_remove("SOCKET_PATCH_MIRROR_AUTHORIZATION"); + cmd +} + +struct Run { + code: Option, + stdout: String, + stderr: String, +} + +impl Run { + fn json(&self) -> serde_json::Value { + serde_json::from_str(&self.stdout).unwrap_or_else(|e| { + panic!( + "stdout is not JSON: {e}\nstdout:\n{}\nstderr:\n{}", + self.stdout, self.stderr + ) + }) + } +} + +fn run(mut cmd: std::process::Command) -> Run { + let out = cmd.output().expect("run socket-patch"); + Run { + code: out.status.code(), + stdout: String::from_utf8_lossy(&out.stdout).into_owned(), + stderr: String::from_utf8_lossy(&out.stderr).into_owned(), + } +} + +/// `scan --mode hosted --json` against `api`, with `extra` flags; the run +/// must exit 0. +fn hosted_scan(cwd: &Path, api: &MockServer, extra: &[&str], env: &[(&str, &str)]) -> Run { + let mut cmd = cli(); + cmd.args([ + "scan", + "--mode=hosted", + "--json", + "--yes", + "--cwd", + cwd.to_str().unwrap(), + "--api-url", + &api.uri(), + "--org", + ORG, + "--api-token", + "fake", + ]) + .args(extra); + for (key, value) in env { + cmd.env(key, value); + } + let out = run(cmd); + assert_eq!(out.code, Some(0), "{}\n{}", out.stdout, out.stderr); + out +} + +/// `(action, errorCode)` of the run's event for `pkg`. +fn event(env: &serde_json::Value, pkg: Pkg) -> (String, Option) { + let events = env["events"] + .as_array() + .unwrap_or_else(|| panic!("{env:#}")); + let event = events + .iter() + .find(|e| e["uuid"] == pkg.uuid) + .unwrap_or_else(|| panic!("no event for {}: {env:#}", pkg.name)); + ( + event["action"].as_str().unwrap_or_default().to_string(), + event["errorCode"].as_str().map(str::to_string), + ) +} + +fn applied() -> (String, Option) { + ("applied".to_string(), None) +} + +fn skipped(code: &str) -> (String, Option) { + ("skipped".to_string(), Some(code.to_string())) +} + +/// The requests `server` received, as `METHOD path`. +async fn requests(server: &MockServer) -> Vec { + server + .received_requests() + .await + .unwrap_or_default() + .iter() + .map(|r| format!("{} {}", r.method, r.url.path())) + .collect() +} + +/// The everyday shape: a run given a mirror pins the mirror's URL with the +/// patched integrity after asking the mirror once, a re-run asks nothing +/// and changes nothing, the environment variable is the flag, `list` reads +/// the pin only when given the mirror, and a run that was not given the +/// mirror leaves the pin alone and names the host to configure. +#[tokio::test] +#[serial] +async fn a_run_pins_the_mirror_and_a_run_without_it_leaves_the_pin_alone() { + let api = MockServer::start().await; + mock_socket(&api, &[FIRST]).await; + let mirror = MockServer::start().await; + mirror_answers(&mirror, FIRST, 200).await; + let base = mirror_base(&mirror); + let check = format!("HEAD {MIRROR_PREFIX}{}", FIRST.served_path()); + + let tmp = tempfile::tempdir().unwrap(); + write_npm_project(tmp.path(), &[FIRST]); + + let first = hosted_scan(tmp.path(), &api, &["--patch-mirror-url", &base], &[]); + let env = first.json(); + assert_eq!(event(&env, FIRST), applied(), "{env:#}"); + assert!( + !warning_codes(&env).iter().any(|c| c.contains("mirror")), + "a mirror that serves the patch raises no mirror warning: {env:#}" + ); + let pinned = lock(tmp.path()); + assert!(pinned.contains(&mirrored_url(&mirror, FIRST)), "{pinned}"); + assert!(pinned.contains(&FIRST.sha512()), "{pinned}"); + assert!( + !pinned.contains(&api.uri()) && !pinned.contains("UPSTREAM"), + "neither the patch server's origin nor the upstream entry is left: {pinned}" + ); + assert_eq!( + requests(&mirror).await, + vec![check.clone()], + "the run asks the mirror once, before writing, and downloads nothing from it" + ); + + // A re-run finds its own pin: same lock, and nothing more to ask. + let again = hosted_scan(tmp.path(), &api, &["--patch-mirror-url", &base], &[]); + assert_eq!(event(&again.json(), FIRST), applied(), "{}", again.stdout); + assert_eq!( + lock(tmp.path()), + pinned, + "a re-run leaves the lock as it is" + ); + assert_eq!( + requests(&mirror).await, + vec![check.clone()], + "a pin the project already holds is not asked about again" + ); + + // The environment variable is the flag, on a project of its own. + let fresh = tempfile::tempdir().unwrap(); + write_npm_project(fresh.path(), &[FIRST]); + let by_env = hosted_scan( + fresh.path(), + &api, + &[], + &[("SOCKET_PATCH_MIRROR_URL", &format!("{base}/"))], + ); + assert_eq!(event(&by_env.json(), FIRST), applied(), "{}", by_env.stdout); + assert_eq!(lock(fresh.path()), pinned, "the same pin as the flag wrote"); + + // `list` reads the pin with the mirror, and nothing without it. + let list = |extra: &[&str]| { + let mut cmd = cli(); + cmd.args(["list", "--json", "--cwd", tmp.path().to_str().unwrap()]) + .args(extra); + let out = run(cmd); + assert_eq!(out.code, Some(0), "{}\n{}", out.stdout, out.stderr); + out.stdout + }; + assert!(list(&["--patch-mirror-url", &base]).contains(FIRST.uuid)); + assert!( + !list(&[]).contains(FIRST.uuid), + "a host the run was not given is no patch host" + ); + // The mirror is its base, not its host: another repository of the same + // host is not where the patch server is mirrored. + let sibling = format!("{}/artifactory/team-local", mirror.uri()); + assert_ne!(sibling, base); + assert!( + !list(&["--patch-mirror-url", &sibling]).contains(FIRST.uuid), + "a pin under another base of the mirror's host is no hosted pin" + ); + + // A run without the mirror: nothing moves onto the patch server. + let bare = hosted_scan(tmp.path(), &api, &[], &[]); + let env = bare.json(); + assert_eq!( + event(&env, FIRST), + skipped("redirect_pin_on_unconfigured_host"), + "{env:#}" + ); + let host = mirror.uri().trim_start_matches("http://").to_string(); + assert!( + bare.stdout.contains(&host) && bare.stdout.contains("--patch-mirror-url"), + "the warning names the host and the setting: {}", + bare.stdout + ); + assert_eq!(lock(tmp.path()), pinned, "the mirrored pin is untouched"); +} + +/// What the run needs for itself it downloads from the patch server, not +/// from the mirror, which may want credentials the run does not have. A +/// yarn classic pin reads the served tarball (its sha1 goes into the +/// lock); here the mirror answers the check and nothing else, so a +/// download sent its way would fail the pin. +#[tokio::test] +#[serial] +async fn the_runs_own_downloads_read_the_patch_server_not_the_mirror() { + let api = MockServer::start().await; + mock_socket(&api, &[FIRST]).await; + let mirror = MockServer::start().await; + mirror_answers(&mirror, FIRST, 200).await; + + let tmp = tempfile::tempdir().unwrap(); + let name = FIRST.name; + std::fs::write( + tmp.path().join("package.json"), + format!( + r#"{{ "name": "consumer", "version": "0.0.0", "dependencies": {{ "{name}": "{VERSION}" }} }}"# + ), + ) + .unwrap(); + std::fs::write( + tmp.path().join("yarn.lock"), + format!( + "# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.\n\ + # yarn lockfile v1\n\n\n\ + {name}@{VERSION}:\n version \"{VERSION}\"\n \ + resolved \"https://registry.yarnpkg.com/{name}/-/{name}-{VERSION}.tgz#bbbb\"\n \ + integrity sha512-UPSTREAMupstream==\n" + ), + ) + .unwrap(); + + let out = hosted_scan( + tmp.path(), + &api, + &["--patch-mirror-url", &mirror_base(&mirror)], + &[], + ); + let env = out.json(); + assert_eq!(event(&env, FIRST), applied(), "{env:#}"); + + use sha1::Digest as _; + let sha1 = hex::encode(sha1::Sha1::digest(FIRST.tarball())); + let pinned = std::fs::read_to_string(tmp.path().join("yarn.lock")).unwrap(); + assert!( + pinned.contains(&format!( + "resolved \"{}#{sha1}\"", + mirrored_url(&mirror, FIRST) + )), + "the lock pins the mirror, with the sha1 of the tarball the run read: {pinned}" + ); + assert!( + requests(&api) + .await + .contains(&format!("GET {}", FIRST.served_path())), + "the tarball was read from the patch server" + ); + assert_eq!( + requests(&mirror).await, + vec![format!("HEAD {MIRROR_PREFIX}{}", FIRST.served_path())], + "the mirror was asked the one check and sent no download" + ); +} + +/// The run's check, answer by answer. A redirect is not followed: the pin +/// is written with the "not checked" warning, and the redirect's target +/// never hears from the run. A miss for one patch withholds that patch +/// alone. A miss for every patch, or no answer at all, is said of the +/// mirror, and nothing is pinned. +#[tokio::test] +#[serial] +async fn each_answer_to_the_mirror_check_is_acted_on() { + let api = MockServer::start().await; + mock_socket(&api, &[FIRST, SECOND]).await; + let project = || { + let tmp = tempfile::tempdir().unwrap(); + write_npm_project(tmp.path(), &[FIRST, SECOND]); + tmp + }; + + // A redirect, to a server that would say yes. + let elsewhere = MockServer::start().await; + Mock::given(method("HEAD")) + .respond_with(ResponseTemplate::new(200)) + .mount(&elsewhere) + .await; + let redirecting = MockServer::start().await; + Mock::given(method("HEAD")) + .respond_with( + ResponseTemplate::new(302) + .insert_header("Location", format!("{}/login", elsewhere.uri()).as_str()), + ) + .mount(&redirecting) + .await; + let tmp = project(); + let out = hosted_scan( + tmp.path(), + &api, + &["--patch-mirror-url", &mirror_base(&redirecting)], + &[], + ); + let env = out.json(); + assert_eq!(event(&env, FIRST), applied(), "{env:#}"); + assert_eq!(event(&env, SECOND), applied(), "{env:#}"); + assert_eq!( + warning_codes(&env) + .iter() + .filter(|c| c.as_str() == "redirect_mirror_unverified") + .count(), + 1, + "{env:#}" + ); + assert!( + requests(&elsewhere).await.is_empty(), + "the redirect is not followed" + ); + assert_eq!( + requests(&redirecting).await.len(), + 1, + "one answer that says nothing about the patch ends the check" + ); + + // One patch the mirror serves, one it does not. + let partial = MockServer::start().await; + mirror_answers(&partial, FIRST, 200).await; + mirror_answers(&partial, SECOND, 404).await; + let tmp = project(); + let out = hosted_scan( + tmp.path(), + &api, + &["--patch-mirror-url", &mirror_base(&partial)], + &[], + ); + let env = out.json(); + assert_eq!(event(&env, FIRST), applied(), "{env:#}"); + assert_eq!( + event(&env, SECOND), + skipped("redirect_mirror_unavailable"), + "{env:#}" + ); + let pinned = lock(tmp.path()); + assert!(pinned.contains(&mirrored_url(&partial, FIRST)), "{pinned}"); + assert!( + pinned.contains(&format!("sha512-UPSTREAM{}==", SECOND.name)) + && !pinned.contains(SECOND.uuid), + "the patch the mirror does not serve keeps its registry entry: {pinned}" + ); + assert!( + out.stdout + .contains(&format!("{}: the patch mirror", SECOND.purl())), + "the warning is about that package, not the mirror: {}", + out.stdout + ); + + // Nothing served (HTTP 404 and 500), and nothing listening. + let closed = { + let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + format!( + "http://{}/artifactory/socket-patch", + listener.local_addr().unwrap() + ) + }; + let missing = MockServer::start().await; + let failing = MockServer::start().await; + for pkg in [FIRST, SECOND] { + mirror_answers(&failing, pkg, 500).await; + } + for base in [mirror_base(&missing), mirror_base(&failing), closed] { + let tmp = project(); + let pristine = lock(tmp.path()); + let out = hosted_scan(tmp.path(), &api, &["--patch-mirror-url", &base], &[]); + let env = out.json(); + for pkg in [FIRST, SECOND] { + assert_eq!( + event(&env, pkg), + skipped("redirect_mirror_unavailable"), + "{base}: {env:#}" + ); + } + assert_eq!( + warning_codes(&env) + .iter() + .filter(|c| c.as_str() == "redirect_mirror_unavailable") + .count(), + 1, + "{base}: one warning, about the mirror: {env:#}" + ); + assert_eq!(lock(tmp.path()), pristine, "{base}: nothing is written"); + assert!( + !tmp.path().join(".npmrc").exists(), + "{base}: no install setting is written for a pin that was not" + ); + assert!( + !out.stdout.contains(TOKEN) && !out.stderr.contains(TOKEN), + "the grant token is not printed: {}\n{}", + out.stdout, + out.stderr + ); + } +} + +/// A mirror that wants credentials cannot be checked by a run that has +/// none: the pin is written, with a warning. Given the operator's header +/// value, the run checks the mirror with it, and sends it nowhere else: +/// not to the Socket API, not into a file, not into its own output. +#[tokio::test] +#[serial] +async fn the_mirror_credential_reaches_the_mirror_and_nothing_else() { + const SECRET: &str = "Bearer mirror-s3cret"; + let api = MockServer::start().await; + mock_socket(&api, &[FIRST]).await; + let mirror = MockServer::start().await; + let at = format!("{MIRROR_PREFIX}{}", FIRST.served_path()); + Mock::given(method("HEAD")) + .and(path(at.clone())) + .and(header("authorization", SECRET)) + .respond_with(ResponseTemplate::new(200)) + .mount(&mirror) + .await; + Mock::given(method("HEAD")) + .and(path(at)) + .respond_with( + ResponseTemplate::new(401) + .insert_header("WWW-Authenticate", "Basic realm=\"Artifactory Realm\""), + ) + .mount(&mirror) + .await; + let base = mirror_base(&mirror); + + // No credential: pinned, with the warning. + let tmp = tempfile::tempdir().unwrap(); + write_npm_project(tmp.path(), &[FIRST]); + let out = hosted_scan(tmp.path(), &api, &["--patch-mirror-url", &base], &[]); + let env = out.json(); + assert_eq!(event(&env, FIRST), applied(), "{env:#}"); + assert!( + warning_codes(&env).contains(&"redirect_mirror_unverified".to_string()), + "{env:#}" + ); + assert!(lock(tmp.path()).contains(&mirrored_url(&mirror, FIRST))); + + // With the operator's credential: checked, no warning. + let tmp = tempfile::tempdir().unwrap(); + write_npm_project(tmp.path(), &[FIRST]); + let out = hosted_scan( + tmp.path(), + &api, + &["--patch-mirror-url", &base], + &[("SOCKET_PATCH_MIRROR_AUTHORIZATION", SECRET)], + ); + let env = out.json(); + assert_eq!(event(&env, FIRST), applied(), "{env:#}"); + assert!( + !warning_codes(&env).iter().any(|c| c.contains("mirror")), + "{env:#}" + ); + assert!(lock(tmp.path()).contains(&mirrored_url(&mirror, FIRST))); + + let api_requests = api.received_requests().await.unwrap_or_default(); + assert!(!api_requests.is_empty()); + for request in &api_requests { + for (name, value) in request.headers.iter() { + assert!( + !value.to_str().unwrap_or_default().contains("mirror-s3cret"), + "the Socket API received the mirror credential in `{name}`" + ); + } + } + assert!(!out.stdout.contains("mirror-s3cret") && !out.stderr.contains("mirror-s3cret")); + for file in ["package-lock.json", ".npmrc"] { + let text = std::fs::read_to_string(tmp.path().join(file)).unwrap_or_default(); + assert!(!text.contains("mirror-s3cret"), "{file}: {text}"); + } +} + +/// A base that cannot prefix a committed URL is refused before the project +/// is read, and the refusal never repeats the value: a base given with +/// credentials in it must not be echoed into a terminal or a CI log. +#[test] +fn a_base_that_cannot_be_committed_is_refused_without_echoing_it() { + let tmp = tempfile::tempdir().unwrap(); + write_npm_project(tmp.path(), &[FIRST]); + let pristine = lock(tmp.path()); + for (base, why) in [ + ("https://user:secret@repo.example/x", "credentials"), + ("repo.example/artifactory/socket-patch", "http(s)"), + ("https://repo.example/x?repo=y", "query"), + ] { + for command in ["scan", "list"] { + let mut cmd = cli(); + cmd.args([ + command, + "--cwd", + tmp.path().to_str().unwrap(), + "--patch-mirror-url", + base, + ]); + let out = run(cmd); + assert_eq!( + out.code, + Some(2), + "{command} {base}: {}\n{}", + out.stdout, + out.stderr + ); + assert!(out.stderr.contains(why), "{base}: {}", out.stderr); + assert!(!out.stderr.contains("secret"), "{}", out.stderr); + } + // `apply` does not use the mirror and runs unattended from install + // hooks: a base it cannot use must not fail the install. + let mut cmd = cli(); + cmd.args(["apply", "--cwd", tmp.path().to_str().unwrap()]) + .env("SOCKET_PATCH_MIRROR_URL", base); + let out = run(cmd); + assert_eq!( + out.code, + Some(0), + "apply {base}: {}\n{}", + out.stdout, + out.stderr + ); + assert!(!out.stderr.contains("secret"), "{}", out.stderr); + } + assert_eq!(lock(tmp.path()), pristine); +} diff --git a/crates/socket-patch-core/src/api/client.rs b/crates/socket-patch-core/src/api/client.rs index 6d3cf9f80..fb137ea93 100644 --- a/crates/socket-patch-core/src/api/client.rs +++ b/crates/socket-patch-core/src/api/client.rs @@ -2091,6 +2091,40 @@ impl ApiClient { .await } + /// A checker for a patch-server mirror (`--patch-mirror-url`): one HTTP + /// client for all of a run's checks ([`MirrorProber::probe`]). + /// `authorization`, when given, is the operator's own `Authorization` + /// header value for the mirror (`SOCKET_PATCH_MIRROR_AUTHORIZATION`) + /// and goes to no other host. `Err` carries why no check can be made. + pub fn mirror_prober(&self, authorization: Option<&str>) -> Result { + let authorization = match authorization.map(str::trim).filter(|v| !v.is_empty()) { + Some(value) => { + let mut value = header::HeaderValue::from_str(value).map_err(|_| { + "SOCKET_PATCH_MIRROR_AUTHORIZATION is not a valid header value".to_string() + })?; + value.set_sensitive(true); + Some(value) + } + None => None, + }; + let mut headers = HeaderMap::new(); + headers.insert( + header::USER_AGENT, + HeaderValue::from_static(USER_AGENT_VALUE), + ); + let client = crate::utils::http::client_builder() + .default_headers(headers) + .redirect(reqwest::redirect::Policy::none()) + .referer(false) + .build() + .map_err(|_| "the HTTP client could not be built".to_string())?; + Ok(MirrorProber { + client, + authorization, + attempt_timeout: self.vendor_retry.attempt_timeout, + }) + } + /// One GET of a grant-tokenized serve URL, its body capped at /// `max_bytes`; `noun` names the download in errors. The grant token in /// the path is the authorization, so this uses a plain (no-auth) client. @@ -2241,6 +2275,76 @@ impl ApiClient { } } +/// What a patch-server mirror answered for one mirrored artifact URL +/// ([`MirrorProber::probe`]). +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum MirrorProbe { + /// It serves the artifact. + Serves, + /// It asked for credentials the run does not have (the HTTP status): + /// whether it serves the artifact is unknown. + NeedsCredentials(u16), + /// It redirected the request (the HTTP status), which the check does + /// not follow: whether it serves the artifact is unknown. + Redirected(u16), + /// It answered, and does not serve the artifact; the reason carries no + /// URL. + Refused(String), + /// It could not be reached, or did not answer in time; the reason + /// carries no URL. Another request would wait as long for the same. + Unreachable(String), +} + +/// Checks a patch-server mirror, one `HEAD` per mirrored artifact URL +/// ([`ApiClient::mirror_prober`]). +pub struct MirrorProber { + client: reqwest::Client, + authorization: Option, + attempt_timeout: Duration, +} + +impl MirrorProber { + /// Ask the mirror whether it serves `url`, a mirrored artifact URL, + /// with one `HEAD`. The request carries no Socket credential. + /// + /// Redirects are not followed: the mirrored URL holds the grant token, + /// which a followed redirect would hand to the next host as a + /// `Referer`, and a gateway that redirects to a login page answering + /// 200 would otherwise read as a mirror that serves the patch. No + /// retry: this is a check before anything is written, and a mirror + /// that cannot answer one request is not one to pin installs to. + pub async fn probe(&self, url: &str) -> MirrorProbe { + if !(url.starts_with("https://") || url.starts_with("http://")) { + return MirrorProbe::Refused("it is not an http(s) URL".to_string()); + } + debug_log(&format!("HEAD patch mirror {}", redact_url(url))); + let mut request = self.client.head(url); + if let Some(value) = &self.authorization { + request = request.header(header::AUTHORIZATION, value.clone()); + } + match tokio::time::timeout(self.attempt_timeout, request.send()).await { + Ok(Ok(resp)) => match resp.status() { + status if status.is_success() => MirrorProbe::Serves, + StatusCode::UNAUTHORIZED + | StatusCode::FORBIDDEN + | StatusCode::PROXY_AUTHENTICATION_REQUIRED => { + MirrorProbe::NeedsCredentials(resp.status().as_u16()) + } + status if status.is_redirection() => MirrorProbe::Redirected(status.as_u16()), + status => MirrorProbe::Refused(format!("it answered HTTP {}", status.as_u16())), + }, + Ok(Err(e)) => MirrorProbe::Unreachable(format!( + "the request failed ({})", + network_error_detail(&e) + )), + Err(_) => MirrorProbe::Unreachable(format!( + "it did not answer within {:?}", + self.attempt_timeout + )), + } + } +} + /// The unspent remainder of a retry budget: a first attempt that failed /// retryably and was set aside, carrying the failure its retry still owes a /// debug line and the `Retry-After` it still owes a pause. Only diff --git a/crates/socket-patch-core/src/hosted/engine.rs b/crates/socket-patch-core/src/hosted/engine.rs index 09b507b2e..105a9bed5 100644 --- a/crates/socket-patch-core/src/hosted/engine.rs +++ b/crates/socket-patch-core/src/hosted/engine.rs @@ -383,6 +383,257 @@ pub fn build_candidates( candidates } +/// Skip reason and warning code of a patch whose ecosystem cannot be served +/// through a patch-server mirror (`--patch-mirror-url`): nothing is written +/// for it, so it stays as it is. +pub const MIRROR_UNSUPPORTED: &str = "redirect_mirror_unsupported"; + +/// Skip reason and warning code of a grant that cannot be moved onto the +/// run's mirror: a URL without a served path, or a mirrored URL the grant +/// gate refuses. Nothing is written for it. +pub const MIRROR_URL_REFUSED: &str = "redirect_mirror_url_refused"; + +/// Where each mirrored reference is served from: the URL the patch +/// server's grant named, keyed by the mirrored URL the candidate now +/// carries. The run's own downloads (a wheel's metadata, a tarball's +/// `package.json`, a gem to refresh a stale install with) read the served +/// URL: the grant token in it is all the authorization that takes, where +/// the mirror may ask for credentials the run was never given. +#[derive(Debug, Clone, Default)] +pub struct MirrorSources { + /// The mirror's base URL (empty when the run has no mirror). + base: String, + /// Mirrored artifact URL -> served artifact URL. + served: BTreeMap, + /// `(mirrored, served)` registry index URLs, without a trailing slash: + /// a URL below a mirrored index is served below its served twin. + registries: Vec<(String, String)>, +} + +impl MirrorSources { + /// The URL to download `url` from: where the patch server serves it + /// when `url` is a mirrored artifact or lies under a mirrored registry + /// index, else itself. + pub fn fetch_url<'a>(&'a self, url: &'a str) -> std::borrow::Cow<'a, str> { + use std::borrow::Cow; + if let Some(served) = self.served.get(url) { + return Cow::Borrowed(served); + } + for (mirrored, served) in &self.registries { + if let Some(rest) = url.strip_prefix(mirrored.as_str()) { + if rest.is_empty() || rest.starts_with('/') { + return Cow::Owned(format!("{served}{rest}")); + } + } + } + Cow::Borrowed(url) + } + + /// [`Self::fetch_url`] for a candidate's own artifact URL, borrowed: + /// the served URL of a mirrored artifact, else the URL itself. + pub fn fetch_artifact_url<'a>(&'a self, artifact_url: &'a str) -> &'a str { + self.served + .get(artifact_url) + .map_or(artifact_url, String::as_str) + } + + /// The base URL of the mirror these references were moved onto. + pub fn base(&self) -> &str { + &self.base + } + + /// The mirrored artifact URLs, in order. + pub fn mirrored_urls(&self) -> impl Iterator { + self.served.keys().map(String::as_str) + } + + /// The URLs the patch server serves the mirrored artifacts at. + pub fn served_urls(&self) -> impl Iterator { + self.served.values().map(String::as_str) + } + + pub fn is_empty(&self) -> bool { + self.served.is_empty() + } +} + +/// Why `ecosystem`'s hosted reference cannot move onto a mirror, if it +/// cannot. A Cargo sparse index and a NuGet service index are documents the +/// patch server renders with its own absolute download URLs in them +/// (`config.json`'s `dl`, the service index's resource ids), so a client +/// reading the index through the mirror is sent straight back to the patch +/// server for the package. +fn mirror_unsupported(ecosystem: &str) -> Option<&'static str> { + match ecosystem { + "cargo" => Some( + "a Cargo registry index names the patch server's own download URL, which a mirror \ + cannot rewrite", + ), + "nuget" => Some( + "a NuGet service index names the patch server's own package URLs, which a mirror \ + cannot rewrite", + ), + _ => None, + } +} + +/// Move every candidate's hosted reference onto `mirror` +/// (`--patch-mirror-url`): its artifact URL and its registry index URL, +/// everything before the path swapped for the mirror's base. Run right +/// after [`build_candidates`], so the takeover, the withholds and every +/// rewriter see the URL the project's files will hold (a re-run finds its +/// own earlier pin, not a foreign one). +/// +/// * A Go candidate is left alone: its pin is the module path +/// `patch.socket.dev/gopatch/`, which names no host to fetch from +/// (GOPROXY decides that). +/// * A Cargo or NuGet candidate is skipped with [`MIRROR_UNSUPPORTED`]: +/// see [`mirror_unsupported`]. A direct pin is not written in its place, +/// because the operator configured a mirror to say installs must not +/// reach the patch server. +/// * A candidate whose mirrored URLs the grant gate refuses is skipped +/// with [`MIRROR_URL_REFUSED`]. +/// +/// `patch_server_origins` must hold the mirror's base: it is how the grant +/// gate takes a plain-`http` mirror, and how every reader recognizes the +/// mirrored references afterwards. +pub fn mirror_candidates( + candidates: Vec, + mirror: &crate::patch::redirect::mirror::PatchMirror, + patch_server_origins: &[String], + skipped: &mut Vec, + warnings: &mut Vec, +) -> (Vec, MirrorSources) { + use crate::patch::redirect::grant; + let mut sources = MirrorSources { + base: mirror.base().to_string(), + ..MirrorSources::default() + }; + let mut kept = Vec::with_capacity(candidates.len()); + for mut candidate in candidates { + let dep = &mut candidate.dep; + if dep.ecosystem == "golang" { + kept.push(candidate); + continue; + } + let mut skip = |code: &str, why: String, purl: &str, uuid: &str| { + // Not "it stays unpatched": an earlier run may have pinned it + // on the patch server, which this run does not read here. + let detail = format!("{purl}: {why}; nothing was changed for it"); + warnings.push(warning(code, detail.clone())); + skipped.push(SkippedPatch { + detail: Some(detail), + ..SkippedPatch::new(purl, uuid, code) + }); + }; + if let Some(why) = mirror_unsupported(&dep.ecosystem) { + skip( + MIRROR_UNSUPPORTED, + format!( + "{why}, so it cannot be installed through the patch mirror; use \ + `--mode vendored` for it" + ), + &candidate.purl, + &dep.patch_uuid, + ); + continue; + } + let served_artifact = dep.artifact_url.clone(); + // `None`: an index URL that cannot move; `Some(None)`: no override. + let mirrored_index = match &dep.registry_override { + Some(ov) => mirror.rebase(&ov.index_url).map(Some), + None => Some(None), + }; + let (Some(mirrored_artifact), Some(mirrored_index)) = + (mirror.rebase(&served_artifact), mirrored_index) + else { + skip( + MIRROR_URL_REFUSED, + "the patch server's reference for it has no path to serve through the patch \ + mirror" + .to_string(), + &candidate.purl, + &dep.patch_uuid, + ); + continue; + }; + dep.artifact_url = mirrored_artifact; + let mut registry = None; + if let (Some(ov), Some(index_url)) = (dep.registry_override.as_mut(), mirrored_index) { + registry = Some(( + index_url.trim_end_matches('/').to_string(), + ov.index_url.trim_end_matches('/').to_string(), + )); + ov.index_url = index_url; + } + if let Err(flaw) = grant::check(dep, patch_server_origins) { + let reason = match flaw { + grant::Flaw::Malformed(reason) | grant::Flaw::InsecureUrl(reason) => reason, + }; + skip( + MIRROR_URL_REFUSED, + format!("its reference on the patch mirror was refused ({reason})"), + &candidate.purl, + &dep.patch_uuid, + ); + continue; + } + sources + .served + .insert(dep.artifact_url.clone(), served_artifact); + sources.registries.extend(registry); + kept.push(candidate); + } + (kept, sources) +} + +/// Skip reason and warning code of a patch the project already pins +/// through a host this run was not given: nothing is changed for it. +pub const PIN_ON_UNCONFIGURED_HOST: &str = "redirect_pin_on_unconfigured_host"; + +/// Withhold every candidate whose patch one of `files` already pins through +/// a host the run does not know as a patch server: a URL with the hosted +/// shape for the candidate's patch uuid on an origin outside +/// `accepted_origins` (and Socket's own). That is what a project wired +/// through a patch mirror looks like to a run that was not given +/// `--patch-mirror-url`, and rewriting it would move a working pin onto a +/// host the project's builds may not be able to reach. The files stay as +/// they are and the run says which host to configure. +/// +/// `accepted_origins` must hold everything this run can itself write or +/// supersede: the configured origins, the mirror, and the origins of this +/// run's grants (a pin an earlier run of the same server wrote is its own). +pub fn withhold_foreign_host_pins( + files: &BTreeMap, + accepted_origins: &[String], + candidates: &mut Vec, + skipped: &mut Vec, + warnings: &mut Vec, +) { + candidates.retain(|c| { + let uuid = c.dep.patch_uuid.as_str(); + let Some((file, host)) = files.iter().find_map(|(path, text)| { + crate::vex::discover::foreign_hosted_host_in_text(text, uuid, accepted_origins) + .map(|host| (path.as_str(), host)) + }) else { + return true; + }; + let detail = format!( + "{}: {file} already pins patch {uuid} through {host}, which this run was not given \ + as a patch mirror or patch server, so nothing was changed for it; if that host is \ + your patch mirror set --patch-mirror-url (SOCKET_PATCH_MIRROR_URL) to its base \ + URL, if it is another patch server pass --patch-server-url, and re-run", + c.purl + ); + warnings.push(warning(PIN_ON_UNCONFIGURED_HOST, detail.clone())); + skipped.push(SkippedPatch { + detail: Some(detail), + ..SkippedPatch::new(&c.purl, uuid, PIN_ON_UNCONFIGURED_HOST) + }); + false + }); +} + /// Whether an npm candidate would rewrite a `bun.lockb` that is a symbolic /// link (atomic replacement cannot preserve a link; previews refuse too). pub fn bun_lockb_symlinked(view: &ProjectView<'_>, candidates: &[Candidate]) -> bool { @@ -4448,6 +4699,315 @@ mod tests { } } + /// A candidate of `ecosystem` on Socket's patch server, shaped the way + /// the reference endpoint grants it, with the registry override its + /// ecosystem carries. + fn mirror_candidate(ecosystem: &str, name: &str, version: &str) -> Candidate { + use crate::patch::redirect::{RegistryOverride, RegistryOverrideIdentifiers}; + const TOKEN: &str = "22222222-2222-4222-8222-222222222222"; + let registry = |kind: &str, tail: &str| { + Some(RegistryOverride { + kind: kind.into(), + index_url: format!( + "https://patch.socket.dev/patch-registry/{ecosystem}/{TOKEN}/{PATCH_UUID}/{tail}" + ), + identifiers: RegistryOverrideIdentifiers::default(), + }) + }; + let mut dep = grant_dep( + ecosystem, + format!( + "https://patch.socket.dev/patch/{ecosystem}/{name}/{version}/{TOKEN}/{PATCH_UUID}/artifact.bin" + ), + ); + dep.name = name.into(); + dep.version = version.into(); + dep.token = TOKEN.into(); + dep.registry_override = match ecosystem { + "gem" => registry("rubygems-compact-index", ""), + "maven" => registry("maven2", "maven2"), + "cargo" => registry("cargo-sparse", "index/"), + "nuget" => registry("nuget-v3", "index.json"), + "golang" => registry("goproxy", ""), + _ => None, + }; + Candidate { + purl: format!("pkg:{ecosystem}/{name}@{version}"), + dep, + } + } + + /// `--patch-mirror-url` moves every reference a client fetches by URL + /// onto the mirror (same path), leaves a Go module pin alone, and + /// withholds the two registries whose index names the patch server's + /// own host: nothing direct is written in their place. + #[test] + fn mirror_candidates_moves_url_references_and_withholds_what_a_mirror_cannot_carry() { + use crate::patch::redirect::mirror::PatchMirror; + let mirror = PatchMirror::parse("https://repo.example/artifactory/socket-patch").unwrap(); + let origins = vec![mirror.base().to_string()]; + let candidates = vec![ + mirror_candidate("npm", "left-pad", "1.3.0"), + mirror_candidate("pypi", "urllib3", "1.26.18"), + mirror_candidate("composer", "acme/lib", "1.0.0"), + mirror_candidate("gem", "rack", "3.0.0"), + mirror_candidate("maven", "org.acme/lib", "1.0.0"), + mirror_candidate("golang", "github.com/acme/lib", "v1.0.0"), + mirror_candidate("cargo", "serde", "1.0.0"), + mirror_candidate("nuget", "Newtonsoft.Json", "13.0.3"), + ]; + let served: Vec = candidates.iter().map(|c| c.dep.clone()).collect(); + let (mut skipped, mut warnings) = (Vec::new(), Vec::new()); + let (kept, sources) = + mirror_candidates(candidates, &mirror, &origins, &mut skipped, &mut warnings); + + let ecosystems: Vec<&str> = kept.iter().map(|c| c.dep.ecosystem.as_str()).collect(); + assert_eq!( + ecosystems, + ["npm", "pypi", "composer", "gem", "maven", "golang"] + ); + for (candidate, before) in kept.iter().zip(&served) { + let dep = &candidate.dep; + if dep.ecosystem == "golang" { + assert_eq!(dep.artifact_url, before.artifact_url); + assert_eq!( + dep.registry_override.as_ref().unwrap().index_url, + before.registry_override.as_ref().unwrap().index_url + ); + assert_eq!( + sources.fetch_url(&dep.artifact_url), + dep.artifact_url.as_str() + ); + continue; + } + // Same path, the mirror's base in front of it. + let path = before + .artifact_url + .strip_prefix("https://patch.socket.dev") + .unwrap(); + assert_eq!( + dep.artifact_url, + format!("https://repo.example/artifactory/socket-patch{path}") + ); + // The run's own downloads still read the patch server. + assert_eq!( + sources.fetch_url(&dep.artifact_url), + before.artifact_url.as_str() + ); + if let (Some(ov), Some(before_ov)) = (&dep.registry_override, &before.registry_override) + { + let path = before_ov + .index_url + .strip_prefix("https://patch.socket.dev") + .unwrap(); + assert_eq!( + ov.index_url, + format!("https://repo.example/artifactory/socket-patch{path}") + ); + // A file under the mirrored index is downloaded from under + // the served one. + let under = format!("{}/gems/x.gem", ov.index_url.trim_end_matches('/')); + assert_eq!( + sources.fetch_url(&under), + format!("{}/gems/x.gem", before_ov.index_url.trim_end_matches('/')) + ); + } + // The mirrored reference names the same patch and token. + assert_eq!( + crate::origin::hosted_patch_uuid(&dep.artifact_url, &origins).as_deref(), + Some(PATCH_UUID) + ); + assert_eq!(dep.token, before.token); + } + assert_eq!( + skipped + .iter() + .map(|s| (s.purl.as_str(), s.reason.as_str())) + .collect::>(), + [ + ("pkg:cargo/serde@1.0.0", MIRROR_UNSUPPORTED), + ("pkg:nuget/Newtonsoft.Json@13.0.3", MIRROR_UNSUPPORTED), + ] + ); + assert_eq!(warnings.len(), 2); + for w in &warnings { + assert_eq!(w.code, MIRROR_UNSUPPORTED); + assert!(w.detail.contains("--mode vendored"), "{}", w.detail); + assert!( + !w.detail.contains("22222222-2222-4222-8222-222222222222"), + "a skip detail never carries the grant token: {}", + w.detail + ); + } + } + + /// A plain-`http` mirror is taken because the operator named it (its + /// base is one of the run's origins); a grant that has no path to + /// serve is withheld instead of being pinned at the mirror's root. + #[test] + fn mirror_candidates_takes_an_http_mirror_and_refuses_a_pathless_grant() { + use crate::patch::redirect::mirror::PatchMirror; + let mirror = PatchMirror::parse("http://artifactory.corp:8081/artifactory/sp").unwrap(); + let origins = vec![mirror.base().to_string()]; + let mut pathless = mirror_candidate("npm", "left-pad", "1.3.0"); + pathless.dep.artifact_url = "https://patch.socket.dev".into(); + let (mut skipped, mut warnings) = (Vec::new(), Vec::new()); + let (kept, sources) = mirror_candidates( + vec![mirror_candidate("npm", "left-pad", "1.3.0"), pathless], + &mirror, + &origins, + &mut skipped, + &mut warnings, + ); + assert_eq!(kept.len(), 1); + assert!(kept[0] + .dep + .artifact_url + .starts_with("http://artifactory.corp:8081/artifactory/sp/patch/npm/left-pad/")); + assert_eq!(sources.mirrored_urls().count(), 1); + assert_eq!(skipped.len(), 1); + assert_eq!(skipped[0].reason, MIRROR_URL_REFUSED); + + // Without the mirror among the origins the grant gate refuses the + // plain-http URL, so the caller's allowlist is load-bearing. + let (mut skipped, mut warnings) = (Vec::new(), Vec::new()); + let (kept, _) = mirror_candidates( + vec![mirror_candidate("npm", "left-pad", "1.3.0")], + &mirror, + &[], + &mut skipped, + &mut warnings, + ); + assert!(kept.is_empty()); + assert_eq!(skipped[0].reason, MIRROR_URL_REFUSED); + } + + /// A patch the project's files already pin through a host the run was + /// not given is withheld; one pinned on an accepted origin, or not + /// pinned at all, is not. + #[test] + fn a_pin_on_an_unconfigured_host_is_withheld_and_named() { + let candidate = mirror_candidate("npm", "left-pad", "1.3.0"); + let path = candidate + .dep + .artifact_url + .strip_prefix("https://patch.socket.dev") + .unwrap() + .to_string(); + let lock_at = |base: &str| { + BTreeMap::from([( + "package-lock.json".to_string(), + format!(r#"{{"resolved": "{base}{path}", "integrity": "sha512-x"}}"#), + )]) + }; + let run = |files: &BTreeMap, accepted: &[String]| { + let mut candidates = vec![candidate.clone()]; + let (mut skipped, mut warnings) = (Vec::new(), Vec::new()); + withhold_foreign_host_pins( + files, + accepted, + &mut candidates, + &mut skipped, + &mut warnings, + ); + (candidates.len(), skipped, warnings) + }; + let mirror = "https://repo.example/artifactory/socket-patch"; + + // The mirror was not given: withheld, the host and the setting named. + let (kept, skipped, warnings) = run(&lock_at(mirror), &[]); + assert_eq!(kept, 0); + assert_eq!(skipped[0].reason, PIN_ON_UNCONFIGURED_HOST); + assert!( + warnings[0].detail.contains("repo.example") + && warnings[0].detail.contains("--patch-mirror-url") + && warnings[0].detail.contains("package-lock.json"), + "{}", + warnings[0].detail + ); + // The mirror was given: the run's own pin. + assert_eq!(run(&lock_at(mirror), &[mirror.to_string()]).0, 1); + // Pinned on Socket's own server, or still on the registry. + assert_eq!(run(&lock_at("https://patch.socket.dev"), &[]).0, 1); + let registry = BTreeMap::from([( + "package-lock.json".to_string(), + r#"{"resolved": "https://registry.npmjs.org/left-pad/-/left-pad-1.3.0.tgz"}"# + .to_string(), + )]); + assert_eq!(run(®istry, &[]).0, 1); + // A lock that spells the url inside a specifier, as bun and yarn + // berry key a tarball dependency (`name@https://…`). + let specifier = BTreeMap::from([( + "yarn.lock".to_string(), + format!("\"left-pad@{mirror}{path}\":\n resolution: \"left-pad@{mirror}{path}\"\n"), + )]); + let (kept, skipped, _) = run(&specifier, &[]); + assert_eq!(kept, 0); + assert_eq!(skipped[0].reason, PIN_ON_UNCONFIGURED_HOST); + assert_eq!(run(&specifier, &[mirror.to_string()]).0, 1); + // The same text read for where it pins the patch: the mirror when + // the mirror is a known origin, nothing when it is not, and the + // patch server for a direct pin. + let pinned_on = |files: &BTreeMap, origins: &[String]| { + files.values().find_map(|text| { + crate::vex::discover::hosted_host_in_text(text, PATCH_UUID, origins) + }) + }; + assert_eq!( + pinned_on(&specifier, &[mirror.to_string()]).as_deref(), + Some("repo.example") + ); + assert_eq!(pinned_on(&specifier, &[]), None); + assert_eq!( + pinned_on(&lock_at("https://patch.socket.dev"), &[]).as_deref(), + Some("patch.socket.dev") + ); + // An older composer.lock escapes every slash. Pinned on Socket's + // own server it is the run's own pin, not one on a host called + // `https`; pinned on a mirror the run was not given, it is withheld + // and the mirror's host is named. + let escaped = |base: &str| { + let url = format!("{base}{path}").replace('/', "\\/"); + BTreeMap::from([( + "composer.lock".to_string(), + format!(r#"{{"dist": {{"url": "{url}"}}}}"#), + )]) + }; + assert!(escaped(mirror)["composer.lock"].contains(r"https:\/\/repo.example\/")); + assert_eq!(run(&escaped("https://patch.socket.dev"), &[]).0, 1); + assert_eq!(run(&escaped(mirror), &[mirror.to_string()]).0, 1); + let (kept, _, warnings) = run(&escaped(mirror), &[]); + assert_eq!(kept, 0); + assert!( + warnings[0].detail.contains("repo.example"), + "{}", + warnings[0].detail + ); + assert_eq!( + pinned_on(&escaped(mirror), &[mirror.to_string()]).as_deref(), + Some("repo.example") + ); + // Another repository of the mirror's host is not the mirror, once + // the mirror is listed as one (`mirror_entry`, as the CLI lists it). + let sibling = "https://repo.example/artifactory/team-local"; + let listed = [crate::origin::mirror_entry(mirror)]; + assert_eq!(run(&lock_at(mirror), &listed).0, 1); + assert_eq!(run(&lock_at(sibling), &listed).0, 0); + assert_eq!(pinned_on(&lock_at(sibling), &listed), None); + assert_eq!( + pinned_on(&lock_at(mirror), &listed).as_deref(), + Some("repo.example") + ); + // Another patch's uuid on that host says nothing about this one. + let other = lock_at(mirror).into_iter().map(|(k, v)| { + ( + k, + v.replace(PATCH_UUID, "99999999-9999-4999-8999-999999999999"), + ) + }); + assert_eq!(run(&other.collect(), &[]).0, 1); + } + /// The hosted skip details reach `--json`: neither the grant token nor /// any userinfo survives, however reqwest re-renders the URL (a /// trailing `/`, a different quoting) and under whatever root the diff --git a/crates/socket-patch-core/src/hosted/memory/stages.rs b/crates/socket-patch-core/src/hosted/memory/stages.rs index 345ad98b2..79a99af09 100644 --- a/crates/socket-patch-core/src/hosted/memory/stages.rs +++ b/crates/socket-patch-core/src/hosted/memory/stages.rs @@ -195,6 +195,21 @@ pub(crate) async fn plan( engine::withhold_user_patched(&view, &read.files, &mut candidates, &mut skipped, false) .await, ); + // A patch the project already pins through a host these grants do not + // name (a patch mirror the embedding host knows nothing of) is left + // alone, as on disk: rewriting it would move a working pin onto the + // patch server. + let own_origins = crate::patch::redirect::upstream::foreign_dep_origins( + candidates.iter().map(|c| &c.dep), + &[], + ); + engine::withhold_foreign_host_pins( + &read.files, + &own_origins, + &mut candidates, + &mut skipped, + &mut pre_warnings, + ); let wheels = engine::wheel_targets(&candidates, &read.files) .into_iter() .map(|(dep, sha256)| (dep.artifact_url.clone(), sha256.to_string())) diff --git a/crates/socket-patch-core/src/origin.rs b/crates/socket-patch-core/src/origin.rs index ca2df6608..41f9d851a 100644 --- a/crates/socket-patch-core/src/origin.rs +++ b/crates/socket-patch-core/src/origin.rs @@ -180,6 +180,19 @@ pub(crate) fn hosted_shape_uuid(url: &str) -> Option { hosted_patch_uuid(parsed.as_str(), &[origin]) } +/// The `host[:port]` a hosted-shape `url` routes to, read the way +/// [`hosted_shape_uuid`] reads it (normalized, userinfo dropped): the name +/// to show for a pin on an origin a run was not given. +pub(crate) fn hosted_shape_host(url: &str) -> Option { + let normalized = normalized_hosted_url(url)?; + let parsed = reqwest::Url::parse(&normalized).ok()?; + let host = parsed.host_str()?; + Some(match parsed.port() { + Some(port) => format!("{host}:{port}"), + None => host.to_string(), + }) +} + /// `url` parsed, when it is an `http(s)` url with no userinfo on an /// accepted origin — Socket's patch server or one of `extra_origins` — and /// whether that origin is Socket's own: the origin half of @@ -221,20 +234,112 @@ pub fn hosted_patch_url_uuids(url: &str, extra_origins: &[String]) -> Option String { + format!("{MIRROR_ENTRY_PREFIX}{}", base.trim()) +} + +/// The url of an allowlist `entry`, and whether the entry is a mirror's +/// base ([`mirror_entry`]). +fn entry_url(entry: &str) -> (&str, bool) { + let entry = entry.trim(); + match entry.strip_prefix(MIRROR_ENTRY_PREFIX) { + Some(base) => (base, true), + None => (entry, false), + } +} + /// Whether `url` is on one of `origins` (the operator's `--patch-server-url` -/// deployments), compared on scheme + host + port: the one rule by which a -/// configured origin is recognized, a plain-`http` one included. +/// deployments and patch mirror), compared on scheme + host + port: the one +/// rule by which a configured origin is recognized, a plain-`http` one +/// included. A mirror's entry ([`mirror_entry`]) also needs the url's path +/// to be at or below the mirror's base. pub(crate) fn on_configured_origin(url: &reqwest::Url, origins: &[String]) -> bool { origins.iter().any(|origin| { - reqwest::Url::parse(origin.trim()).is_ok_and(|o| { + let (origin, mirror) = entry_url(origin); + reqwest::Url::parse(origin).is_ok_and(|o| { o.scheme() == url.scheme() && o.host_str().is_some() && o.host_str() == url.host_str() && o.port_or_known_default() == url.port_or_known_default() + && (!mirror || under_base_path(url.path(), o.path())) }) }) } +/// Whether `path` is `base` or below it, a whole segment at a time. A base +/// of `/` (a mirror given with no path) holds every path. +fn under_base_path(path: &str, base: &str) -> bool { + let base = base.trim_end_matches('/'); + base.is_empty() + || path + .strip_prefix(base) + .is_some_and(|rest| rest.is_empty() || rest.starts_with('/')) +} + +/// `url` reduced to its origin, `scheme://host[:port]`: an allowlist entry +/// that holds every path of that host. `None` for what is not a url with a +/// host. +pub fn bare_origin(url: &str) -> Option { + let parsed = reqwest::Url::parse(url.trim()).ok()?; + let host = parsed.host_str()?; + Some(match parsed.port() { + Some(port) => format!("{}://{host}:{port}", parsed.scheme()), + None => format!("{}://{host}", parsed.scheme()), + }) +} + +/// The path of a hosted artifact `url` up to and including the `/` before +/// the level the patch server serves patches under (`patch//…`, +/// `patch-registry//…`, `gopatch/…`), when that level is NOT the first +/// of the path: the base of a deployment that sits under a path prefix, +/// which is what a patch mirror is. `None` for a url served from the root +/// of its host, and for one in no such shape. +fn hosted_base_path(url: &str) -> Option { + let parsed = reqwest::Url::parse(&normalized_hosted_url(url)?).ok()?; + let segments: Vec<&str> = parsed.path_segments()?.collect(); + let level = (0..segments.len()).find(|&i| { + segments[i] == GO_SERVE_ROOT + || (SERVE_ROOTS.contains(&segments[i]) + && segments + .get(i + 1) + .is_some_and(|eco| ECOSYSTEM_DIRS.contains(eco))) + })?; + (level > 0).then(|| format!("/{}/", segments[..level].join("/"))) +} + +/// Whether `existing` may be an earlier pin of the artifact `current` is +/// served as, as far as their BASE goes: when `current` is served under a +/// path prefix (a patch mirror's repository), `existing` must sit under +/// that same prefix. The rewriters that take over an earlier hosted pin in +/// place recognize it by origin and file name, which is enough on a host +/// that serves nothing but patches. A mirror shares its host with the +/// project's own packages, so there the origin alone would claim a fork of +/// the same name and version as an earlier Socket pin. +/// +/// Both urls are read as a client reads them, not as text: `existing` in +/// any spelling a lock uses ([`normalized_hosted_url`]), with `.` and `..` +/// segments resolved, so `/../other/x` is not under ``. The +/// origin is the caller's to compare. +pub(crate) fn same_hosted_base(existing: &str, current: &str) -> bool { + let Some(base) = hosted_base_path(current) else { + return true; + }; + normalized_hosted_url(existing) + .and_then(|url| reqwest::Url::parse(&url).ok()) + .is_some_and(|url| url.path().starts_with(&base)) +} + /// The host of `url` when it is a PLAIN `http(s)` url — printable ASCII /// only, no `%`, `\\` or userinfo, a DNS name of letters, digits, `.` and /// `-` whose last label starts with a letter — lowercased, which is then @@ -314,7 +419,7 @@ impl<'a> OriginPolicy<'a> { self.hosts.get_or_init(|| { let mut hosts = BTreeSet::from([SOCKET_PATCH_SERVER_HOST.to_string()]); hosts.extend(self.origins.iter().filter_map(|o| { - reqwest::Url::parse(o.trim()) + reqwest::Url::parse(entry_url(o).0) .ok()? .host_str() .map(str::to_ascii_lowercase) @@ -1167,4 +1272,126 @@ mod tests { assert_eq!(plain_url_domain(not_plain), None, "{not_plain}"); } } + + /// A mirror's entry holds what is under the mirror's base and nothing + /// else on its host: a patch mirror is one repository of an artifact + /// host that serves many. A `--patch-server-url` entry holds its whole + /// origin, whatever path it was given with. + #[test] + fn a_mirror_entry_holds_only_what_is_under_its_base() { + let uuid = "11111111-1111-4111-8111-111111111111"; + let tail = format!("patch/npm/left-pad/1.3.0/tok/{uuid}/left-pad-1.3.0.tgz"); + let mirror = vec![mirror_entry( + "https://repo.example/artifactory/socket-patch", + )]; + assert_eq!( + mirror, + ["mirror+https://repo.example/artifactory/socket-patch"] + ); + let under = |base: &str| format!("https://repo.example/{base}/{tail}"); + assert_eq!( + hosted_patch_uuid(&under("artifactory/socket-patch"), &mirror).as_deref(), + Some(uuid) + ); + for elsewhere in [ + "artifactory/team-local", + "artifactory/socket-patch-evil", + "artifactory", + "other/artifactory/socket-patch", + ] { + assert_eq!( + hosted_patch_uuid(&under(elsewhere), &mirror), + None, + "{elsewhere} is not the mirror" + ); + } + // A trailing slash on the base changes nothing; a mirror given with + // no path holds its whole host; and so does a patch-server entry, + // with or without a path. + for whole in [ + vec![mirror_entry( + "https://repo.example/artifactory/socket-patch/", + )], + vec![mirror_entry("https://repo.example")], + vec![mirror_entry("https://repo.example/")], + vec!["https://repo.example".to_string()], + vec!["https://repo.example/some/base".to_string()], + ] { + assert_eq!( + hosted_patch_uuid(&under("artifactory/socket-patch"), &whole).as_deref(), + Some(uuid), + "{whole:?}" + ); + } + assert_eq!( + hosted_patch_uuid( + &under("artifactory/team-local"), + &["https://repo.example/artifactory/socket-patch".to_string()] + ) + .as_deref(), + Some(uuid), + "an unmarked entry is an origin" + ); + // The mirror's host is one the identity sweep looks for. + assert!(OriginPolicy::new(&mirror).hosts().contains("repo.example")); + assert_eq!( + bare_origin("http://127.0.0.1:8081/artifactory/x?y#z").as_deref(), + Some("http://127.0.0.1:8081") + ); + assert_eq!( + bare_origin("https://Repo.Example/a").as_deref(), + Some("https://repo.example") + ); + assert_eq!(bare_origin("repo.example/a"), None); + } + + /// An earlier pin shares the base its replacement is served under: on a + /// mirror, a url elsewhere on the mirror's host is the project's own. + #[test] + fn an_earlier_pin_shares_the_base_of_a_mirrored_artifact() { + let leaf = "urllib3-1.26.18-py2.py3-none-any.whl"; + let base = "https://repo.example/artifactory/socket-patch"; + let mirrored = format!("{base}/patch/pypi/urllib3/1.26.18/tok/u/{leaf}"); + assert_eq!( + hosted_base_path(&mirrored).as_deref(), + Some("/artifactory/socket-patch/") + ); + let earlier = format!("{base}/patch/pypi/urllib3/1.26.18/old/u2/{leaf}"); + for ours in [ + format!("{earlier}#sha256=0"), + // The spellings a lock may hold the same url in. + earlier.replace('/', "\\/"), + earlier.replace(':', "%3A").replace('/', "%2F"), + earlier.replace("https://repo.example", "HTTPS://Repo.Example"), + earlier.replace("https://repo.example", "https://repo.example:443"), + ] { + assert!(same_hosted_base(&ours, &mirrored), "{ours}"); + } + for own in [ + format!("https://repo.example/artifactory/pypi-local/{leaf}"), + format!("https://repo.example/artifactory/socket-patch-2/patch/pypi/urllib3/1.26.18/t/u/{leaf}"), + format!("https://repo.example/{leaf}"), + // A path that only starts with the base as text. + format!("{base}/../pypi-local/{leaf}"), + format!("{base}/%2e%2e/pypi-local/{leaf}"), + // Not a url every client reads alike. + format!("{base}/patch/pypi/urllib3/1.26.18/old/u2/ {leaf}"), + ] { + assert!(!same_hosted_base(&own, &mirrored), "{own}"); + } + // Served from the root of its host, or in no served shape: the + // origin and the file name decide, as before. + for current in [ + format!("https://patch.socket.dev/patch/pypi/urllib3/1.26.18/tok/u/{leaf}"), + format!("http://127.0.0.1:4000/files/{leaf}"), + "https://patch.socket.dev/gopatch/u".to_string(), + ] { + assert_eq!(hosted_base_path(¤t), None, "{current}"); + assert!(same_hosted_base("https://anything.example/x", ¤t)); + } + assert_eq!( + hosted_base_path("https://repo.example/m/gopatch/u").as_deref(), + Some("/m/") + ); + } } diff --git a/crates/socket-patch-core/src/patch/redirect/grant.rs b/crates/socket-patch-core/src/patch/redirect/grant.rs index da51253e9..f6adfd600 100644 --- a/crates/socket-patch-core/src/patch/redirect/grant.rs +++ b/crates/socket-patch-core/src/patch/redirect/grant.rs @@ -98,7 +98,7 @@ fn is_berry_checksum(s: &str) -> bool { /// Ruby and `.npmrc`), `<` `>` `&` (XML), `#` (a fragment: the locks append /// their own `#` / `#sha256=`), and `;` `|` `^`. Whitespace, control /// characters and anything outside ASCII are never URL characters. -fn is_url_char(c: char) -> bool { +pub(crate) fn is_url_char(c: char) -> bool { c.is_ascii_alphanumeric() || "-._~:/?[]@!()*+,=%".contains(c) } diff --git a/crates/socket-patch-core/src/patch/redirect/mirror.rs b/crates/socket-patch-core/src/patch/redirect/mirror.rs new file mode 100644 index 000000000..a54424ddd --- /dev/null +++ b/crates/socket-patch-core/src/patch/redirect/mirror.rs @@ -0,0 +1,222 @@ +//! A mirror of Socket's patch server: the base URL of a caching proxy the +//! operator runs in front of it (an Artifactory or Nexus remote repository, +//! a reverse proxy), given with `--patch-mirror-url`. +//! +//! The mirror serves the patch server's own paths under its base, so a +//! hosted reference moves onto it by swapping everything before the path: +//! +//! ```text +//! https://patch.socket.dev/patch/npm/left-pad/1.3.0///left-pad-1.3.0.tgz +//! https://repo.example/artifactory/socket-patch/patch/npm/left-pad/1.3.0///left-pad-1.3.0.tgz +//! ``` +//! +//! The path, and with it the grant token and the patch uuid, is unchanged, +//! which is what lets every reader recover the patch from a mirrored +//! reference exactly as from a direct one (`crate::origin`), once the +//! mirror's origin is one of the run's configured patch-server origins. +//! +//! The base is operator configuration (a flag or an environment variable), +//! never repository content: it names where every later install fetches +//! from, and which host's references `vex` attests. + +/// The base URL of a patch-server mirror, validated and without a trailing +/// slash. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct PatchMirror { + base: String, +} + +/// Bound on the base: a mirrored URL is the base plus a served path, and +/// the whole must stay inside the grant gate's URL bound. +const MAX_BASE: usize = 1024; + +impl PatchMirror { + /// Parse the operator's base URL. It is spliced in front of served + /// paths and written into files the project commits, so it is held to + /// the same character set as a grant's URLs (`grant::is_url_char`) and + /// may carry neither credentials, a query nor a fragment: a lockfile is + /// no place for a password, and a query or fragment would end up in the + /// middle of the mirrored URL. + pub fn parse(raw: &str) -> Result { + let trimmed = raw.trim(); + let base = trimmed.trim_end_matches('/'); + if base.is_empty() { + return Err("it is empty".to_string()); + } + if base.len() > MAX_BASE { + return Err(format!("it is longer than {MAX_BASE} bytes")); + } + if !(base.starts_with("https://") || base.starts_with("http://")) { + return Err("it is not an http(s) URL".to_string()); + } + if !base.chars().all(super::grant::is_url_char) { + return Err( + "it holds a character that cannot be written into a lockfile URL".to_string(), + ); + } + if base.contains('?') { + return Err("it carries a query string".to_string()); + } + // A percent-encoded `/` or `\` in the base would end up in every + // mirrored URL, and no reader takes such a URL as a hosted + // reference (`origin::hosted_patch_uuid`). + let lower = base.to_ascii_lowercase(); + if lower.contains("%2f") || lower.contains("%5c") { + return Err("it holds a percent-encoded slash".to_string()); + } + // The authority as written: the URL parser skips extra slashes + // after an `http(s)` scheme, so `https:///x` would name host `x`. + let authority = base + .split_once("://") + .map_or("", |(_, rest)| rest.split('/').next().unwrap_or("")); + let parsed = reqwest::Url::parse(base).map_err(|_| "it is not a valid URL".to_string())?; + if authority.is_empty() || parsed.host_str().is_none_or(str::is_empty) { + return Err("it names no host".to_string()); + } + if authority.contains('@') || !parsed.username().is_empty() || parsed.password().is_some() { + return Err( + "it carries credentials; a mirrored URL is committed with the lockfile".to_string(), + ); + } + // The parser reads `\` as `/` and drops tabs; the character check + // above already refused both, so the text is what the parser saw. + Ok(PatchMirror { + base: base.to_string(), + }) + } + + /// The base URL, without a trailing slash. + pub fn base(&self) -> &str { + &self.base + } + + /// `url` moved onto the mirror: the base in place of its scheme and + /// authority, its path (and query) byte for byte. `None` for anything + /// but an `http(s)` URL with a path. + pub fn rebase(&self, url: &str) -> Option { + let rest = url + .strip_prefix("https://") + .or_else(|| url.strip_prefix("http://"))?; + let path_at = rest.find('/')?; + let (authority, path) = rest.split_at(path_at); + if authority.is_empty() || path.len() < 2 { + return None; + } + Some(format!("{}{path}", self.base)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const SERVED: &str = "https://patch.socket.dev/patch/npm/@scope/pkg/1.0.0/\ + 11111111-1111-4111-8111-111111111111/22222222-2222-4222-8222-222222222222/pkg-1.0.0.tgz"; + + #[test] + fn a_base_is_kept_without_its_trailing_slashes() { + let mirror = PatchMirror::parse(" https://repo.example/artifactory/socket-patch// ") + .expect("valid base"); + assert_eq!( + mirror.base(), + "https://repo.example/artifactory/socket-patch" + ); + // A bare origin is a base too (a reverse proxy at the root). + assert_eq!( + PatchMirror::parse("http://127.0.0.1:8081/") + .expect("valid") + .base(), + "http://127.0.0.1:8081" + ); + } + + #[test] + fn a_base_that_cannot_prefix_a_committed_url_is_refused() { + for (raw, why) in [ + ("", "empty"), + (" ", "empty"), + ("repo.example/artifactory", "http(s)"), + ("ftp://repo.example/x", "http(s)"), + ("https://user:secret@repo.example/x", "credentials"), + ("https://user@repo.example/x", "credentials"), + ("https://@repo.example/x", "credentials"), + ("https://repo.example/a%2Fb", "percent-encoded"), + ("https://repo.example/a%5cb", "percent-encoded"), + ("https://repo.example/x?repo=y", "query"), + ("https://repo.example/x#frag", "character"), + ("https://repo.example/a b", "character"), + ("https://repo.example/${HOME}", "character"), + ("https://repo.example/\"x", "character"), + ("https:///x", "host"), + ] { + let err = PatchMirror::parse(raw).expect_err(raw); + assert!(err.contains(why), "{raw:?}: {err}"); + } + assert!(PatchMirror::parse(&format!("https://h/{}", "a".repeat(MAX_BASE))).is_err()); + } + + #[test] + fn rebase_swaps_everything_before_the_path_and_nothing_after_it() { + let mirror = PatchMirror::parse("https://repo.example/artifactory/socket-patch").unwrap(); + assert_eq!( + mirror.rebase(SERVED).as_deref(), + Some( + "https://repo.example/artifactory/socket-patch/patch/npm/@scope/pkg/1.0.0/\ + 11111111-1111-4111-8111-111111111111/22222222-2222-4222-8222-222222222222/\ + pkg-1.0.0.tgz" + ) + ); + // The path is copied, never re-encoded; a port and a plain-http + // served origin make no difference. + assert_eq!( + mirror + .rebase("http://127.0.0.1:4026/patch-registry/gem/t/u/info/%40x?y=1") + .as_deref(), + Some("https://repo.example/artifactory/socket-patch/patch-registry/gem/t/u/info/%40x?y=1") + ); + // A registry base with a trailing slash keeps it. + assert_eq!( + mirror + .rebase("https://patch.socket.dev/patch-registry/gem/t/u/") + .as_deref(), + Some("https://repo.example/artifactory/socket-patch/patch-registry/gem/t/u/") + ); + } + + #[test] + fn rebase_refuses_what_is_not_a_served_url() { + let mirror = PatchMirror::parse("https://repo.example/m").unwrap(); + for url in [ + "", + "patch.socket.dev/patch/npm/x", + "sparse+https://patch.socket.dev/patch-registry/cargo/t/u/index/", + "https://patch.socket.dev", + "https://patch.socket.dev/", + "https:///patch/npm/x", + "file:.socket/vendor/npm/u/x-1.0.0.tgz", + ] { + assert_eq!(mirror.rebase(url), None, "{url:?}"); + } + } + + #[test] + fn a_rebased_reference_names_the_same_patch_on_the_mirror_origin() { + let mirror = PatchMirror::parse("https://repo.example/artifactory/socket-patch").unwrap(); + let mirrored = mirror.rebase(SERVED).unwrap(); + let origins = [mirror.base().to_string()]; + assert_eq!( + crate::origin::hosted_patch_uuid(&mirrored, &origins).as_deref(), + Some("22222222-2222-4222-8222-222222222222") + ); + // Without the mirror among the run's origins it is no reference. + assert_eq!(crate::origin::hosted_patch_uuid(&mirrored, &[]), None); + assert_eq!( + super::super::grant_token_path_segment( + &mirrored, + "22222222-2222-4222-8222-222222222222" + ) + .as_deref(), + Some("11111111-1111-4111-8111-111111111111") + ); + } +} diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 61cea1b9c..350328ccf 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -51,6 +51,7 @@ mod group_equivalence_tests; #[cfg(test)] mod lock_index_equivalence_tests; mod maven_index; +pub mod mirror; pub mod npm_allow_scripts; pub mod npmrc; mod pdm; @@ -5983,7 +5984,10 @@ const BERRY_MANIFEST: &str = "package.json"; /// this run's own artifact URL, or a URL on the same patch server (scheme, /// host and port of `artifact_url`) whose leaf names the package version. /// A user's own tarball for the package — a mirror, a fork — is on another -/// origin and is never ours to rewrite. +/// origin and is never ours to rewrite. When `artifact_url` is served under +/// a patch mirror's base, whose host serves the project's own packages +/// too, the URL must also sit under that base +/// ([`crate::origin::same_hosted_base`]). fn berry_hosted_pin_is_ours( url: &str, name: &str, @@ -6006,7 +6010,7 @@ fn berry_hosted_pin_is_ours( } _ => false, }; - names_it && same_origin + names_it && same_origin && crate::origin::same_hosted_base(url, artifact_url) } /// Whether `value` (a `resolutions` value) is a hosted tarball pin written @@ -6689,6 +6693,9 @@ fn rewrite_bun_lock( /// override — nothing about the patch server's URL layout is assumed — and /// anything that fails to parse fails the match (closed): user URL deps live /// on other origins, and another version's artifact has a different leaf. +/// Under a patch mirror, whose host serves the project's own packages too, +/// the two must also share the mirror's base +/// ([`crate::origin::same_hosted_base`]). pub(crate) fn is_prior_hosted_bun_spec(spec: &str, fname: &str, current_url: &str) -> bool { let Some(old_url) = spec .strip_prefix(fname) @@ -6712,7 +6719,9 @@ pub(crate) fn is_prior_hosted_bun_spec(spec: &str, fname: &str, current_url: &st Some((&url[..path_start], leaf)) } match (origin_and_leaf(old_url), origin_and_leaf(current_url)) { - (Some(old), Some(new)) => old == new, + (Some(old), Some(new)) => { + old == new && crate::origin::same_hosted_base(old_url, current_url) + } _ => false, } } @@ -15987,6 +15996,60 @@ mod tests { assert!(fresh.confirmed_yarn_berry_uuids.is_empty()); } + /// Under a patch mirror, whose host serves the project's own packages + /// too, yarn berry and Bun take over an earlier pin only under the + /// mirror's base: a tarball of the same name and version in another + /// repository of that host is the project's own source. + #[test] + fn a_tarball_elsewhere_on_the_mirrors_host_is_not_an_earlier_pin() { + let base = "https://repo.example/artifactory/socket-patch"; + let leaf = "left-pad-1.3.0.tgz"; + let current = format!("{base}/patch/npm/left-pad/1.3.0/new/u2/{leaf}"); + let earlier = format!("{base}/patch/npm/left-pad/1.3.0/old/u1/{leaf}"); + let fork = format!("https://repo.example/artifactory/npm-local/left-pad/-/{leaf}"); + + assert!(berry_hosted_pin_is_ours( + &earlier, + "left-pad", + Some("1.3.0"), + ¤t + )); + assert!(berry_hosted_pin_is_ours( + &earlier, "left-pad", None, ¤t + )); + assert!(!berry_hosted_pin_is_ours( + &fork, + "left-pad", + Some("1.3.0"), + ¤t + )); + assert!(!berry_hosted_pin_is_ours(&fork, "left-pad", None, ¤t)); + + let spec = |url: &str| format!("left-pad@{url}"); + assert!(is_prior_hosted_bun_spec( + &spec(&earlier), + "left-pad", + ¤t + )); + assert!(!is_prior_hosted_bun_spec( + &spec(&fork), + "left-pad", + ¤t + )); + + // On the patch server itself the origin and the file name decide, + // as they always have: nothing else is served from that host. + let direct = current.replace(base, "https://patch.socket.dev"); + let stale = earlier.replace(base, "https://patch.socket.dev"); + assert!(berry_hosted_pin_is_ours( + &stale, + "left-pad", + Some("1.3.0"), + &direct + )); + assert!(is_prior_hosted_bun_spec(&spec(&stale), "left-pad", &direct)); + } + /// A lock written by an earlier release carries the old /// `npm:…::__archiveUrl=` pin; a repeat hosted run re-pins it (#404). #[test] diff --git a/crates/socket-patch-core/src/utils/process.rs b/crates/socket-patch-core/src/utils/process.rs index 036617486..5de62a0b7 100644 --- a/crates/socket-patch-core/src/utils/process.rs +++ b/crates/socket-patch-core/src/utils/process.rs @@ -64,6 +64,10 @@ pub const CHILD_ENV_SCRUB: &[&str] = &[ // proxy URL. "SOCKET_CLI_CONFIG", "SOCKET_CLI_API_PROXY", + // The operator's credential for their patch mirror + // (`--patch-mirror-url`): only this CLI's own check of the mirror + // sends it. + "SOCKET_PATCH_MIRROR_AUTHORIZATION", // The other accepted socket-cli aliases (`env_compat::PEER_ENV_ALIASES`) // and the canonical name of the one that is not a flag. "SOCKET_CLI_ORG_SLUG", @@ -80,6 +84,7 @@ pub const CHILD_ENV_SCRUB: &[&str] = &[ "SOCKET_VENDOR_SOURCE", "SOCKET_VENDOR_URL", "SOCKET_PATCH_SERVER_URL", + "SOCKET_PATCH_MIRROR_URL", "SOCKET_OFFLINE", "SOCKET_STRICT", "SOCKET_GLOBAL", diff --git a/crates/socket-patch-core/src/utils/python_lock.rs b/crates/socket-patch-core/src/utils/python_lock.rs index abd4c4c52..6313477fd 100644 --- a/crates/socket-patch-core/src/utils/python_lock.rs +++ b/crates/socket-patch-core/src/utils/python_lock.rs @@ -419,7 +419,10 @@ fn pair_lines(old: &[(&str, &str)], new: &[(&str, &str)]) -> Vec> /// current artifact URL, fragments ignored. Grant tokens and patch uuids live /// in the path between them, so a rotated token or a superseded patch (new /// uuid) takes over the stale pin in place instead of being refused as a -/// foreign source or stranding it on a URL that no longer serves. +/// foreign source or stranding it on a URL that no longer serves. Under a +/// patch mirror the two must also share the mirror's base +/// ([`crate::origin::same_hosted_base`]): the mirror's host serves the +/// project's own packages too. pub(crate) fn is_prior_hosted_url(existing: &str, current: &str) -> bool { fn origin_and_leaf(url: &str) -> Option<(&str, &str)> { if !url.starts_with("https://") && !url.starts_with("http://") { @@ -435,7 +438,7 @@ pub(crate) fn is_prior_hosted_url(existing: &str, current: &str) -> bool { Some((&url[..path_start], leaf)) } match (origin_and_leaf(existing), origin_and_leaf(current)) { - (Some(old), Some(new)) => old == new, + (Some(old), Some(new)) => old == new && crate::origin::same_hosted_base(existing, current), _ => false, } } @@ -1352,6 +1355,34 @@ pub fn paired_metadata_path(lock: &str) -> Option { #[cfg(test)] mod tests { + /// An earlier hosted pin is taken over in place; a url that only + /// shares the host and the file name of a MIRRORED artifact is the + /// project's own source and is not. + #[test] + fn an_earlier_pin_is_ours_only_under_the_base_it_is_served_from() { + let leaf = "urllib3-1.26.18-py2.py3-none-any.whl"; + let direct = format!("https://patch.socket.dev/patch/pypi/urllib3/1.26.18/new/u2/{leaf}"); + let stale = + format!("https://patch.socket.dev/patch/pypi/urllib3/1.26.18/old/u1/{leaf}#sha256=0"); + assert!(super::is_prior_hosted_url(&stale, &direct)); + assert!(!super::is_prior_hosted_url( + &format!("https://files.example/{leaf}"), + &direct + )); + + let base = "https://repo.example/artifactory/socket-patch"; + let mirrored = format!("{base}/patch/pypi/urllib3/1.26.18/new/u2/{leaf}"); + assert!(super::is_prior_hosted_url( + &format!("{base}/patch/pypi/urllib3/1.26.18/old/u1/{leaf}"), + &mirrored + )); + // The project's own fork, in another repository of the same host. + assert!(!super::is_prior_hosted_url( + &format!("https://repo.example/artifactory/pypi-local/{leaf}"), + &mirrored + )); + } + use super::*; /// An entry the rewrite does not apply to (a non-registry, non-direct diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 1588246fc..8df5f54f7 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -3142,24 +3142,83 @@ async fn foreign_hosted_url_in_files( let Ok(text) = crate::utils::fs::read_regular_to_string(&root.join(&file)).await else { continue; }; - for token in url_tokens_around(&text, uuid) { - let schemed; - let url = if token.contains("://") || token.contains("%3A%2F%2F") { - token - } else { - schemed = format!("https://{token}"); - &schemed - }; - if crate::origin::hosted_patch_uuid(url, origins).as_deref() != Some(uuid) - && crate::origin::hosted_shape_uuid(url).as_deref() == Some(uuid) - { - return true; - } + if foreign_hosted_host_in_text(&text, uuid, origins).is_some() { + return true; } } false } +/// The host of the first url in `text` that has the hosted shape for `uuid` +/// on an origin that is neither Socket's patch server nor one of `origins` +/// (the test [`foreign_hosted_url_in_files`] applies to each file): where a +/// pin of that patch routes to when this run was not given its host — a +/// patch mirror or a staging server left out of the run's settings, or a +/// look-alike. `None` when every url naming the uuid is on an accepted +/// origin, or names it in no hosted shape. +pub fn foreign_hosted_host_in_text(text: &str, uuid: &str, origins: &[String]) -> Option { + if !is_canonical_uuid(uuid) || !text.contains(uuid) { + return None; + } + for token in url_tokens_around(text, uuid) { + // An older composer.lock escapes every slash (`https:\/\/…`): the + // url is the unescaped text. Read escaped, it has no `://`, and the + // scheme-less branch below would take `https` for its host. + let token = token.replace("\\/", "/"); + // A lock may spell the url inside a specifier (`name@https://…`, as + // bun and yarn berry key a tarball dependency): the url starts at + // its scheme. + let token = from_scheme(&token); + let schemed; + let url = if token.contains("://") || token.contains("%3A%2F%2F") { + token + } else { + schemed = format!("https://{token}"); + &schemed + }; + if crate::origin::hosted_patch_uuid(url, origins).as_deref() != Some(uuid) + && crate::origin::hosted_shape_uuid(url).as_deref() == Some(uuid) + { + return Some( + crate::origin::hosted_shape_host(url).unwrap_or_else(|| "another host".to_string()), + ); + } + } + None +} + +/// The host of the first url in `text` that is a hosted reference to `uuid` +/// on Socket's patch server or one of `origins`: where the project pins +/// that patch, when it pins it on a host the caller names. `None` when no +/// url naming the uuid is one. +pub fn hosted_host_in_text(text: &str, uuid: &str, origins: &[String]) -> Option { + if !is_canonical_uuid(uuid) || !text.contains(uuid) { + return None; + } + url_tokens_around(text, uuid) + .into_iter() + .map(|token| token.replace("\\/", "/")) + .map(|token| from_scheme(&token).to_string()) + .filter(|token| token.contains("://") || token.contains("%3A%2F%2F")) + .find(|url| crate::origin::hosted_patch_uuid(url, origins).as_deref() == Some(uuid)) + .map(|url| { + crate::origin::hosted_shape_host(&url).unwrap_or_else(|| "the patch server".to_string()) + }) +} + +/// `token` from the scheme of the url it holds (`name@https://host/…` is +/// `https://host/…`; a Cargo `sparse+https://…` keeps its source kind), or +/// `token` itself when it holds no `://`. +fn from_scheme(token: &str) -> &str { + let Some(at) = token.find("://") else { + return token; + }; + let start = token[..at] + .rfind(|c: char| !(c.is_ascii_alphanumeric() || matches!(c, '+' | '.' | '-'))) + .map_or(0, |i| i + 1); + &token[start..] +} + /// The url-shaped token around each occurrence of `needle` in `text`: /// widened in both directions up to whitespace, a quote, markup or a /// key / binding separator (`=`, `&`, `,`, brackets). diff --git a/docs/configuration.md b/docs/configuration.md index 5c25ead05..3dbc6db43 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -55,6 +55,14 @@ Empty environment values are treated as unset. bundle on Linux). `SSL_CERT_FILE` / `SSL_CERT_DIR` replace the system store with the named bundle or directory, which is how to trust a TLS-inspecting proxy's private CA. Certificate verification is never disabled. +- `SOCKET_PATCH_MIRROR_URL` (experimental) names a mirror of Socket's patch + server, such as an Artifactory remote repository, for builds that may only + fetch from an internal host. Hosted mode then pins patches under that base, + and every command reads the pins there. Set it wherever Socket Patch runs + for the project; a repository file cannot set it. It is for Socket + Enterprise customers and may change in any release: ask Socket for the + setup guide. The rules are in the + [CLI contract](../crates/socket-patch-cli/CLI_CONTRACT.md#patch-mirror-experimental). - `--offline` prohibits network access. `scan` and `get` require the patch API and refuse offline operation. Other commands can use locally available state; missing records or artifacts can still prevent completion.