diff --git a/.agents/docs/2026-11-20-test-architecture-upgrade-design.md b/.agents/docs/2026-11-20-test-architecture-upgrade-design.md index c4d84fc29..02c556dc0 100644 --- a/.agents/docs/2026-11-20-test-architecture-upgrade-design.md +++ b/.agents/docs/2026-11-20-test-architecture-upgrade-design.md @@ -985,6 +985,27 @@ doctest 适配器(5.14)、依赖的调试信息(5.15)。boost-m 自己 给出(`take_refused`),`other` 同时记入面的记录。拒绝 `system`、工具链命令的参数错误、`self config` 与 `doctor` 的提示写死 gcc 16.1.0;改为读取内建表中本机的首次运行,`doctor` 的提示改为 `mcpp toolchain install --target `。 +17. **发布之后:命令的时限与 `pr` 档的门禁(运行 44937152470,`pr` 档的两个分片)。** `sets/985` 为每个系统打包的一步在第 10 分钟被 + 结束,所在套件陈述 45 分钟而远未用尽;同一分片的 `sets`、`images`、`stack` 同时运行,`sets` 用时 2140 s,三个分片 + 的运行中为 1162 s。原因是 5.8 的一项没有实现:mtest 0.11.1 使命令不超过用例余下的时间,kit 却仍以十分钟为每条命令 + 的缺省,两者取小。此前 1033 与 955 各自把一条命令提高到 30 分钟,是同一缺口的两次绕行。kit 改为:命令不陈述时限时取 + 用例余下的时间(`time_left()`),没有任何作用域陈述时限时取十分钟;两处提高删除,`images` 按测量陈述套件的时间 + (30 分钟;两个分片中 1102 s,三个分片中 707 s,运行的缺省为 1200 s)。 + 同一运行的门禁另有一项与改动无关的失败:面的账本列出 14 项没有通过的用例,均只由 `full` 档的作业或用例用到(Windows + 的 `coff-def`、构建图内部调用的 `dyndep --split-module` 与 `link-output --aside`、只在某些行上到达的原因标记)。需求 + 由运行组中通过的用例满足,`pr` 档只运行改动影响的作业与用例,于是每次 `pr` 运行的门禁都失败。账本是全部用例的性质, + 改由运行全部用例的集合判定:mtest 0.11.3 的门禁只在这样的集合中判定账本,其他集合判定失败与预算并注明未判定账本; + `ci.yml` 以运行的集合调用门禁(`gate --set`)。新增面的项的拉取请求以 `ci:full` 确认它有通过的用例(贡献规程)。 +18. **发布之后:单元分片的数目(main 的运行 44945453323,`full` 档的两个分片)。** 单元分片的数目此前取 main 上一次 + 运行中各程序用时之和,除以一个分片的四个工作进程在 30 分钟内的工作量。三个分片的运行 44930053323(三个分片分别用时 + 1236、834、1438 s)由此得出两个分片;两个分片分别用时 2544 与 2432 s,`trees` 用尽它陈述的 40 分钟(三个分片中为 + 1369 s),`sets`、`images`、`stack` 在同一分片中同时运行;本拉取请求的 `pr` 运行同为两个分片,用时 2099 与 + 1429 s,`trees` 为 1981 s,两次之差在 runner 的处理器之间(5.9)。用时之和除以工作进程数,假定四个工作进程始终各有程序 + 可运行,且程序的用时不随同时运行的程序变化;openkal 的程序在机器的一份预算中并行构建,分片越少,同时运行的越多, + 各自越慢,一个分片的时间由其中最长的程序与同时运行的程序决定。分片数改为取 main 上一次运行的分片数 N 与其中最长 + 分片的用时 W,N × W 按每片 30 分钟折算(`ci-unit.yml`;两次运行都得出三个分片)。N 与 W 从各分片作业保存的存储 + 读出:作业的存储经导入保存为 mtest 的运行记录,其中没有分片;历史中另有子系统作业的运行。套件陈述的时间不变:它们界定 + 挂起,在三个分片中留有余量。 ## 15. 子方案 diff --git a/.agents/skills/mcpp-contributing/SKILL.md b/.agents/skills/mcpp-contributing/SKILL.md index 2b52343da..19836a49b 100644 --- a/.agents/skills/mcpp-contributing/SKILL.md +++ b/.agents/skills/mcpp-contributing/SKILL.md @@ -214,6 +214,10 @@ gh run view --log-failed # 查看失败日志 直接显示在 PR 的检查中。不要另外以 `gh workflow run ci.yml --ref <分支>` 启动一次运行,也不要取消 PR 的运行: 分支上的运行不出现在 PR 的检查中,两次运行还会同时占用 runner。 +面的账本(`mcpp emit surface` 的每一项由通过的用例用到或豁免)是全部用例的性质,只由 `full` 的门禁判定;`pr` 的门禁 +判定失败与预算,并注明没有判定账本(mtest 0.11.3)。新增命令、选项、清单键或原因标记的 PR 以 `ci:full` 确认它有 +通过的用例。 + ```bash gh pr edit --add-label ci:full # 先打标签 git push # 再推送:这次运行跑 full diff --git a/.github/versions.env b/.github/versions.env index 6047f7855..62c356a56 100644 --- a/.github/versions.env +++ b/.github/versions.env @@ -1,5 +1,5 @@ # The engines CI pins, read by the composite actions (.github/actions/mtest). # mtest: an xlings package (xim:mtest) whose sha256 the index states; the release # bundle's sha256 checks the fallback download. -MTEST_VERSION=0.11.2 -MTEST_SHA256=88144bacd76ff3af3957b29560e55af813906db2f682f8cefae1869a2be74de9 +MTEST_VERSION=0.11.3 +MTEST_SHA256=29e597e2177589dc06f0f9064f86b71198200b425acfa168b8dffa0c259885db diff --git a/.github/workflows/ci-unit.yml b/.github/workflows/ci-unit.yml index c512022f4..eaeb97c6f 100644 --- a/.github/workflows/ci-unit.yml +++ b/.github/workflows/ci-unit.yml @@ -85,7 +85,7 @@ on: description: the number of tests the archive holds value: ${{ jobs.archive.outputs.archived }} shards: - description: the run stage's shards, from the measured work of main's last run + description: the run stage's shards, from the shards of main's last run and the longest of them value: ${{ jobs.archive.outputs.shards }} env: @@ -198,27 +198,46 @@ jobs: fi python3 .github/tools/test_history.py take "$RUNNER_TEMP/unit/history" "$d" "$d"/*/ - # THE SHARDS FROM THE PLAN (design 2026-11-20 §5.17, CI4): the measured work of main's last - # run, its shards' programs' durations summed (the history holds that run's shards alone, - # test_history.py), over what one shard does in 30 minutes with its four workers; at least - # one, at most eight. With no history, four, as before. + # THE SHARDS FROM THE PLAN (design 2026-11-20 §5.17, CI4): main's last run took N shards and + # the longest of them W, so the run is N × W of shard time; this one takes as many shards as + # that is in 30 minutes each, at least one, at most eight. N and W are read from the stores + # the shard jobs kept (`mtest-unit---a`, each holding that shard's + # runs, test_history.py): the store's import (test-report) keeps a run in the form of + # mtest's records, which state no shard, and the subsystems' store beside them is another + # job's. A shard's time is that of its longest programs and of those that + # run beside one another, not its programs' durations over four workers: the openkal + # programs build in parallel within the machine's one budget, and in fewer shards they run + # beside one another and each takes longer. A run of three shards (1236, 834 and 1438 s) + # gave two by its durations summed, and the two took 2544 and 2432 s, `trees` 1369 s in the + # first run and past the 40 minutes it states in the second (CI, 2026-11-21). With no + # history of shards, four. - name: The run stage's shards id: shards shell: bash + env: + KEY: ${{ inputs.name }} run: | set -uo pipefail - work=$(find "$RUNNER_TEMP/unit/history" -name programs.ndjson -print0 2>/dev/null \ - | xargs -0 -r cat | jq -s 'map(.ms // 0) | add // 0') - if [ -z "$work" ] || [ "${work%.*}" = 0 ]; then - echo 'shards=[1,2,3,4]' >> "$GITHUB_OUTPUT"; echo "no measured work: four shards"; exit 0 + # A re-run's store (`-a2`) holds the attempt before it and its `--failed` run: the + # shards are counted by number, and the longest run is the whole shard's. + was=0; longest=0 + for s in "$RUNNER_TEMP"/unit-stores/mtest-unit-"$KEY"-*-a*/; do + [[ "$(basename "$s")" =~ ^mtest-unit-"$KEY"-([0-9]+)-a[0-9]+$ ]] || continue + [ -f "$s/index.ndjson" ] || continue + [ "${BASH_REMATCH[1]}" -le "$was" ] || was=${BASH_REMATCH[1]} + ms=$(jq -s 'map(.ms // 0) | max // 0 | floor' "$s/index.ndjson" 2>/dev/null || echo 0) + [ "${ms:-0}" -le "$longest" ] || longest=$ms + done + if [ "$was" -le 0 ] || [ "$longest" -le 0 ]; then + echo 'shards=[1,2,3,4]' >> "$GITHUB_OUTPUT"; echo "no measured shards: four shards"; exit 0 fi - per=$((30 * 60 * 1000 * 4)) - n=$(( (${work%.*} + per - 1) / per )) + per=$((30 * 60 * 1000)) + n=$(( (was * longest + per - 1) / per )) [ "$n" -ge 1 ] || n=1 [ "$n" -le 8 ] || n=8 list=$(seq -s, 1 "$n") echo "shards=[$list]" >> "$GITHUB_OUTPUT" - echo "measured work ${work%.*} ms: $n shard(s)" + echo "main's last run: $was shard(s), the longest $((longest / 1000)) s: $n shard(s)" - name: Upload the tests and their history if: steps.archive.outputs.archived != '0' diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1a152ee2a..8f0e2fe5c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -400,9 +400,10 @@ jobs: name: linux runs-on: ubuntu-24.04 archived: ${{ needs.unit-linux-archive.outputs.archived }} - # The shards the archive stage planned from main's measured work (four - # without a history): the measured run of 2026-11-13 reached 65 of its - # programs, 3970 s of work, in the 45 minutes of one job of four workers. + # The shards the archive stage planned from main's last run, its shards and + # the longest of them (four without a history): the measured run of + # 2026-11-13 reached 65 of its programs, 3970 s of work, in the 45 minutes + # of one job of four workers. shards: ${{ needs.unit-linux-archive.outputs.shards || '[1,2,3,4]' }} form: ${{ needs.release-form.result == 'success' && 'release' || 'host' }} # The toolchains whose capabilities the cases probe: build-linux's `prewarm`. @@ -577,13 +578,17 @@ jobs: --sections summary,failures,slowest,builds,steps,margin,budget,ledgers,flaky,path # THE GATE OVER THE RUN'S GROUP (design 2026-11-20 §5.16, §5.17): no case over the budget # without a waiver, every item of the surface used by a passing case or waived, and no waiver - # that excuses nothing. + # that excuses nothing. The run's set is the gate's: the surface is a ledger of every case, and + # `pr` runs the jobs and cases a change affects, so its gate judges the budget and notes that + # it did not judge the ledger (mtest 0.11.3); a `full` run judges both. - name: The gate over the run's group shell: bash + env: + SET: ${{ needs.changes.outputs.set }} run: | set -euo pipefail cd tests/e2e - "$MTEST" gate + "$MTEST" gate --set "$SET" - uses: actions/upload-artifact@v4 if: always() with: diff --git a/tests/e2e/openkal/images.cpp b/tests/e2e/openkal/images.cpp index 4b3d6cb2f..e70b5658c 100644 --- a/tests/e2e/openkal/images.cpp +++ b/tests/e2e/openkal/images.cpp @@ -163,7 +163,10 @@ __attribute__((no_stack_protector)) int ok_main(void) { constexpr std::string_view caught_main = "import std;\nint main(int argc, char** argv) {\n try { throw std::runtime_error(\"caught\"); }\n catch (const std::exception& e) { std::println(\"image: {} ({} args)\", e.what(), argc); }\n return 0;\n}\n"; constexpr std::string_view smoke_test = "import std;\nint main() { std::vector v{2, 1}; std::ranges::sort(v); return v.front() == 1 ? 0 : 1; }\n"; -E2E images{"images", s::home_fixture, [] { +// The ok images of this architecture and the runtime each first run builds: 707 s in a Linux shard +// of three, 1102 s in a shard of two, of the run's 1200 (CI, 2026-11-21); stated, so the run's +// limit is not raised for every test. +E2E images{"images", s::home_fixture, timeout{std::chrono::minutes{30}}, [] { "942 an ok image is built once and run by okrun"_test[s::llvm][s::python3] = [] { // A both rows publish bin/app.oke, an ET_DYN of their architecture, and no unchecked // image; B okrun runs this machine's image with its arguments, environment, a callback, @@ -512,9 +515,9 @@ int main(int argc, char** argv) { {"tests/smoke.cpp", "import std;\nint main() { return std::format(\"{}-{}\", 1, 2) == \"1-2\" ? 0 : 1; }\n"}}); const std::string ht = std::string(host_arch) + "-openkal"; // The first run builds the platform's runtime (1,636 edges): beyond ten minutes on a - // four-processor Windows runner whose other tests share the run's budget (CI, 2026-11-21). - auto r = app.mcpp({"run", "--target", ht, "--", "one", "two words"}, - {.env = {{"MCPP_E2E_1033", "here"}}, .timeout = std::chrono::minutes{30}}); + // four-processor Windows runner whose other tests share the run's budget (CI, 2026-11-21), + // within the suite's time. + auto r = app.mcpp({"run", "--target", ht, "--", "one", "two words"}, {.env = {{"MCPP_E2E_1033", "here"}}}); expect(fatal(e2e::succeeded(r))) << "A: mcpp run --target " << ht << ": " << s::text(r); expect(fatal(e2e::contains(s::text(r), "args=[one][two words] env=here sum=55"))) << "A: the image's output: " << s::text(r); r = app.mcpp({"test", "--target", ht}); diff --git a/tests/e2e/openkal/trees.cpp b/tests/e2e/openkal/trees.cpp index dea080acf..da877d477 100644 --- a/tests/e2e/openkal/trees.cpp +++ b/tests/e2e/openkal/trees.cpp @@ -117,8 +117,9 @@ E2E trees{"trees", s::home_fixture, timeout{std::chrono::minutes{40}}, [] { s::project app({{"mcpp.toml", "[package]\nname = \"uni\"\nversion = \"0.1.0\"\n"}, {"src/main.cpp", "import std;\nint main(int argc, char** argv) {\n try { throw std::runtime_error(\"universal\"); }\n catch (const std::exception& e) { std::println(\"{} {} args\", e.what(), argc); }\n return 0;\n}\n"}}); // A. The first pack builds the openkal set for both architectures: beyond ten minutes on a - // four-processor Linux runner whose other tests share the run's budget (CI, 2026-11-21). - auto r = app.mcpp({"pack", "--target", "openkal", "--pkg-format", "dir"}, {.timeout = std::chrono::minutes{30}}); + // four-processor Linux runner whose other tests share the run's budget (CI, 2026-11-21), + // within the suite's time. + auto r = app.mcpp({"pack", "--target", "openkal", "--pkg-format", "dir"}); expect(fatal(e2e::succeeded(r))) << "A: pack: " << s::text(r); const auto p = app.at("target/dist/uni-0.1.0-openkal"); for (std::string_view f : {"lib/x86_64/uni.oke", "lib/aarch64/uni.oke", "bin/uni", "bin/uni.exe", "mcpp-package.json"}) diff --git a/tests/e2e/test/protocol.cpp b/tests/e2e/test/protocol.cpp index d8e72f14b..ece756f3f 100644 --- a/tests/e2e/test/protocol.cpp +++ b/tests/e2e/test/protocol.cpp @@ -1,5 +1,6 @@ // Test programs that speak mtest's test protocol: their cases, live case records, the spec's -// launch, capabilities, checks and output (formerly tests/e2e/1035, 1065 and 1070). +// launch, capabilities, checks and output (formerly tests/e2e/1035, 1065 and 1070); run fixtures; +// the time the kit gives a command within its case. import std; import mcpplibs.mtest; import mcpp.test.support; @@ -452,6 +453,23 @@ int main() {{ infra = infra || (p.value("status", "") == "infra" && p["fixtures"] == json::parse(R"(["home"])")); expect(infra) << "no program's record says infra"; }; + + "1141 a command of the kit has the time its case has left"_test[timeout{std::chrono::minutes{45}}] = [] { + // Design 2026-11-20 §5.8 (K1): a command that states no time takes what its case has left + // of the time its scopes state, not ten minutes below it; a command that states one keeps + // it. The case states 45 minutes, so a command composed at its start has more than 44. + const auto cwd = s::case_dir(); + const auto left = s::mcpp_command({"--version"}, cwd).timeout; + expect(fatal(left > std::chrono::minutes{44} && left <= std::chrono::minutes{45})) << "the command's time is " << left.count() << " ms"; + expect(fatal(s::mcpp_command({"--version"}, cwd, {.timeout = std::chrono::seconds{60}}).timeout == std::chrono::seconds{60})); + expect(fatal(e2e::succeeded(s::mcpp({"--version"}, cwd)))); + }; + + "1142 a command of the kit takes ten minutes where no scope states a time"_test = [] { + // Where neither the case nor a scope around it states a time, the runner's limit would end + // the program without the case's report of a hung command: ten minutes bound it. + expect(fatal(s::mcpp_command({"--version"}, s::case_dir()).timeout == std::chrono::minutes{10})); + }; }}; } // namespace diff --git a/tests/support/src/support.cpp b/tests/support/src/support.cpp index 0628176d0..2a111fb6c 100644 --- a/tests/support/src/support.cpp +++ b/tests/support/src/support.cpp @@ -389,7 +389,14 @@ e2e::command mcpp_command(std::vector args, const fs::path& cwd, co c.env.emplace_back("MCPP_VERBOSE", std::nullopt); for (const auto& kv : o.env) c.env.push_back(kv); c.input = o.input; - c.timeout = o.timeout; + // A COMMAND HAS THE TIME ITS CASE HAS LEFT (design 2026-11-20 §5.8, K1). The kit also held each + // command to ten minutes, below the time a suite states: a pack of every system in `sets/985` + // ran past ten minutes beside three other openkal programs on a four-processor runner, its + // suite well within the 45 minutes it states, and failed as a hang (CI, 2026-11-21). A time limit + // bounds a hang; how long work takes is a measure of the run's records, not a verdict. Where no + // scope states a time, ten minutes bounds a command, since the runner's limit would end the + // program without the case's report of it. + c.timeout = o.timeout.value_or(mcpplibs::mtest::time_left().value_or(std::chrono::minutes{10})); return c; } diff --git a/tests/support/src/support.cppm b/tests/support/src/support.cppm index 38614bf3b..2ab5f92cb 100644 --- a/tests/support/src/support.cppm +++ b/tests/support/src/support.cppm @@ -186,7 +186,9 @@ struct options { // Set (a value) or removed (nothing) for the run, after the home. std::vector>> env; std::string input; - std::chrono::milliseconds timeout{std::chrono::minutes{10}}; + // Nothing: what the case has left of the time its scopes state, or ten minutes where none + // states one (mcpp_command). + std::optional timeout; // A program and its arguments mcpp runs under (`unshare -rn`, a network namespace of its own), // so that a command run so is the case's like any other: its use of the surface is recorded. std::vector wrap;