Repository navigation
Run cloud integration tests with -parallel 8 - #7871
Merged
Merged
Conversation
The cloud acceptance package hit its 5h20m go test timeout on all clouds when many main runs overlapped: the tests wait on the workspace, so the package's wall time is its total test time divided by -parallel, and at -parallel 4 the full suite (~19h of test time) needs ~4h50m. Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
✅ Integration test reportCommit: b45c609
Top 3 slowest tests (at least 2 minutes):
|
There was a problem hiding this comment.
Automated review at b45c609. Each finding was verified against the code.
No blocking issues found.
This PR bumps -parallel from 4 to 8 in the integration and integration-short Taskfile tasks to reduce wall-clock time for cloud CI runs that were hitting the 5h20m go test timeout. The change is minimal and mechanical (two token edits plus a clarifying comment), and the code does exactly what it claims. No defects found.
andrewnester
approved these changes
Nov 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Raise
-parallelfrom 4 to 8 in theintegrationandintegration-shorttasks, the ones the cloud CI runs on main.Why
The cloud acceptance package hits the 5h20m
go test -timeout(set in #7749) on aws, azure and gcp, linux and windows, when many main runs overlap. No test hangs. The tests still running at the timeout differ in every failure, and they keep moving through the package (bundle/invariant/, resources/postgres_, volumes, apps, ...). The package is saturated:-parallel. 2-core linux and 4-core windows runners take about the same time.-parallel 4.At
-parallel 8the same work finishes in roughly half the time, which leaves margin under the timeout even when the workspace is slow.Judgment call: 8 rather than a bigger jump. It doubles each job's load on the shared test workspaces, which already serve 4 tests per job. The cloud-select task (PR runs, at most 50 tests, 2h timeout) is unchanged.
Tests
Ran
TestAccept/bundle/invariant/no_drift(180 tests) on aws with-parallel 8. It finished in 8m47s. At-parallel 4, these tests add up to ~23 min in a quiet CI run. 152 passed. The 28 cluster/job-cluster variants failed in my local environment only, on a missing instance pool (Can't find an instance pool with id: [ID]. (404 RESOURCE_DOES_NOT_EXIST)), which is unrelated to parallelism.Dashboard
AI agent (opus5.5, isaac_claude)
This pull request and its description were written by Isaac.