CI: remove some inter-job dependencies, run cross-compile task with make -j, use /tmp for Go build cache #5856
Conversation
nalind
force-pushed
the
ci-deps
branch
4 times, most recently
from
b5b3b59 to
02d7502
Compare
|
LGTM |
nalind
force-pushed
the
ci-deps
branch
4 times, most recently
from
ebe486a to
f7b1a39
Compare
nalind
force-pushed
the
ci-deps
branch
2 times, most recently
from
49ab45e to
cdb2120
Compare
.cirrus.yml
Outdated
| - make cross CGO_ENABLED=0 | ||
|
|
||
| - make -j cross CGO_ENABLED=0 | ||
| - du -sh `go env GOCACHE` |
There was a problem hiding this comment.
I think we can drop the du or do you still need that?
There was a problem hiding this comment.
I'm concerned about using /tmp for the Go cache, but you're probably right that we don't need to be worrying about that in every task.
.cirrus.yml
Outdated
| @@ -170,6 +170,7 @@ unit_task: | |||
| setup_script: '${SCRIPT_BASE}/setup.sh |& ${_TIMESTAMP}' | |||
| build_script: '${SCRIPT_BASE}/build.sh |& ${_TIMESTAMP}' | |||
| unit_test_script: '${SCRIPT_BASE}/test.sh unit |& ${_TIMESTAMP}' | |||
| du_script: du -sh `go env GOCACHE` | |||
There was a problem hiding this comment.
same here and in the other places?
.cirrus.yml
Outdated
| depends_on: &smoke_vendor_cross | ||
| - smoke | ||
| - vendor | ||
| - cross_build |
There was a problem hiding this comment.
I wonder if we should at least keep the dependency for vendor & smoke test as those are fast enough.
In podman we have basically two stages build/validate -> actual tests:
https://raw.githubusercontent.com/wiki/containers/podman/cirrus-map.svg
So I think it makes sense for buildah to follow that, unless we really like to safe the the 7 mins form the smoke test?
There was a problem hiding this comment.
Having added depends_on bits back, it looks like we're getting just about everything else stalled until the cross-build check finishes. Replacing some of the "only_if" directives with "skip" directives looks like it will avoid this weirdness.
| @@ -93,7 +94,7 @@ smoke_task: | |||
| name: "Smoke Test" | |||
|
|
|||
| gce_instance: | |||
| memory: "12Gb" | |||
There was a problem hiding this comment.
pre existing but defining a task with no cpu seems odd
There was a problem hiding this comment.
Sure. We'll try 4, maybe it'll run this task a bit faster.
Tell make to run all of the cross-compile tasks at once. This uses a lot of space in /tmp, and a later patch will also use it for the build cache, so we request more memory for the task. Also request more CPUs. Signed-off-by: Nalin Dahyabhai <[email protected]>
nalind
changed the title
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Luap99, nalind The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Remove the bits of the CI configuration that prevent some jobs from starting unless/until the cross-compile tests have passed. That costs more time than we're willing to wait, at this time at least. Use the skip directive to avoid having to get all of the dependencies and only_if directives to agree. Signed-off-by: Nalin Dahyabhai <[email protected]>
Point the compiler's build cache at /tmp. This generally requires more memory, but it's faster, and the build cache would be discarded anyway. Signed-off-by: Nalin Dahyabhai <[email protected]>
The unit tests don't use the binary, so we shouldn't be spending time compiling it. Signed-off-by: Nalin Dahyabhai <[email protected]>
|
/lgtm |
What type of PR is this?
/kind other
What this PR does / why we need it:
As a quick attempt to speed up our CI's total start-to-finish time, this hits what we think will be low-hanging fruit.
How to verify it
Check the clock duration for https://cirrus-ci.com/github/containers/buildah/pull%2F5856 compared to builds for other pull requests.
Which issue(s) this PR fixes:
None
Special notes for your reviewer:
Does this PR introduce a user-facing change?