Skip to content

Cache vexctl on every OS so release 504s can't fail coverage - #2039

Merged
Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/cache-vexctl-all-os
Nov 9, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 1 commit into
mainfrom
ci-janitor/cache-vexctl-all-os

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

coverage (a required check behind ci-ok) and the test (windows-latest, *) legs download vexctl from github.com/openvex/vexctl/releases/download/... on every run, with no cache. Only macOS restored it from the actions cache.

On 2026-11-09, github.com release downloads returned 504 for minutes at a time. curl's --retry 5 gives up after about 31s, which is too short to ride that out:

The merge queue was empty during these windows. Had a merge_group run been in flight, a coverage failure would have evicted its entry and cost a full rebuild.

Root cause

A required job has a hard dependency on a third-party release CDN for a binary that never changes (pinned version, pinned sha256).

Fix

  • test: the existing vexctl cache restore/save (macOS-only) now runs on every OS. Linux and Windows download only on a cache miss. The cache key and path are unchanged: vexctl-<ver>-go<go>-<os>-<arch>.
  • coverage: adds the same restore → install → save steps (key vexctl-<ver>-Linux-X64-coverage). Shard 1 saves, from main only.
  • A restored binary still goes through the pinned sha256sum -c check, so a cache hit can't substitute a different binary.
  • Saves stay main-only, as before. Restore is continue-on-error with a 5-minute cap, so a cache outage falls back to the download.

No test moved or was removed. Required check names are unchanged.

Proof

  • python3 -B -m unittest discover -s scripts/tests: 411 tests OK
  • YAML parses. zizmor --offline and actionlint findings on ci.yml are identical to origin/main (no new findings).
  • The first main push after merge populates the Linux and Windows keys. Until then PR runs miss the cache and download exactly as today.

https://claude.ai/code/session_011f9aeQtKqZexk5DRhSS2Gg


Generated by Claude Code

The required coverage job and the Windows test legs download
vexctl from a github.com release on every run. On 2026-11-09
those downloads returned 504 for minutes at a time (05:43 and
06:11 UTC), longer than curl's ~30s of retries. That failed
coverage (2) on PR #2018 and test (windows-latest, 1) on the
nightly; in the merge queue it would evict the entry.

Restore the binary from the actions cache on every OS, as macOS
already does, and download only on a miss. The restored file is
still checked against the pinned sha256. Saves stay main-only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011f9aeQtKqZexk5DRhSS2Gg
@mikolalysenko Mikola Lysenko (mikolalysenko) added the ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) label Nov 9, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

bugbot run


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 7fb1fa5. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Nov 9, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Ready for review at 7fb1fa5a9bd94f8a7045e547f036a36dbc061051.

  • CI: 32/32 checks pass (including ci-ok and Cursor Bugbot), 19 skipped. No main-wide failures.
  • Bugbot: reviewed 7fb1fa5 with no findings; no unresolved review threads.
  • Reviewer note: workflow-only change (.github/workflows/ci.yml). The vexctl cache only populates on the first main push after merge, so PR runs keep downloading (and can still hit the github.com release 504s seen today on Never patch a pnpm, yarn or vlt dependency the user patched #2018) until then.

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Final review brief

What it does. The vexctl binary that the test and coverage jobs need was downloaded from github.com releases on every Linux and Windows run, and 504s from that CDN failed required jobs this morning. Now Linux and Windows restore vexctl from the actions cache, as macOS already did, and download only on a cache miss. coverage gets its own restore, install and save steps. This PR changes only the workflow file.

Risk: low. Only .github/workflows/ci.yml changes. A restored binary still has to pass the pinned sha256sum -c check. Restore is continue-on-error with a 5-minute cap, so a cache outage falls back to today's download. Saves run only on main, and only after the install step has verified the binary. Required check names don't change.

Look here

  • ci.yml:464-483: test's restore no longer runs on macOS only. The key is unchanged and still includes the Go version and OS.
  • ci.yml:523-529: the download is skipped on a cache hit, but the sha256 check always runs.
  • ci.yml:551-560: the save drops the macOS-only condition.
  • ci.yml:726-768: coverage gets restore, install and save (-coverage key, saved by shard 1 on main only). VEXCTL_VERSION moved to job-level env so the cache key can use it.

Verified

  • Read the full diff. CACHE_HIT is wired into both install steps. On Windows the restored file is vexctl.exe and is checked against the Windows sha. steps.go runs on every OS, so the test key is well-formed everywhere. The macOS compile path is unchanged.
  • The two test shards on main can race to save the same key. macOS already did this; cache/save warns and doesn't fail.
  • CHANGELOG.md is untouched.
  • CI: 32/32 pass (19 skipped), including ci-ok, clippy and Cursor Bugbot. Bugbot had no findings, and there are no unresolved threads.

Changes I made: none.

Open questions: none. Note that PR runs keep downloading until the first push to main after merge fills the Linux and Windows cache keys.

Auto-merge is armed, so approving sends this straight to the merge queue.


Generated by Claude Code

@mikolalysenko
Mikola Lysenko (mikolalysenko) added this pull request to the merge queue Nov 9, 2026
Merged via the queue into main with commit 20f7eea Nov 9, 2026
51 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the ci-janitor/cache-vexctl-all-os branch November 9, 2026 19:46
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Nov 9, 2026
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-janitor Opened by the CI janitor routine (flakes, redundant tests, CI perf) Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants