jfraeys/infra
Watch
1
0
Fork
You've already forked infra
 0
Code Issues 21 Pull requests 4 Projects Releases Packages Wiki Activity Actions

ci(lint): pinned promtool, sops and age so the alert-rule and key-isolation tests run in CI (stacked on #131) #132

Merged
jfraeys merged 1 commit from jfraeys/ci/pinned-test-tools into main 2026-10-13 17:41:19 -04:00
Conversation 0 Commits 1 Files changed 3 +54 -18
jfraeys commented 2026-10-13 17:18:01 -04:00
Owner
Copy link

Merge #131 first (this builds on its amtool step and replaces it with one combined step).

The lint log showed two suites silently skipping: 'SKIP: docker is not available; alert rule tests not run' and 'SKIP: sops and age-keygen are required'. So the promtool alert-rule tests and the project key-isolation test only ever ran on the operator's Mac.

  • One install step provides promtool 3.15.0 (the running production Prometheus), amtool 0.34.1, sops 3.13.3 (as staging pins it) and age 1.3.2, each SHA-256 verified. age publishes no checksum file; its hash was taken from the v1.3.2 asset when pinned (trust on first use).
  • scripts/tests/test-alert-rules.sh: local promtool if on PATH, else Docker, else SKIP (it used to exit at the top without Docker).
  • docs/tool-versions.yml: promtool and age registered (tier C, checksum); sops gains the lint.yml pin. check-tool-versions reports all up to date; its tests pass.
  • Verified by running the workflow's own install step in a Docker-less python:3.11 container: all four checksums OK, correct versions, amtool config test and key-isolation test pass. The promtool suite passed 4/4 SUCCESS in 3 of 3 runs with a native aarch64 promtool 3.15.0.

One thing to watch on the first CI run. With the x86 promtool 3.15.0 running under QEMU emulation on an Apple Silicon Mac, promtool crashed twice with Go's 'fatal error: concurrent map writes' (util/stats TimerGroup) while running the unit tests. The same version natively (aarch64) passed 3/3, and locally in the prom/prometheus:v3 image the suite always passes. I believe this is an emulation timing artifact: the CI runner is native amd64. If the lint job's alert-rule step ever shows that crash on the runner, it is a real promtool race: retry the step, and pin another promtool version.

🤖 Generated with Claude Code

**Merge #131 first** (this builds on its amtool step and replaces it with one combined step). The lint log showed two suites silently skipping: 'SKIP: docker is not available; alert rule tests not run' and 'SKIP: sops and age-keygen are required'. So the promtool alert-rule tests and the project key-isolation test only ever ran on the operator's Mac. - One install step provides promtool 3.15.0 (the running production Prometheus), amtool 0.34.1, sops 3.13.3 (as staging pins it) and age 1.3.2, each SHA-256 verified. age publishes no checksum file; its hash was taken from the v1.3.2 asset when pinned (trust on first use). - scripts/tests/test-alert-rules.sh: local promtool if on PATH, else Docker, else SKIP (it used to exit at the top without Docker). - docs/tool-versions.yml: promtool and age registered (tier C, checksum); sops gains the lint.yml pin. check-tool-versions reports all up to date; its tests pass. - Verified by running the workflow's own install step in a Docker-less python:3.11 container: all four checksums OK, correct versions, amtool config test and key-isolation test pass. The promtool suite passed 4/4 SUCCESS in 3 of 3 runs with a native aarch64 promtool 3.15.0. **One thing to watch on the first CI run.** With the x86 promtool 3.15.0 running under QEMU emulation on an Apple Silicon Mac, promtool crashed twice with Go's 'fatal error: concurrent map writes' (util/stats TimerGroup) while running the unit tests. The same version natively (aarch64) passed 3/3, and locally in the prom/prometheus:v3 image the suite always passes. I believe this is an emulation timing artifact: the CI runner is native amd64. If the lint job's alert-rule step ever shows that crash on the runner, it is a real promtool race: retry the step, and pin another promtool version. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
ci(lint): run amtool check-config in CI with a checksum-pinned amtool
All checks were successful
Lint / lint (pull_request) Successful in 1m30s
Details
Secret scan / scan (pull_request) Successful in 12s
Details
59ce6cc4b5
 
The lint runners have no Docker, so the Alertmanager config test skipped its amtool step there. The job now installs a pinned
amtool (v0.34.1, SHA-256 checked, registered in docs/tool-versions.yml); the test uses a local amtool, else Docker, else skips.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
ci(lint): pinned promtool, amtool, sops and age so the alert-rule and key-isolation tests run in CI
All checks were successful
Lint / lint (pull_request) Successful in 2m39s
Details
Secret scan / scan (pull_request) Successful in 13s
Details
5600d6eecc
 
The runners have no Docker, sops or age, so test-alert-rules.sh and test-project-key-isolation.sh printed SKIP in the lint job.
One checksum-verified install step now provides all four (promtool 3.15.0 = running Prometheus); test-alert-rules.sh uses a
local promtool, else Docker, else skips. Registered in docs/tool-versions.yml.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jfraeys/infra!132
No description provided.