feat(gitops): Terraform plan/promote/apply/drift-check workflows + multi-repo CI provisioner #2

Merged
jfraeys merged 2 commits from phase1/gitops-workflows into main 2026-10-02 16:26:00 -04:00
Owner

GitOps plan phase 1.

  • ci_ephemeral_provisioner now watches a list of repos (coaching-site + infra) instead of one hardcoded repo. Job discovery was already instance-wide; only get_task_status() was repo-scoped, so the change resolves repo_id to owner/repo and threads it through cycle.json and VM reuse.
  • terraform-plan.yml: plan against the linode workspace on PRs touching terraform/**.
  • staging-validate.yml: new promote job (needs: validate) with a pre-promotion production drift gate, then opens or updates a main to prod PR. Removes the old 'run locally' message.
  • terraform-apply.yml: applies on push to prod. The gitops_watcher signal step is a placeholder until phase 4.
  • drift-check.yml: scheduled refresh-only plan across linode, ovh, ovh-dr.

Not verified live (no Forgejo runner available when written): runner pickup of infra jobs, state-backend credentials on the runner, and the repo secrets these workflows read (FORGEJO_ADMIN_TOKEN, LINODE_TOKEN, CLOUDFLARE_API_TOKEN).

🤖 Generated with Claude Code

GitOps plan phase 1. - ci_ephemeral_provisioner now watches a list of repos (coaching-site + infra) instead of one hardcoded repo. Job discovery was already instance-wide; only get_task_status() was repo-scoped, so the change resolves repo_id to owner/repo and threads it through cycle.json and VM reuse. - terraform-plan.yml: plan against the linode workspace on PRs touching terraform/**. - staging-validate.yml: new promote job (needs: validate) with a pre-promotion production drift gate, then opens or updates a main to prod PR. Removes the old 'run locally' message. - terraform-apply.yml: applies on push to prod. The gitops_watcher signal step is a placeholder until phase 4. - drift-check.yml: scheduled refresh-only plan across linode, ovh, ovh-dr. Not verified live (no Forgejo runner available when written): runner pickup of infra jobs, state-backend credentials on the runner, and the repo secrets these workflows read (FORGEJO_ADMIN_TOKEN, LINODE_TOKEN, CLOUDFLARE_API_TOKEN). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
GitOps plan phase 1, step 0 (prerequisite for everything else): the
ephemeral CI runner controller only ever watched jfraeys/coaching-site's
queued jobs -- new infra workflows targeting runs-on: ephemeral-large
would queue forever with nothing picking them up.

Turns out job *discovery* (find_queued_jobs(), via
/admin/runners/jobs) was already instance-wide -- only get_task_status()
is actually repo-scoped by Forgejo's own API design, and each job object
only carries a numeric repo_id, not an owner/repo string. So the real
fix is smaller than "poll N repos": resolve repo_id -> (owner, repo) for
a configured watch list once per reconcile pass that needs it, filter
discovered jobs down to watched repos, and thread owner/repo through
cycle.json so get_task_status() (and VM-reuse's next-job selection, which
has the same instance-wide-discovery gap) query the right repo.

One repo-scoped token already covers every repo its owner can access
(Forgejo scopes are account-wide permission levels, not per-repo ACLs),
so no per-repo token/secret was needed -- just the existing
CI_EPHEMERAL_FORGEJO_REPO_TOKEN.

- ci_ephemeral_provisioner_forgejo_owner/_repo (single pair) ->
  ci_ephemeral_provisioner_watched_repos (list), defaults to
  [jfraeys/coaching-site, jfraeys/infra]
- CI_EPHEMERAL_FORGEJO_OWNER/_REPO -> CI_EPHEMERAL_WATCHED_REPOS
  (comma-separated)
- get_task_status(task_id) -> get_task_status(task_id, owner, repo)
- main()'s job selection and both VM-reuse call sites now filter to
  jobs whose repo_id resolves to a watched repo, and store owner/repo
  on the cycle -- a job from an unrecognized repo reusing our label is
  left alone, not guessed at
GitOps plan phase 1, steps 1a-1d. Real credentials CI already has for
staging (LINODE_TOKEN, CLOUDFLARE_API_TOKEN) extend cleanly to owning
the Terraform layer end-to-end -- no invariant conflict, unlike the
Ansible layer (see roles/gitops_watcher, phase 4, not yet built).

New: .forgejo/workflows/terraform-plan.yml (1a) -- posts `tofu plan`
against the pinned `linode` workspace as a PR comment on any PR touching
terraform/**. Read-only, never applies. Explicit workspace select always
-- never the orphaned "default" workspace (docs/incidents.md: ungated by
local.skip_dns, ~62 unrelated resources).

New: .forgejo/workflows/drift-check.yml (1d) -- scheduled (6h) +
workflow_dispatch `tofu plan -refresh-only -detailed-exitcode` across
linode/ovh/ovh-dr (matrix), opens a Forgejo issue on drift. This is the
direct, automatic fix for the incident that prompted this whole plan:
Terraform declared cloudflare_record.proxied=true for vpn.jfraeys.com,
live had drifted to false. -refresh-only already does this comparison
correctly; the gap was purely "nobody runs it on a schedule."

Modified: .forgejo/workflows/staging-validate.yml -- added a `promote`
job (needs: validate) implementing 1b (promotion gate) and setting up
1c's trigger:
  - Pre-promotion production drift check: same -refresh-only mechanism
    as drift-check.yml, but run synchronously at promotion time --
    answers "is prod already wrong, don't build on top of it" rather
    than only "did something change since the last deploy" (drift-check
    still runs on its own schedule too, for drift introduced *between*
    promotions).
  - On clean drift check: opens/updates a main->prod PR with the staging
    summary, drift-check result, and a synthetic-secrets Ansible preview
    attached. Merging that PR is the entire production approval event --
    no separate approve command.
  - Removed the old "Run locally to deploy: ./setup --confirm-prod"
    message -- this replaces it.
  - Implemented as a same-workflow `needs:` job dependency rather than a
    separate promote-to-prod.yml relying on workflow_run/workflow_call,
    since Forgejo Actions' support for cross-workflow triggers wasn't
    confirmed -- needs: is unambiguously supported and guarantees
    ordering either way. Flagged as an open question in the plan;
    resolved this way during implementation.

New: .forgejo/workflows/terraform-apply.yml (1c) -- applies Terraform
automatically on push to `prod` (i.e. exactly when the promotion PR
above gets merged). Deliberately does not touch Ansible-managed config
-- roles/gitops_watcher (phase 4) owns that with real secrets this CI
job never has. Includes a placeholder step for signaling gitops_watcher
once that role exists.

All four files' YAML validated with pyyaml, embedded bash blocks synax-
checked with `bash -n`, embedded Python snippets compile-checked --
no live Forgejo Actions runner available in this environment to test
an actual run.
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
jfraeys/infra!2
No description provided.