feat(ops): alert when a port is published to the internet that nothing allows (stacked on #106) #107

Merged

jfraeys merged 2 commits from jfraeys/feat/exposure-drift-alert into main 2026-10-13 13:03:53 -04:00

Conversation 0 Commits 2 Files changed 25 +1047 -24

jfraeys commented 2026-10-13 12:50:03 -04:00

Owner

Copy link

Merge #106 first (stacked on the infra_metrics role).

The LLDAP admin panel sat on the public internet because UFW cannot see Docker-published ports, and only a staging check found it. This makes production tell you within an hour next time.

• roles/infra_metrics: an hourly timer runs playbooks/files/published_ports.py --prom ... (the same script and the same allowlist staging's connectivity check uses) and writes node-exporter textfile metrics: infra_public_ports_unexpected{container,port}, the total, and a completion timestamp. If docker or iptables is unavailable it exits non-zero and leaves the old file to go stale.
• One allowlist: infra_public_ports_allowed in inventory/group_vars/all.yml (80/tcp, 443/tcp, 3478/udp, 8443/tcp). validate-connectivity.yml now reads it, so staging and production cannot disagree about what is allowed.
• Alerts: PublicPortUnexpected (critical, 5 minutes) and InfraExposureCheckStale (warning, no completed check for 3 hours).
• published_ports.py refactored into functions (find_unexpected, prom_text, main) with identical default CLI output, which staging depends on.

Tests: 8 unit tests with faked docker/iptables (flags only what is neither allowed nor dropped; a DROP in either IP family counts; drops never excuse UDP; empty allowlist flags all public ports; default output unchanged; metrics written atomically with mode 0644; clean host writes zero; a failing check leaves the old file) plus 3 promtool scenarios (fires after the 5-minute window with the right labels, a clean host never alerts, a stopped timer goes stale), mutation-checked. All 36 rules pass promtool check. ansible-lint and yamllint clean. While writing the test I had the for: 5m boundary wrong (it fires at the second evaluation, not before); corrected the test, not the rule.

Not exercised on a real node: the timer and first run; check with systemctl list-timers infra-exposure.timer and ls /var/lib/node_exporter/textfile_collector/infra_exposure.prom after deploying with --tags infra_metrics,prometheus.

🤖 Generated with Claude Code

**Merge #106 first** (stacked on the infra_metrics role). The LLDAP admin panel sat on the public internet because UFW cannot see Docker-published ports, and only a staging check found it. This makes production tell you within an hour next time. - roles/infra_metrics: an hourly timer runs playbooks/files/published_ports.py --prom ... (the same script and the same allowlist staging's connectivity check uses) and writes node-exporter textfile metrics: infra_public_ports_unexpected{container,port}, the total, and a completion timestamp. If docker or iptables is unavailable it exits non-zero and leaves the old file to go stale. - One allowlist: infra_public_ports_allowed in inventory/group_vars/all.yml (80/tcp, 443/tcp, 3478/udp, 8443/tcp). validate-connectivity.yml now reads it, so staging and production cannot disagree about what is allowed. - Alerts: PublicPortUnexpected (critical, 5 minutes) and InfraExposureCheckStale (warning, no completed check for 3 hours). - published_ports.py refactored into functions (find_unexpected, prom_text, main) with identical default CLI output, which staging depends on. Tests: 8 unit tests with faked docker/iptables (flags only what is neither allowed nor dropped; a DROP in either IP family counts; drops never excuse UDP; empty allowlist flags all public ports; default output unchanged; metrics written atomically with mode 0644; clean host writes zero; a failing check leaves the old file) plus 3 promtool scenarios (fires after the 5-minute window with the right labels, a clean host never alerts, a stopped timer goes stale), mutation-checked. All 36 rules pass promtool check. ansible-lint and yamllint clean. While writing the test I had the for: 5m boundary wrong (it fires at the second evaluation, not before); corrected the test, not the rule. Not exercised on a real node: the timer and first run; check with systemctl list-timers infra-exposure.timer and ls /var/lib/node_exporter/textfile_collector/infra_exposure.prom after deploying with --tags infra_metrics,prometheus. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jfraeys added 7 commits 2026-10-13 12:50:04 -04:00

feat(secrets): one expiry classifier and a manifest-only expiry report ... 44ba182f43

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

feat(secrets): rotation lock for vault read-modify-write cycles ... b35100b7dc

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

feat(secrets): monthly expiry report workflow and the manifest facts the repo knows ... d05d8e0b96

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

feat(ops): tool version registry, monthly upstream survey, and the update/security runbook ... 86986b754f

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

ops: tool versions report runs weekly (Mondays), one issue per month refreshed each week ... 1e4d538f6d

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

feat(ops): secret expiry and tool version drift as Prometheus metrics and alerts ... 168d0f4b02

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

feat(ops): hourly check for unexpectedly public ports, alerting through Prometheus ... 253d2df8e4

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

jfraeys merged commit 27357d42d1 into main 2026-10-13 13:03:53 -04:00

jfraeys referenced this pull request from a commit 2026-10-13 13:03:56 -04:00

Merge pull request 'feat(ops): alert when a port is published to the internet that nothing allows (stacked on #106)' (#107) from jfraeys/feat/exposure-drift-alert into main

jfraeys referenced this pull request 2026-10-13 14:25:41 -04:00

Next: immediate steps (merge, deploy, verify) after the 2026-10-13 session #116

jfraeys referenced this pull request 2026-10-13 14:25:55 -04:00

Roadmap: remaining infra and secrets work (tracker) #122

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

jfraeys/infra!107

No description provided.