fetch_ml/.forgejo/workflows/verification.yml
Jeremie Fraeys fe75b6e27a
build(verification): Add Makefile targets and CI for verification suite
Add verification targets to Makefile:
- verify-schema: Check manifest schema hasn't drifted (V.1)
- test-schema-validation: Test schema validation with examples
- lint-custom: Build and run fetchml-vet analyzers (V.4)
- verify-audit: Run audit chain verification tests (V.7)
- verify-audit-chain: CLI tool for verifying specific log files
- verify-all: Run all verification checks (CI target)
- verify-quick: Fast checks for development
- verify-full: Comprehensive verification with unit/integration tests

Add install targets for verification tools:
- install-property-test-deps: gopter for property-based testing (V.2)
- install-mutation-test-deps: go-mutesting for mutation testing (V.3)
- install-security-scan-deps: gosec, nancy for supply chain (V.6)
- install-scorecard: OpenSSF Scorecard (V.10)

Add Forgejo CI workflow (.forgejo/workflows/verification.yml):
- Runs on every push and PR
- Schema drift detection
- Custom linting
- Audit chain verification
- Security scanning integration

Add verification documentation (docs/src/verification.md):
- V.1: Schema validation details
- V.4: Custom linting rules
- V.7: Audit chain verification
- CI integration guide
2026-02-23 19:44:25 -05:00

170 lines
5.4 KiB
YAML

name: Verification & Maintenance
on:
push:
branches: [main, develop]
pull_request:
branches: [main, develop]
schedule:
# Run nightly fault injection and scorecard evaluation
- cron: '0 3 * * *'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
# V.1: Schema Validation
schema-drift-check:
name: V.1 - Schema Drift Detection
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.25'
- name: Verify manifest schema unchanged
run: go test ./internal/manifest/... -run TestSchemaUnchanged -v
- name: Test schema validation (valid manifests)
run: go test ./internal/manifest/... -run TestSchemaValidatesExampleManifest -v
- name: Test schema validation (invalid manifests rejected)
run: go test ./internal/manifest/... -run TestSchemaRejectsInvalidManifest -v
- name: Verify schema version matches constant
run: go test ./internal/manifest/... -run TestSchemaVersionMatchesConst -v
# V.4: Custom Linting Rules
custom-lint:
name: V.4 - Custom Go Vet Analyzers
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.25'
- name: Build custom linting tool
run: go build -o bin/fetchml-vet ./tools/fetchml-vet/cmd/fetchml-vet/
- name: Run custom lint rules
run: |
go vet -vettool=bin/fetchml-vet ./internal/... ./cmd/... 2>&1 | tee lint-results.txt || true
# Fail if any custom lint errors found
if grep -q "bare CreateDetector\|Artifacts without Environment\|inline credential\|HIPAA.*incomplete" lint-results.txt; then
echo "Custom lint violations detected"
exit 1
fi
# V.7: Audit Chain Verification
audit-verification:
name: V.7 - Audit Chain Integrity
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.25'
- name: Run audit chain verifier tests
run: go test ./tests/unit/audit/... -run TestChainVerifier -v
- name: Build audit verifier tool
run: go build -o bin/audit-verifier ./cmd/audit-verifier/
- name: Test audit verifier CLI
run: |
# Create a test audit log
mkdir -p /tmp/audit-test
echo '{"timestamp":"2026-02-23T12:00:00Z","event_type":"job_started","user_id":"test","success":true,"sequence_num":1,"prev_hash":"","event_hash":"abc123"}' > /tmp/audit-test/test.log
# Verify it works (should detect tampering or pass based on hash)
./bin/audit-verifier -log-path=/tmp/audit-test/test.log || true
# V.6: Continuous Security Scanning (extends security-scan.yml)
security-scan-extended:
name: V.6 - Extended Security Scanning
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.25'
- name: Run Nancy (dependency audit)
run: |
go install github.com/sonatype-nexus-community/nancy@latest
go list -json -deps ./... | nancy sleuth --stdout || true
- name: Run govulncheck
uses: golang/govulncheck-action@v1
with:
go-version-input: '1.25'
go-package: ./...
# V.10: OpenSSF Scorecard (weekly)
scorecard:
name: V.10 - OpenSSF Scorecard
if: github.event.schedule == '0 3 * * *'
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
persist-credentials: false
- name: Setup Go
uses: actions/setup-go@v5
with:
go-version: '1.25'
- name: Install and run Scorecard
run: |
go install github.com/ossf/scorecard/v4/cmd/scorecard@latest
scorecard --repo ${{ github.repository }} --format json > scorecard.json || true
cat scorecard.json | jq '.score' || echo "Scorecard evaluation complete"
- name: Upload scorecard results
uses: actions/upload-artifact@v4
with:
name: scorecard-results
path: scorecard.json
# All verification checks summary
verify-summary:
name: Verification Summary
needs: [schema-drift-check, custom-lint, audit-verification, security-scan-extended]
runs-on: ubuntu-latest
if: always()
steps:
- name: Summary
run: |
echo "Verification & Maintenance Checks Complete"
echo "=========================================="
echo "V.1 Schema Validation: ${{ needs.schema-drift-check.result }}"
echo "V.4 Custom Lint: ${{ needs.custom-lint.result }}"
echo "V.7 Audit Verification: ${{ needs.audit-verification.result }}"
echo "V.6 Security Scan: ${{ needs.security-scan-extended.result }}"
- name: Check for failures
if: |
needs.schema-drift-check.result == 'failure' ||
needs.custom-lint.result == 'failure' ||
needs.audit-verification.result == 'failure' ||
needs.security-scan-extended.result == 'failure'
run: |
echo "One or more verification checks failed"
exit 1