jfraeysd/fetch_ml
0
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
fetch_ml/scripts
History
Jeremie Fraeys 93d6d63d8d
chore(deploy): update Docker compose files and add MinIO lifecycle policies 
Docker Compose updates:
- docker-compose.dev.yml: add GPU support, local scheduler and worker
- docker-compose.staging.yml: production-like staging with SSL termination
- docker-compose.test.yml: ephemeral test environment with seeded data

MinIO lifecycle management:
- Add lifecycle-dev.json: 7-day retention for dev artifacts
- Add lifecycle-staging.json: 30-day retention with transition to cold

Build improvements:
- Makefile: add native library build targets and cross-platform support
- scripts/release/cleanup.sh: improved artifact cleanup with dry-run mode
2026-03-12 12:06:16 -04:00
..
benchmarks chore(build): update build system, Dockerfiles, and dependencies 2026-03-08 13:03:48 -04:00
build chore(build): update build system, Dockerfiles, and dependencies 2026-03-08 13:03:48 -04:00
ci chore(build): update build system, Dockerfiles, and dependencies 2026-03-08 13:03:48 -04:00
deploy feat: add new API handlers, build scripts, and ADRs 2026-03-04 13:24:27 -05:00
dev chore(build): update build system, Dockerfiles, and dependencies 2026-03-08 13:03:48 -04:00
lib chore(build): update build system, scripts, and additional tests 2026-02-12 12:05:55 -05:00
maintenance test: modernize test suite for streamlined infrastructure 2026-03-04 13:24:24 -05:00
release chore(deploy): update Docker compose files and add MinIO lifecycle policies 2026-03-12 12:06:16 -04:00
testing chore: migrate scripts from docker-compose v1 to v2 2026-03-04 13:22:26 -05:00
check-audit-sink.sh chore(tools): update scripts, native libs, and documentation 2026-02-26 12:08:58 -05:00
README.md chore: move detect_native.go and setup_monitoring.py to dev/ 2026-02-18 17:57:57 -05:00

README.md

Scripts Directory

Organized utility scripts for FetchML development, testing, and release management.

Development Scripts (dev/)

smoke-test.sh

Purpose: End-to-end smoke testing for dev/prod environments
Usage: ./scripts/dev/smoke-test.sh [dev|prod] [--native]

Tests full stack health by starting docker-compose services and verifying:

  • API health endpoints
  • Prometheus metrics ingestion (dev mode)
  • Native library integration (with --native flag)

Examples:

./scripts/dev/smoke-test.sh dev          # Standard smoke test
./scripts/dev/smoke-test.sh prod --native # Prod with native libs

manage-artifacts.sh

Purpose: Local benchmark artifact management
Usage: ./scripts/dev/manage-artifacts.sh [list|clean|compare|export]

Manages benchmark run artifacts in .local-artifacts/.

detect-native.go

Purpose: Native library detection utility
Usage: go run ./scripts/dev/detect-native.go

Detects native C++ libraries and CGO configuration. Shows which native libs are available.

setup-monitoring.py

Purpose: Development monitoring setup
Usage: python3 ./scripts/dev/setup-monitoring.py

Creates Grafana provisioning structure for local development.

CI Scripts (ci/)

checks.sh

Purpose: Maintainability enforcement
Usage: ./scripts/ci/checks.sh or make ci-checks

test.sh

Purpose: CI test runner
Usage: ./scripts/ci/test.sh

verify-paths.sh

Purpose: Path convention verification
Usage: ./scripts/ci/verify-paths.sh

Release Scripts (release/)

cleanup.sh

Purpose: Unified cleanup utility
Usage: ./scripts/release/cleanup.sh [all|docker|podman|logs|secrets|state|testdata]

prepare.sh

Purpose: Release preparation
Usage: ./scripts/release/prepare.sh

sanitize.sh

Purpose: Config sanitization
Usage: ./scripts/release/sanitize.sh

verify.sh

Purpose: Pre-release verification
Usage: ./scripts/release/verify.sh or make release-check

Testing Scripts (testing/)

gen-ssh-test-keys.sh

Purpose: Generate SSH keys for TUI testing
Usage: ./scripts/testing/gen-ssh-test-keys.sh

tui-ssh-test.sh

Purpose: Manual TUI SSH testing
Usage: ./scripts/testing/tui-ssh-test.sh

test-prod.sh

Purpose: Production validation
Usage: ./scripts/testing/test-prod.sh

Benchmark Scripts (benchmarks/)

run-benchmarks-local.sh

Purpose: Local benchmark execution
Usage: ./scripts/benchmarks/run-benchmarks-local.sh

Maintenance Scripts (maintenance/)

cleanup.sh

Purpose: Archive cleanup
Usage: ./scripts/maintenance/cleanup.sh [all|dry-run]

cleanup-benchmarks.sh

Purpose: Benchmark lifecycle management
Usage: ./scripts/maintenance/cleanup-benchmarks.sh [benchmarks|all|aggressive|status]

Shared Library (lib/)

common.sh

Purpose: Common shell functions
Usage: source "$(dirname "$0")/lib/common.sh"

Legacy Consolidation

Old Script New Location
ci-checks.sh ci/checks.sh
ci-test.sh ci/test.sh
smoke-test.sh dev/smoke-test.sh
smoke-test-native.sh merged into dev/smoke-test.sh --native
manage-artifacts.sh dev/manage-artifacts.sh
verify-paths.sh ci/verify-paths.sh
release/verify-release.sh release/verify.sh
release/prepare-release.sh release/prepare.sh
release/sanitize-configs.sh release/sanitize.sh
release/cleanup-*.sh (6 files) release/cleanup.sh