fetch_ml/scripts/README.md
Jeremie Fraeys 8b75f71a6a
refactor: reorganize scripts into categorized structure
Consolidate 26+ scattered scripts into maintainable hierarchy:

New Structure:
- ci/          CI/CD validation (checks.sh, test.sh, verify-paths.sh)
- dev/         Development workflow (smoke-test.sh, manage-artifacts.sh)
- release/     Release preparation (cleanup.sh, prepare.sh, sanitize.sh, verify.sh, verify-checksums.sh)
- testing/     Test infrastructure (unchanged)
- benchmarks/  Performance tools (track-performance.sh)
- maintenance/ System cleanup (unchanged)
- lib/         Shared functions (unchanged)

Key Changes:
- Unified 6 cleanup-*.sh scripts into release/cleanup.sh with targets
- Merged smoke-test-native.sh into dev/smoke-test.sh --native flag
- Renamed scripts to follow lowercase-hyphen convention
- Moved root-level scripts to appropriate categories
- Updated all Makefile references
- Updated scripts/README.md with new structure

Script count: 26 → 17 (35% reduction)

Breaking Changes:
- Old paths no longer exist, update any direct script calls
- Use make targets (e.g., make ci-checks) for stability
2026-02-18 17:56:59 -05:00

123 lines
3.2 KiB
Markdown

# Scripts Directory
Organized utility scripts for FetchML development, testing, and release management.
## Development Scripts (`dev/`)
### `smoke-test.sh`
**Purpose**: End-to-end smoke testing for dev/prod environments
**Usage**: `./scripts/dev/smoke-test.sh [dev|prod] [--native]`
Tests full stack health by starting docker-compose services and verifying:
- API health endpoints
- Prometheus metrics ingestion (dev mode)
- Native library integration (with `--native` flag)
**Examples**:
```bash
./scripts/dev/smoke-test.sh dev # Standard smoke test
./scripts/dev/smoke-test.sh prod --native # Prod with native libs
```
### `manage-artifacts.sh`
**Purpose**: Local benchmark artifact management
**Usage**: `./scripts/dev/manage-artifacts.sh [list|clean|compare|export]`
Manages benchmark run artifacts in `.local-artifacts/`.
---
## CI Scripts (`ci/`)
### `checks.sh`
**Purpose**: Maintainability enforcement
**Usage**: `./scripts/ci/checks.sh` or `make ci-checks`
### `test.sh`
**Purpose**: CI test runner
**Usage**: `./scripts/ci/test.sh`
### `verify-paths.sh`
**Purpose**: Path convention verification
**Usage**: `./scripts/ci/verify-paths.sh`
---
## Release Scripts (`release/`)
### `cleanup.sh`
**Purpose**: Unified cleanup utility
**Usage**: `./scripts/release/cleanup.sh [all|docker|podman|logs|secrets|state|testdata]`
### `prepare.sh`
**Purpose**: Release preparation
**Usage**: `./scripts/release/prepare.sh`
### `sanitize.sh`
**Purpose**: Config sanitization
**Usage**: `./scripts/release/sanitize.sh`
### `verify.sh`
**Purpose**: Pre-release verification
**Usage**: `./scripts/release/verify.sh` or `make release-check`
---
## Testing Scripts (`testing/`)
### `gen-ssh-test-keys.sh`
**Purpose**: Generate SSH keys for TUI testing
**Usage**: `./scripts/testing/gen-ssh-test-keys.sh`
### `tui-ssh-test.sh`
**Purpose**: Manual TUI SSH testing
**Usage**: `./scripts/testing/tui-ssh-test.sh`
### `test-prod.sh`
**Purpose**: Production validation
**Usage**: `./scripts/testing/test-prod.sh`
---
## Benchmark Scripts (`benchmarks/`)
### `run-benchmarks-local.sh`
**Purpose**: Local benchmark execution
**Usage**: `./scripts/benchmarks/run-benchmarks-local.sh`
---
## Maintenance Scripts (`maintenance/`)
### `cleanup.sh`
**Purpose**: Archive cleanup
**Usage**: `./scripts/maintenance/cleanup.sh [all|dry-run]`
### `cleanup-benchmarks.sh`
**Purpose**: Benchmark lifecycle management
**Usage**: `./scripts/maintenance/cleanup-benchmarks.sh [benchmarks|all|aggressive|status]`
---
## Shared Library (`lib/`)
### `common.sh`
**Purpose**: Common shell functions
**Usage**: `source "$(dirname "$0")/lib/common.sh"`
---
## Legacy Consolidation
| Old Script | New Location |
|------------|--------------|
| `ci-checks.sh` | `ci/checks.sh` |
| `ci-test.sh` | `ci/test.sh` |
| `smoke-test.sh` | `dev/smoke-test.sh` |
| `smoke-test-native.sh` | *merged into dev/smoke-test.sh --native* |
| `manage-artifacts.sh` | `dev/manage-artifacts.sh` |
| `verify-paths.sh` | `ci/verify-paths.sh` |
| `release/verify-release.sh` | `release/verify.sh` |
| `release/prepare-release.sh` | `release/prepare.sh` |
| `release/sanitize-configs.sh` | `release/sanitize.sh` |
| `release/cleanup-*.sh` (6 files) | `release/cleanup.sh` |