bb25743b0f
- Add production setup scripts for automated deployment - Include monitoring setup and configuration validation - Add legacy setup scripts for various Linux distributions - Implement Bitwarden integration for secure credential management - Add development and production environment setup - Include comprehensive management tools and utilities - Add shell script library with common functions Provides complete automation for setup, deployment, and management of FetchML platform in development and production environments.
94 lines
2.7 KiB
Markdown
94 lines
2.7 KiB
Markdown
# Scripts Directory
|
|
|
|
This directory contains setup and utility scripts for FetchML.
|
|
|
|
## Production Scripts
|
|
|
|
### `setup-prod.sh`
|
|
**Purpose**: Automated production setup for Rocky Linux bare metal deployment
|
|
**Usage**: `sudo ./scripts/setup-prod.sh [base_path] [user] [group]`
|
|
**What it does**:
|
|
- Creates system user and groups
|
|
- Sets up directory structure (`/data/ml-experiments/*`)
|
|
- Installs dependencies (Go, Podman, Redis)
|
|
- Configures GPU support for Podman
|
|
- Creates systemd service files
|
|
- Sets up log rotation
|
|
|
|
**Example**:
|
|
```bash
|
|
sudo ./scripts/setup-prod.sh /data/ml-experiments ml-user ml-group
|
|
```
|
|
|
|
### `validate-prod-config.sh`
|
|
**Purpose**: Validates production configuration files
|
|
**Usage**: `./scripts/validate-prod-config.sh [api-config] [worker-config]`
|
|
**What it does**:
|
|
- Checks config file syntax
|
|
- Verifies base_path consistency
|
|
- Tests Redis connectivity
|
|
- Validates Podman setup
|
|
- Checks directory permissions
|
|
|
|
**Example**:
|
|
```bash
|
|
./scripts/validate-prod-config.sh configs/config-prod.yaml configs/worker-prod.toml
|
|
```
|
|
|
|
## Legacy Setup Scripts (Deprecated)
|
|
|
|
The following scripts are from earlier iterations and are **deprecated** in favor of `setup-prod.sh`:
|
|
|
|
- `setup_rocky.sh` - Use `setup-prod.sh` instead
|
|
- `setup_ubuntu.sh` - Ubuntu support (not primary target)
|
|
- `auto_setup.sh` - Old automated setup (superseded)
|
|
- `setup_common.sh` - Common functions (integrated into setup-prod.sh)
|
|
- `quick_start.sh` - Quick dev setup (use docker-compose on macOS instead)
|
|
- `test_tools.sh` - Tool testing (integrated into validate-prod-config.sh)
|
|
|
|
### Cleanup Recommendation
|
|
These legacy scripts can be removed or archived. The current production setup only needs:
|
|
- `setup-prod.sh`
|
|
- `validate-prod-config.sh`
|
|
|
|
## Usage Workflow
|
|
|
|
### First-Time Production Setup
|
|
```bash
|
|
# 1. Run production setup
|
|
sudo ./scripts/setup-prod.sh
|
|
|
|
# 2. Copy and configure
|
|
sudo cp configs/config-prod.yaml /etc/fetch_ml/config.yaml
|
|
sudo cp configs/worker-prod.toml /etc/fetch_ml/worker.toml
|
|
sudo vim /etc/fetch_ml/config.yaml # Update API keys, etc.
|
|
|
|
# 3. Build and install
|
|
make prod
|
|
sudo make install
|
|
|
|
# 4. Validate
|
|
./scripts/validate-prod-config.sh /etc/fetch_ml/config.yaml /etc/fetch_ml/worker.toml
|
|
|
|
# 5. Start services
|
|
sudo systemctl start fetchml-api fetchml-worker
|
|
sudo systemctl enable fetchml-api fetchml-worker
|
|
```
|
|
|
|
### Development Setup (macOS)
|
|
```bash
|
|
# Use docker-compose for local development
|
|
docker-compose up -d
|
|
|
|
# Or run components directly
|
|
make dev
|
|
./bin/api-server -config configs/config-local.yaml
|
|
```
|
|
|
|
## Script Maintenance
|
|
|
|
When adding new scripts:
|
|
1. Add executable permission: `chmod +x scripts/new-script.sh`
|
|
2. Add header comment with purpose and usage
|
|
3. Update this README
|
|
4. Use consistent error handling and logging
|