Commit graph

18 commits

Author SHA1 Message Date
Jeremie Fraeys
b3a0c78903
config: add Plugin GPU Quota, plugins, and audit logging to configs
- Add Plugin GPU Quota config section to scheduler.yaml.example

- Add audit logging config to homelab-secure.yaml (HIPAA-compliant)

- Add Jupyter and vLLM plugin configs to all worker configs:

  - Security settings (passwords, trusted channels, blocked packages)

  - Resource limits (GPU, memory, CPU)

  - Model cache paths and quantization options for vLLM

- Disable plugins in HIPAA deployment mode for compliance

- Update deployments README with plugin services and GPU quotas
2026-02-26 14:34:42 -05:00
Jeremie Fraeys
c459285cab
chore(deploy): update deployment configs and TUI for scheduler
Update deployment and CLI tooling:
- TUI models (jobs, state) with scheduler data
- TUI store with scheduler endpoints
- TUI config with scheduler settings
- Deployment Makefile with scheduler targets
- Deploy script with scheduler registration
- Docker Compose files with scheduler services
- Remove obsolete Dockerfiles (api-server, full-prod, test)
- Update remaining Dockerfiles with scheduler integration
2026-02-26 12:08:31 -05:00
Jeremie Fraeys
685f79c4a7
ci(deploy): add Forgejo workflows and deployment automation
Add CI/CD pipelines for Forgejo/GitHub Actions:
- build.yml - Main build pipeline with matrix builds
- deploy-staging.yml - Automated staging deployment
- deploy-prod.yml - Production deployment with rollback support
- security-modes-test.yml - Security mode validation tests

Add deployment artifacts:
- docker-compose.staging.yml for staging environment
- ROLLBACK.md with rollback procedures and playbooks

Supports multi-environment deployment workflow with proper
gates between staging and production.
2026-02-26 12:04:23 -05:00
Jeremie Fraeys
86f9ae5a7e
docs(config): reorganize configuration structure and add documentation
Restructure configuration files for better organization:
- Add scheduler configuration examples (scheduler.yaml.example)
- Reorganize worker configs into subdirectories:
  - distributed/ - Multi-node cluster configurations
  - standalone/ - Single-node deployment configs
- Add environment-specific configs:
  - dev-local.yaml, docker-dev.yaml, docker-prod.yaml
  - homelab-secure.yaml, worker-prod.toml
- Add deployment configs for different security modes:
  - docker-standard.yaml, docker-hipaa.yaml, docker-dev.yaml

Add documentation:
- configs/README.md with configuration guidelines
- configs/SECURITY.md with security configuration best practices
2026-02-26 12:04:11 -05:00
Jeremie Fraeys
bcc432a524
fix(deployments): use relative paths instead of FETCHML_REPO_ROOT with wrong fallback
Replace all .. with proper relative paths:

- Build context: Use '.' (current directory = project root when using --project-directory)
- Volume mounts: Use './data/...' instead of '../data/...'
- Config mounts: Use './configs/...' instead of '../configs/...'

The '..' fallback was incorrect - when --project-directory is set to repo root,
'..' would point to parent of repo instead of repo itself. Using '.' or
'./path' correctly resolves relative to project root.

Environment variables for data directories (SMOKE_TEST_DATA_DIR, PROD_DATA_DIR,
HOMELAB_DATA_DIR, LOCAL_DATA_DIR) are preserved for runtime customization.
2026-02-24 11:53:19 -05:00
Jeremie Fraeys
3ff5ef320a
fix(deployments): add HOMELAB_DATA_DIR support to homelab-secure
Update docker-compose.homelab-secure.yml to use HOMELAB_DATA_DIR
environment variable with fallback to data/homelab for all volume mounts.
2026-02-24 11:43:38 -05:00
Jeremie Fraeys
5691b06876
fix(deployments): add env var support for data directories
Update all docker-compose files to use environment variables for data paths:

- docker-compose.local.yml: Use LOCAL_DATA_DIR with fallback to ../data/dev
- docker-compose.prod.yml: Use PROD_DATA_DIR with fallback to data/prod
- docker-compose.prod.smoke.yml: Use SMOKE_TEST_DATA_DIR with fallback

This allows smoke tests and local development to use temp directories
instead of repo-relative paths, avoiding file sharing permission issues
on macOS with Docker Desktop or Colima.
2026-02-24 11:43:11 -05:00
Jeremie Fraeys
ce4106a837
fix(smoke-test): copy monitoring configs to temp directory
Promtail mounts monitoring configs from repo root which fails in Colima:

- Copy monitoring/ directory to temp SMOKE_TEST_DATA_DIR
- Update promtail volume path to use SMOKE_TEST_DATA_DIR for configs
- This ensures all mounts are from accessible temp directories
2026-02-24 11:40:32 -05:00
Jeremie Fraeys
bff2336db2
fix(smoke-test): use temp directory for smoke test data
Use /tmp for smoke test data to avoid file sharing issues on macOS/Colima:

- smoke-test.sh: Create temp dir with mktemp, export SMOKE_TEST_DATA_DIR
- docker-compose.dev.yml: Use SMOKE_TEST_DATA_DIR with fallback to data/dev
- Remove file sharing permission checks (no longer needed with tmp)

This avoids Docker Desktop/Colima file sharing permission issues entirely
by using a system temp directory that's always accessible.
2026-02-24 11:37:45 -05:00
Jeremie Fraeys
2a41032414
fix(deployments): fix docker-compose build context paths
Fix build context resolution in smoke test scripts:

- docker-compose.dev.yml: Use ${FETCHML_REPO_ROOT:-..} for api-server and worker
- docker-compose.prod.smoke.yml: Simplify dockerfile path (remove redundant FETCHML_REPO_ROOT)

Previously used 'context: ..' which resolved incorrectly when docker-compose
was run with --project-directory. Now consistently uses FETCHML_REPO_ROOT env var
for proper path resolution in both dev and prod smoke tests.
2026-02-23 20:30:07 -05:00
Jeremie Fraeys
4b8df60e83
deploy: clean up docker-compose configurations
Remove unnecessary service definitions and debug logging configuration
from docker-compose files across all deployment environments.
2026-02-23 18:04:09 -05:00
Jeremie Fraeys
6d200b5ac2
fix(docker): Use named volume for Redis to fix permission errors
Replace bind mount with Docker named volume for Redis data

This fixes 'operation not permitted' errors on macOS Docker Desktop

where bind mounts fail due to file sharing restrictions
2026-02-23 14:20:23 -05:00
Jeremie Fraeys
c89d970210
refactor: migrate from env var to build tags for native libs
Replace FETCHML_NATIVE_LIBS=1 environment variable with -tags native_libs:

Changes:
- internal/queue/native_queue.go: UseNativeQueue is now const true
- internal/queue/native_queue_stub.go: UseNativeQueue is now const false
- build/docker/simple.Dockerfile: Add -tags native_libs to go build
- deployments/docker-compose.dev.yml: Remove FETCHML_NATIVE_LIBS env var
- native/README.md: Update documentation for build tags
- scripts/test-native-with-redis.sh: New test script with Redis via docker-compose

Benefits:
- Compile-time enforcement (no runtime checks needed)
- Cleaner deployment (no env var management)
- Type safety (const vs var)
- Simpler testing with docker-compose Redis integration
2026-02-21 13:43:58 -05:00
Jeremie Fraeys
b4672a6c25
feat: add TUI SSH usability testing infrastructure
Add comprehensive testing for TUI usability over SSH in production-like environment:

Infrastructure:
- Caddy reverse proxy config for WebSocket and API routing
- Docker Compose with SSH test server container
- TUI test configuration for smoke testing

Test Harness:
- SSH server Go test fixture with container management
- TUI driver with PTY support for automated input/output testing
- 8 E2E tests covering SSH connectivity, TERM propagation,
  API/WebSocket connectivity, and TUI configuration

Scripts:
- SSH key generation for test environment
- Manual testing script with interactive TUI verification

The setup allows automated verification that the BubbleTea TUI works
correctly over SSH with proper terminal handling, alt-screen buffer,
and mouse support through Caddy reverse proxy.
2026-02-18 17:48:02 -05:00
Jeremie Fraeys
8b4e1753d1
chore: update configurations and deployment files
- Add Redis secure configuration
- Update worker configurations for homelab and Docker
- Add Forgejo workflow configurations
- Update docker-compose files with improved networking
- Add Caddy configurations for different environments
2026-02-16 20:38:19 -05:00
Jeremie Fraeys
2209ae24c6
chore(config): update configurations and deployment scripts
- Update API server and worker config schemas
- Refine Docker Compose configurations (dev/prod)
- Update deployment scripts and documentation
2026-02-12 12:05:37 -05:00
Jeremie Fraeys
f726806770 chore(ops): reorganize deployments/monitoring and remove legacy scripts 2026-01-05 12:31:26 -05:00
Jeremie Fraeys
03cead6319 Organize docker-compose files and fix test output paths
- Move docker-compose.prod.yml and docker-compose.homelab-secure.yml to deployments/
- Create deployments/README.md with usage instructions
- Update test scripts to use new deployment paths
- Fix performance regression detection to output to tests/bin/
- All test outputs now properly organized in tests/bin/
2025-12-06 13:45:05 -05:00