jfraeysd/fetch_ml
0
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
fetch_ml/internal
History
Jeremie Fraeys 22f3d66f1d
refactor: Phase 2 - Extract executor implementations 
Created executor package with extracted job execution logic:

1. internal/worker/executor/local.go (104 lines)
   - LocalExecutor implements JobExecutor interface
   - Execute() method for local bash script execution
   - generateScript() helper for creating experiment scripts

2. internal/worker/executor/container.go (229 lines)
   - ContainerExecutor implements JobExecutor interface
   - Execute() method for podman container execution
   - EnvironmentPool interface for image caching
   - Tracking tool provisioning (MLflow, TensorBoard, Wandb)
   - Volume and cache setup
   - selectDependencyManifest() helper

3. internal/worker/executor/runner.go (131 lines)
   - JobRunner orchestrates execution
   - ExecutionMode enum (Auto, Local, Container)
   - Run() method with directory setup and executor selection
   - finalize() for success/failure handling

Key design decisions:
- Executors depend on interfaces (ManifestWriter, not Worker)
- JobRunner composes both executors
- No direct Worker dependencies in executor package
- SetupJobDirectories reused from execution package

Build status: Compiles successfully
2026-02-17 14:14:04 -05:00
..
api refactor: Migrate all test imports from api to api/ws package 2026-02-17 13:52:20 -05:00
audit feat(tracking): add pluggable tracking backends and audit support 2026-01-05 12:33:57 -05:00
auth feat(api): refactor websocket handlers; add health and prometheus middleware 2026-01-05 12:31:07 -05:00
config refactor: Phase 3 - fix config/storage boundaries 2026-02-17 12:49:53 -05:00
container feat(jupyter): improve runtime management and update security/workflow docs 2026-01-05 12:37:27 -05:00
controller Fix multi-user authentication and clean up debug code 2025-12-06 12:35:32 -05:00
domain refactor: extract domain types and consolidate error system (Phases 1-2) 2026-02-17 12:34:28 -05:00
envpool feat(worker): add integrity checks, snapshot staging, and prewarm support 2026-01-05 12:31:13 -05:00
errtypes Fix multi-user authentication and clean up debug code 2025-12-06 12:35:32 -05:00
experiment feat: implement C++ native libraries for performance-critical operations 2026-02-16 20:38:04 -05:00
fileutil Fix multi-user authentication and clean up debug code 2025-12-06 12:35:32 -05:00
jupyter feat(core): API, worker, queue, and manifest improvements 2026-02-12 12:05:17 -05:00
logging feat(jupyter): improve runtime management and update security/workflow docs 2026-01-05 12:37:27 -05:00
manifest feat(core): API, worker, queue, and manifest improvements 2026-02-12 12:05:17 -05:00
metrics feat(api): refactor websocket handlers; add health and prometheus middleware 2026-01-05 12:31:07 -05:00
middleware feat(api): refactor websocket handlers; add health and prometheus middleware 2026-01-05 12:31:07 -05:00
network feat(jupyter): improve runtime management and update security/workflow docs 2026-01-05 12:37:27 -05:00
prommetrics feat(api): refactor websocket handlers; add health and prometheus middleware 2026-01-05 12:31:07 -05:00
queue refactor: Phase 6 - Queue Restructure 2026-02-17 13:41:06 -05:00
resources feat(worker): add integrity checks, snapshot staging, and prewarm support 2026-01-05 12:31:13 -05:00
storage refactor: Phase 3 - fix config/storage boundaries 2026-02-17 12:49:53 -05:00
telemetry Fix multi-user authentication and clean up debug code 2025-12-06 12:35:32 -05:00
tracking feat(tracking): add pluggable tracking backends and audit support 2026-01-05 12:33:57 -05:00
worker refactor: Phase 2 - Extract executor implementations 2026-02-17 14:14:04 -05:00