fetch_ml/docs/src/file-naming-conventions.md
Jeremie Fraeys 3187ff26ea
refactor: complete maintainability phases 1-9 and fix all tests
Test fixes (all 41 test packages now pass):
- Fix ComputeTaskProvenance - add dataset_specs JSON output
- Fix EnforceTaskProvenance - populate all metadata fields in best-effort mode
- Fix PrewarmNextOnce - preserve prewarm state when queue empty
- Fix RunManifest directory creation in SetupJobDirectories
- Add ManifestWriter to test worker (simpleManifestWriter)
- Fix worker ID mismatch (use cfg.WorkerID)
- Fix WebSocket binary protocol responses
- Implement all WebSocket handlers: QueueJob, QueueJobWithSnapshot, StatusRequest,
  CancelJob, Prune, ValidateRequest (with run manifest validation), LogMetric,
  GetExperiment, DatasetList/Register/Info/Search

Maintainability phases completed:
- Phases 1-6: Domain types, error system, config boundaries, worker/API/queue splits
- Phase 7: TUI cleanup - reorganize model package (jobs.go, messages.go, styles.go, keys.go)
- Phase 8: MLServer unification - consolidate worker + TUI into internal/network/mlserver.go
- Phase 9: CI enforcement - add scripts/ci-checks.sh with 5 checks:
  * No internal/ -> cmd/ imports
  * domain/ has zero internal imports
  * File size limit (500 lines, rigid)
  * No circular imports
  * Package naming conventions

Documentation:
- Add docs/src/file-naming-conventions.md
- Add make ci-checks target

Lines changed: +756/-36 (WebSocket fixes), +518/-320 (TUI), +263/-20 (Phase 8-9)
2026-02-17 20:32:14 -05:00

4.9 KiB

File Naming Conventions

This document defines the naming conventions for the FetchML codebase to maintain consistency and clarity.

Package Structure

fetch_ml/
├── cmd/                    # Entry points (wiring only)
│   ├── api-server/        # API server entry point
│   ├── worker/            # Worker entry point
│   ├── tui/               # Terminal UI entry point
│   └── ...
├── internal/              # Internal packages (business logic)
│   ├── domain/            # Core domain types (zero internal deps)
│   ├── api/               # HTTP/WebSocket handlers
│   ├── worker/            # Worker business logic
│   ├── queue/             # Queue implementations
│   ├── storage/           # Storage abstractions
│   ├── network/           # Network utilities
│   └── ...
└── tests/                 # Test suites
    ├── unit/
    ├── integration/
    └── e2e/

Naming Rules

1. General File Naming

  • Use snake_case for file names: job_runner.go, task_queue.go
  • Match file name to primary content: The file name should reflect the main type or function defined within
  • One concern per file: Each file should have a single, clear purpose

2. Package-Level Files

Pattern Purpose Example
doc.go Package documentation internal/domain/doc.go
types.go Shared type definitions internal/api/types.go
errors.go Error definitions internal/queue/errors.go
const.go Constants internal/config/const.go

3. Handler/Service Files

Pattern Purpose Example
<noun>.go Main type definition worker.go, server.go
<noun>_test.go Unit tests worker_test.go
<verb>_<noun>.go Action-focused run_job.go, cancel_task.go
<noun>_<action>.go Noun-focused action job_runner.go, task_scheduler.go

4. Internal Package Organization

internal/worker/
├── worker.go           # Main Worker type (always present)
├── config.go           # Configuration parsing
├── factory.go          # Constructor functions (NewWorker)
├── lifecycle.go        # Start/Stop/Shutdown
├── runloop.go          # Main processing loop
├── metrics.go          # Metrics setup
├── gpu.go              # GPU utilities
└── execution/          # Sub-package for execution logic
    ├── runner.go       # JobRunner type
    ├── setup.go        # Job directory setup
    └── container.go    # Container helpers

5. TUI Model Package (Post-Phase 7)

cmd/tui/internal/model/
├── state.go        # State struct, InitialState()
├── jobs.go         # Job type, JobStatus
├── messages.go     # tea.Msg types
├── styles.go       # Lipgloss styles
├── keys.go         # KeyMap, DefaultKeys()
└── doc.go          # Package docs

6. API Package (Post-Phase 5)

internal/api/
├── server.go           # Server type
├── factory.go          # NewServer()
├── routes.go           # Route registration
├── middleware.go       # Auth, logging middleware
├── errors.go           # API error responses
├── health.go           # Health check handlers
├── jobs/               # Job handlers sub-package
│   ├── handlers.go
│   ├── manifest.go
│   └── paths.go
├── jupyter/            # Jupyter handlers
│   └── handlers.go
├── validate/           # Validation handlers
│   └── handlers.go
└── ws/                 # WebSocket infrastructure
    ├── handler.go
    └── protocol.go

Anti-Patterns to Avoid

Don't: utils.go, helpers.go, misc.go - too vague
Do: path_utils.go, response_helpers.go, string_utils.go - specific purpose

Don't: handler_v2.go, new_worker.go - version numbers in names
Do: Update the existing file or create purpose-specific files

Don't: worker/interface.go - stutter (package name in file name)
Do: worker/interfaces.go or just define in worker.go

Don't: 1000+ line files
Do: Split into focused files (<400 lines each)

Layer Enforcement

The scripts/ci-checks.sh script enforces these architectural rules:

  1. No internal/ → cmd/ imports: Business logic cannot depend on entry points
  2. domain/ has zero internal imports: Core types only use stdlib
  3. File size limits: Warning if files exceed 400 lines
  4. No circular imports: Verified via go list

Run checks with:

make ci-checks

Success Metrics

  • All files <400 lines (warning at 400, error at 600)
  • File name matches primary content
  • Package imports follow layer direction (cmd → internal → domain)
  • CI checks pass: make ci-checks