- Fork child process and capture stdout/stderr via pipe
- Parse FETCHML_METRIC key=value [step=N] lines from output
- Write run_manifest.json with run metadata
- Insert/update ml_runs table in SQLite with PID tracking
- Stream output to output.log file
- Support entrypoint from config or explicit command after --
- Update experiment.zig with unified commands (local + server modes)
- Add init.zig for local project initialization
- Update sync.zig for project synchronization
- Update main.zig to route new local mode commands (experiment, run, log)
- Support automatic mode detection from config (sqlite:// vs wss://)
- Add SQLite amalgamation fetch script (make build-sqlite)
- Embed SQLite in release builds, link system lib in dev
- Create sqlite_embedded.zig utility module
- Unify experiment/run/log commands with auto mode detection
- Add Forgejo CI workflow for building with embedded SQLite
- Update READMEs for local mode and build instructions
SQLite follows rsync embedding pattern: assets/sqlite_release_<os>_<arch>/
Zero external dependencies for release builds.
Reorganize tests for better structure and coverage:
- Move container/security_test.go from internal/ to tests/unit/container/
- Move related tests to proper unit test locations
- Delete orphaned test files (startup_blacklist_test.go)
- Add privacy middleware unit tests
- Add worker config unit tests
- Update E2E tests for homelab and websocket scenarios
- Update test fixtures with utility functions
- Add CLI helper script for arraylist fixes
Add comprehensive research context tracking to jobs:
- Narrative fields: hypothesis, context, intent, expected_outcome
- Experiment groups and tags for organization
- Run comparison (compare command) for diff analysis
- Run search (find command) with criteria filtering
- Run export (export command) for data portability
- Outcome setting (outcome command) for experiment validation
Update queue and requeue commands to support narrative fields.
Add narrative validation to manifest validator.
Add WebSocket handlers for compare, find, export, and outcome operations.
Includes E2E tests for phase 2 features.
- Add handleUnknownCommand() helper function
- Add else clauses to 'i' and 's' switch cases
- Commands like 'invalid_command', 'infoooo', 'syncccc' now properly rejected
- Prevents silent acceptance of invalid commands
- Test TestCLICommandsE2E/CLIErrorHandling now passes
Implement TODO in handleCommandError:
- Log command name with error for crash report context
- Display 'Error in \'{s}': {s}' instead of generic 'Error: {s}'
- Helps users identify which command failed
Build verified: zig build --release=fast
Use utils/json.zig helpers to parse sync progress messages:
- Add json module import
- Rename json variable to json_mode to avoid shadowing
- Parse status, progress, total fields from JSON response
- Show percentage completion for in-progress syncs
- Handle 'complete' and 'error' status codes
Build verified: zig build --release=fast
Replace local file copy with embedded rsync binary for actual remote
synchronization to the server. The embedded rsync is extracted from
assets/ at runtime - no external dependencies required.
Changes:
- Re-add rsync_embedded.zig import
- Replace manual file copying with rsync.sync() call
- Construct remote path as api_key@host:worker_base/commit_id/files/
- Update JSON output to include commit_id
Build verified: zig build --release=fast
- Replace std.process.exit(1) with error.InvalidArgs in sync command
- Replace std.process.exit(1) with error.ValidationFailed in validate command
- Update validate.zig to use protocol.ResponsePacket.deinit() for cleanup
- Build verified: zig build --release=fast
Replace inline WebSocket URL construction with Config.getWebSocketUrl()
helper method in all command files. This eliminates code duplication
and ensures consistent URL formatting across the CLI.
Files updated:
- annotate.zig, dataset.zig, experiment.zig, logs.zig
- narrative.zig, prune.zig, queue.zig, requeue.zig
- sync.zig, validate.zig, watch.zig
The helper properly handles ws:// vs wss:// based on port (443).
Extract common UserContext and authentication logic from cancel.zig and
status.zig into new utils/auth.zig module. Add CommonFlags struct to
utils/flags.zig for shared CLI flags. Add getWebSocketUrl() helper to
Config to eliminate URL construction duplication.
Changes:
- Create cli/src/utils/auth.zig with UserContext and authenticateUser()
- Create cli/src/utils/flags.zig with CommonFlags struct
- Update cancel.zig and status.zig to use shared modules
- Add getWebSocketUrl() helper to config.zig
- Export new modules from utils.zig
Reduces code duplication and improves separation of concerns in the
Zig CLI codebase.
- Move ci-test.sh and setup.sh to scripts/
- Trim docs/src/zig-cli.md to current structure
- Replace hardcoded secrets with placeholders in configs
- Update .gitignore to block .env*, secrets/, keys, build artifacts
- Slim README.md to reflect current CLI/TUI split
- Add cleanup trap to ci-test.sh
- Ensure no secrets are committed
- Add jupyter.zig command with start/stop/status actions
- Update main.zig to include jupyter command in CLI
- CLI now handles all Jupyter setup transparently
- Data scientists only need: ml jupyter start
- Auto-detects container runtime (Podman/Docker)
- Manages container lifecycle automatically
- Provides clear status and error messages
- Fix CLI WebSocket port (9101 vs 9103) in both status and authenticateUser
- Add researcher_user and analyst_user to server config with proper permissions
- Fix API key hashes for all users (complete 64-char SHA256)
- Enable IP whitelist with localhost and private network ranges
- Fix memory leaks in WebSocket handshake (proper key cleanup)
- Fix binary character display in server responses
- All authentication tests now pass: admin, researcher, analyst
Status: Multi-user authentication fully functional
- Add 10ms delay after successful WebSocket handshake
- Use std.posix.nanosleep for proper timing
- This should prevent 'client sent data before handshake complete' error
- CLI WebSocket implementation improved but needs server testing
Status: WebSocket handshake timing improved, ready for testing
- Add Redis configuration to local config
- Fix API key format (api_keys vs apikeys)
- Update CLI to use port 9101
- Disable IP whitelist for testing
- Server now connects to Redis and authenticates
- WebSocket connection reaches server but handshake fails
- CLI needs WebSocket protocol implementation fix
Status: Server running, auth working, WebSocket handshake needs debugging
- Fix YAML tags in auth config struct (json -> yaml)
- Update CLI configs to use pre-hashed API keys
- Remove double hashing in WebSocket client
- Fix port mapping (9102 -> 9103) in CLI commands
- Update permission keys to use jobs:read, jobs:create, etc.
- Clean up all debug logging from CLI and server
- All user roles now authenticate correctly:
* Admin: Can queue jobs and see all jobs
* Researcher: Can queue jobs and see own jobs
* Analyst: Can see status (read-only access)
Multi-user authentication is now fully functional.
- Add modern CLI interface built with Zig for performance
- Include TUI (Terminal User Interface) with bubbletea-like features
- Implement ML experiment commands (run, status, manage)
- Add configuration management and validation
- Include shell completion scripts for bash and zsh
- Add comprehensive CLI testing framework
- Support for multiple ML frameworks and project types
CLI provides fast, efficient interface for ML experiment management
with modern terminal UI and comprehensive feature set.
