Update errors.zig to use consolidated io module:
- Replace colors.printError with io.printError
- Replace colors.printWarning with io.printWarning
- Replace colors.printInfo with io.printInfo
All tests pass.
Fix broken imports in dataset_hash.zig:
- ui/ui.zig and ui/colors.zig don't exist - replaced with std.debug.print
- Updated colors. to io. for consistency with consolidated utilities
- Remove dependency on non-existent ui module
All tests pass.
Update auth.zig import from colors.zig to io.zig:
- colors.zig is now consolidated into io.zig
- Use io module directly for consistency
All tests pass.
Remove unused suggest.zig utility module:
- Not imported by any file in the codebase
- Levenshtein distance functionality not currently needed
- Contains ~200 lines of unused code
If needed in the future for command suggestions, can be restored from git.
All tests pass.
Integrate ProgressBar into queue.zig for multi-job queuing:
- Show progress bar when queuing 2+ jobs (not in JSON mode)
- Update progress after each successful job queue
- Maintain simple output for single job queuing
- Clean up output for batch operations
Benefits:
- Better UX for batch job queuing
- Visual progress indication for long operations
- Consistent with sync command ProgressBar pattern
All tests pass.
Update progress.zig and integrate into sync command:
- progress.zig: update import from colors.zig to io.zig
- sync.zig: add ProgressBar for multi-run sync operations
- Shows progress bar when syncing 2+ runs (not in JSON mode)
- Updates progress after each successful sync
Benefits:
- Better UX for long-running sync operations
- Visual feedback on sync progress
- Maintains clean output for single runs
All tests pass.
Remove redundant logging.zig (28 lines):
- Functions moved to io.zig: printInfo, printSuccess, printWarning, printError, printProgress, confirm
- All functionality preserved with re-exports in utils.zig
Benefits:
- Reduced file count (22 → 21 utils)
- Single source of truth for I/O operations
- No functional changes
Build passes successfully.
Move JSON accessor functions to io.zig:
- jsonGetString, jsonGetInt, jsonGetFloat, jsonGetBool
- json.zig now re-exports from io.zig for backward compatibility
Benefits:
- Single location for all I/O related utilities
- Consistent with terminal/color consolidation
- Reduced file count
Build passes successfully.
Simplify imports by providing direct re-exports:
- utils.isTTY, utils.getWidth (instead of utils.terminal.isTTY)
- utils.reset, utils.red, utils.green (instead of utils.colors.reset)
- Mark colors, terminal, logging as consolidated into io.zig
- Mark rsync modules as deprecated
Benefits:
- Shorter import paths for common utilities
- Reduced typing: utils.red vs utils.colors.red
- Backward compatibility maintained
Build passes successfully.
Consolidate overlapping utilities:
- colors.zig (35 lines) → re-exports from io.zig
- terminal.zig (36 lines) → re-exports from io.zig
- io.zig now contains all terminal, color, and I/O utilities
Benefits:
- Single source of truth for terminal/color logic
- Reduced file count (25 → 23 utils)
- Easier maintenance with all I/O in one place
Build passes successfully.
Move configuration types to queue/mod.zig:
- TrackingConfig with MLflow, TensorBoard, Wandb sub-configs
- QueueOptions with all queue-related options
queue.zig now re-exports from queue/mod.zig for backward compatibility.
Build passes successfully.
Remove simplified placeholders and implement production versions:
- db.zig: Update UUID comment to reflect crypto RNG is already in use
- tls.zig: Implement proper TLS 1.2 ClientHello message construction
- Full record layer header with correct version
- Proper handshake header
- 32-byte cryptographically secure random bytes
- SNI extension with hostname
- ECDHE cipher suites for forward secrecy
- Correct length calculations for all fields
Build passes successfully with production implementations.
Fix broken hash tests to work with current CLI architecture:
- Update import to use @import(src) module system
- Add hash module export to utils.zig
- Make validatePath() public for testing
- Fix Zig 0.15 API: writeFile options struct, var tmp_dir for cleanup
- Fix file paths: use tmp_dir realpath for hashFile
- Replace std.fs.MAX_PATH_BYTES with 4096 buffer
All hash tests now passing.
Remove obsolete native hash implementation files:
- Delete native/hash.zig (superseded by utils/hash.zig)
- Delete utils/native_bridge.zig (replaced by direct TLS)
- Delete utils/native_hash.zig (consolidated into utils/hash.zig)
Cleanup as part of CLI hardening.
Update command structure with improved implementations:
- exec.zig: consolidated command execution
- queue.zig: improved job queuing with narrative support
- run.zig: enhanced local run execution
- dataset.zig, dataset_hash.zig: improved dataset management
Part of CLI hardening for better UX and reliability.
Add progress reporting and offline sync infrastructure:
- progress.zig: progress bars and status reporting
- sync_manager.zig: offline run synchronization manager
Supports resilient operation with server connectivity issues.
Add signal handling, environment detection, and secrets management:
- signals.zig: graceful Ctrl+C handling and signal management
- environment.zig: user environment detection for telemetry
- secrets.zig: secrets redaction for secure logging
Improves CLI reliability and security posture.
Update experiment creation to track sync status:
- Insert experiments with synced=0 (not synced to server)
- Align with schema update in db.zig for offline tracking
Part of offline run synchronization feature.
Add hardware-accelerated hash detection:
- Implement hasShaNi() using CPUID inline assembly for x86_64
- Detect SHA-NI support (bit 29 of EBX in leaf 7, subleaf 0)
- Cross-platform fallback for non-x86_64 architectures
- Enables hardware-accelerated SHA-256 when available
Improves hashing performance on modern Intel/AMD CPUs.
Add TLS transport abstraction for secure WebSocket connections:
- Create tls.zig module with TlsStream struct for TLS-encrypted sockets
- Implement Transport union in client.zig supporting both TCP and TLS
- Update frame.zig and handshake.zig to use Transport abstraction
- Add TLS handshake, read, write, flush, and close operations
- Support TLS 1.2/1.3 protocol versions with error handling
- Zig 0.15 compatible ArrayList API usage
Enables wss:// protocol support for encrypted server communication.
Remove colors dependency from output.zig
Add terminal.zig for TTY detection and terminal width
Update flags.zig with color flag support
Simplify colors.zig to basic ANSI codes
Update main.zig and utils.zig exports
Replace space-padding with consistent tab (\t) alignment in all printUsage() functions.
Add ligature-friendly ASCII symbols:
- => for results/outcomes (renders as ⇒ with ligatures)
- ~> for modifications/changes (renders as ~> with ligatures)
- -> for state transitions (renders as → with ligatures)
- [OK] / [FAIL] for status indicators
All symbols use ASCII 32-126 for xargs-safe, copy-pasteable output.
- Surface GPUDetectionInfo from parseGPUCountFromConfig for detection metadata
- Document FETCH_ML_TOTAL_CPU and FETCH_ML_GPU_SLOTS_PER_GPU env vars
- Add debug logging for all env var overrides to stderr
- Track config-layer auto-detection in GPUDetectionInfo.ConfigLayerAutoDetected
- Add --include-all flag to artifact scanner (includeAll parameter)
- Add AMD production mode enforcement (error in non-local mode)
- Add GPU detector unit tests for env overrides and AMD aliasing
- Add sendSyncRun method for run synchronization
- Add sendRerunRequest method for queue rerun
- Add sync_run (0x26) and rerun_request (0x27) opcodes
- Fix protocol import path to relative path
- Fix db.Stmt type alias usage in sync.zig
Go Worker (internal/worker/native_bridge_libs.go):
- Add global hashCtx with sync.Once for lazy initialization
- Eliminates 5-20ms fh_init/fh_cleanup per hash operation
- Uses runtime.NumCPU() for optimal thread count
- Log initialization time for observability
Zig CLI (cli/src/native/hash.zig):
- Add global_ctx with atomic flag and mutex
- Thread-safe initialization with double-check pattern
- Idempotent init() callable from multiple threads
- Log init time for debugging
- Remove separate 'hash' subcommand
- Integrate native SHA256 hash into 'dataset verify'
- Hash is now computed automatically when verifying datasets
- Shows hash in output (JSON, CSV, and text formats)
- Help text updated to indicate auto-hashing
- Renamed note.zig to annotate.zig (preserves user's preferred naming)
- Updated all references from 'ml note' to 'ml annotate'
- Re-added experiment.zig with create/list/show subcommands
- Updated main.zig dispatch: 'a' for annotate, 'e' for experiment
- Updated printUsage and test block to reflect changes
- queue.zig: Add --rerun <run_id> flag to re-queue completed local runs
- Requires server connection, rejects in offline mode with clear error
- HandleRerun function sends rerun request via WebSocket
- sync.zig: Rewrite for WebSocket experiment sync protocol
- Queries unsynced runs from SQLite ml_runs table
- Builds sync JSON with metrics and params
- Sends sync_run message, waits for sync_ack response
- MarkRunSynced updates synced flag in database
- watch.zig: Add --sync flag for continuous experiment sync
- Auto-sync runs to server every 30 seconds when online
- Mode detection with offline error handling
- note.zig: New unified metadata annotation command
- Supports --text, --hypothesis, --outcome, --confidence, --privacy, --author
- Stores metadata as tags in SQLite ml_tags table
- log.zig: Simplified to unified logs command (fetch/stream only)
- Removed metric/param/tag subcommands (now in run wrapper)
- Supports --follow for live log streaming from server
- cancel.zig: Add local process termination support
- Sends SIGTERM first, waits 5s, then SIGKILL if needed
- Updates run status to CANCELLED in SQLite
- Also supports server job cancellation via WebSocket
- Fork child process and capture stdout/stderr via pipe
- Parse FETCHML_METRIC key=value [step=N] lines from output
- Write run_manifest.json with run metadata
- Insert/update ml_runs table in SQLite with PID tracking
- Stream output to output.log file
- Support entrypoint from config or explicit command after --
- Update experiment.zig with unified commands (local + server modes)
- Add init.zig for local project initialization
- Update sync.zig for project synchronization
- Update main.zig to route new local mode commands (experiment, run, log)
- Support automatic mode detection from config (sqlite:// vs wss://)
