Update progress.zig and integrate into sync command:
- progress.zig: update import from colors.zig to io.zig
- sync.zig: add ProgressBar for multi-run sync operations
- Shows progress bar when syncing 2+ runs (not in JSON mode)
- Updates progress after each successful sync
Benefits:
- Better UX for long-running sync operations
- Visual feedback on sync progress
- Maintains clean output for single runs
All tests pass.
Remove redundant logging.zig (28 lines):
- Functions moved to io.zig: printInfo, printSuccess, printWarning, printError, printProgress, confirm
- All functionality preserved with re-exports in utils.zig
Benefits:
- Reduced file count (22 → 21 utils)
- Single source of truth for I/O operations
- No functional changes
Build passes successfully.
Move JSON accessor functions to io.zig:
- jsonGetString, jsonGetInt, jsonGetFloat, jsonGetBool
- json.zig now re-exports from io.zig for backward compatibility
Benefits:
- Single location for all I/O related utilities
- Consistent with terminal/color consolidation
- Reduced file count
Build passes successfully.
Simplify imports by providing direct re-exports:
- utils.isTTY, utils.getWidth (instead of utils.terminal.isTTY)
- utils.reset, utils.red, utils.green (instead of utils.colors.reset)
- Mark colors, terminal, logging as consolidated into io.zig
- Mark rsync modules as deprecated
Benefits:
- Shorter import paths for common utilities
- Reduced typing: utils.red vs utils.colors.red
- Backward compatibility maintained
Build passes successfully.
Consolidate overlapping utilities:
- colors.zig (35 lines) → re-exports from io.zig
- terminal.zig (36 lines) → re-exports from io.zig
- io.zig now contains all terminal, color, and I/O utilities
Benefits:
- Single source of truth for terminal/color logic
- Reduced file count (25 → 23 utils)
- Easier maintenance with all I/O in one place
Build passes successfully.
Move configuration types to queue/mod.zig:
- TrackingConfig with MLflow, TensorBoard, Wandb sub-configs
- QueueOptions with all queue-related options
queue.zig now re-exports from queue/mod.zig for backward compatibility.
Build passes successfully.
Remove simplified placeholders and implement production versions:
- db.zig: Update UUID comment to reflect crypto RNG is already in use
- tls.zig: Implement proper TLS 1.2 ClientHello message construction
- Full record layer header with correct version
- Proper handshake header
- 32-byte cryptographically secure random bytes
- SNI extension with hostname
- ECDHE cipher suites for forward secrecy
- Correct length calculations for all fields
Build passes successfully with production implementations.
Fix broken hash tests to work with current CLI architecture:
- Update import to use @import(src) module system
- Add hash module export to utils.zig
- Make validatePath() public for testing
- Fix Zig 0.15 API: writeFile options struct, var tmp_dir for cleanup
- Fix file paths: use tmp_dir realpath for hashFile
- Replace std.fs.MAX_PATH_BYTES with 4096 buffer
All hash tests now passing.
Update build system for new modules:
- Makefile: add build targets for new components
- build.zig: include new source files in build graph
Supports new CLI architecture.
Remove obsolete native hash implementation files:
- Delete native/hash.zig (superseded by utils/hash.zig)
- Delete utils/native_bridge.zig (replaced by direct TLS)
- Delete utils/native_hash.zig (consolidated into utils/hash.zig)
Cleanup as part of CLI hardening.
Update command structure with improved implementations:
- exec.zig: consolidated command execution
- queue.zig: improved job queuing with narrative support
- run.zig: enhanced local run execution
- dataset.zig, dataset_hash.zig: improved dataset management
Part of CLI hardening for better UX and reliability.
Add progress reporting and offline sync infrastructure:
- progress.zig: progress bars and status reporting
- sync_manager.zig: offline run synchronization manager
Supports resilient operation with server connectivity issues.
Add signal handling, environment detection, and secrets management:
- signals.zig: graceful Ctrl+C handling and signal management
- environment.zig: user environment detection for telemetry
- secrets.zig: secrets redaction for secure logging
Improves CLI reliability and security posture.
Update experiment creation to track sync status:
- Insert experiments with synced=0 (not synced to server)
- Align with schema update in db.zig for offline tracking
Part of offline run synchronization feature.
Add hardware-accelerated hash detection:
- Implement hasShaNi() using CPUID inline assembly for x86_64
- Detect SHA-NI support (bit 29 of EBX in leaf 7, subleaf 0)
- Cross-platform fallback for non-x86_64 architectures
- Enables hardware-accelerated SHA-256 when available
Improves hashing performance on modern Intel/AMD CPUs.
Add TLS transport abstraction for secure WebSocket connections:
- Create tls.zig module with TlsStream struct for TLS-encrypted sockets
- Implement Transport union in client.zig supporting both TCP and TLS
- Update frame.zig and handshake.zig to use Transport abstraction
- Add TLS handshake, read, write, flush, and close operations
- Support TLS 1.2/1.3 protocol versions with error handling
- Zig 0.15 compatible ArrayList API usage
Enables wss:// protocol support for encrypted server communication.
**Makefile:**
- Add native build targets and test infrastructure
- Update benchmark and CI test commands
**cli/build.zig:**
- Build configuration updates for CLI compilation
Remove colors dependency from output.zig
Add terminal.zig for TTY detection and terminal width
Update flags.zig with color flag support
Simplify colors.zig to basic ANSI codes
Update main.zig and utils.zig exports
Replace space-padding with consistent tab (\t) alignment in all printUsage() functions.
Add ligature-friendly ASCII symbols:
- => for results/outcomes (renders as ⇒ with ligatures)
- ~> for modifications/changes (renders as ~> with ligatures)
- -> for state transitions (renders as → with ligatures)
- [OK] / [FAIL] for status indicators
All symbols use ASCII 32-126 for xargs-safe, copy-pasteable output.
- Surface GPUDetectionInfo from parseGPUCountFromConfig for detection metadata
- Document FETCH_ML_TOTAL_CPU and FETCH_ML_GPU_SLOTS_PER_GPU env vars
- Add debug logging for all env var overrides to stderr
- Track config-layer auto-detection in GPUDetectionInfo.ConfigLayerAutoDetected
- Add --include-all flag to artifact scanner (includeAll parameter)
- Add AMD production mode enforcement (error in non-local mode)
- Add GPU detector unit tests for env overrides and AMD aliasing
- SQLite: fetch from sqlite.org/<YEAR>/<file>.zip.sha256 with embedded fallback
- Rsync: fetch from download.samba.org/.../<file>.tar.gz.sha256 with embedded fallback
- Remove hardcoded SHA256 requirement when official checksums available
Add SHA256 for rsync 3.3.0: 7399e9a6708c32d678a72a63219e96f23be0be2336e50fd1348498d07041df90
This allows the build to proceed without requiring GPG keyring setup in CI
- Add sendSyncRun method for run synchronization
- Add sendRerunRequest method for queue rerun
- Add sync_run (0x26) and rerun_request (0x27) opcodes
- Fix protocol import path to relative path
- Fix db.Stmt type alias usage in sync.zig
Go Worker (internal/worker/native_bridge_libs.go):
- Add global hashCtx with sync.Once for lazy initialization
- Eliminates 5-20ms fh_init/fh_cleanup per hash operation
- Uses runtime.NumCPU() for optimal thread count
- Log initialization time for observability
Zig CLI (cli/src/native/hash.zig):
- Add global_ctx with atomic flag and mutex
- Thread-safe initialization with double-check pattern
- Idempotent init() callable from multiple threads
- Log init time for debugging
- Remove separate 'hash' subcommand
- Integrate native SHA256 hash into 'dataset verify'
- Hash is now computed automatically when verifying datasets
- Shows hash in output (JSON, CSV, and text formats)
- Help text updated to indicate auto-hashing
- Renamed note.zig to annotate.zig (preserves user's preferred naming)
- Updated all references from 'ml note' to 'ml annotate'
- Re-added experiment.zig with create/list/show subcommands
- Updated main.zig dispatch: 'a' for annotate, 'e' for experiment
- Updated printUsage and test block to reflect changes
