Commit graph

15 commits

Author SHA1 Message Date
Jeremie Fraeys
68062831b0
refactor(cli): remove redundant doc comments from command files
Removed duplicate help text from doc comments:
- log.zig: Removed usage examples (in printUsage)
- annotate.zig: Removed usage examples (in printUsage)
- experiment.zig: Removed usage examples (in printUsage)

Rationale: printUsage() already contains detailed help text.
Doc comments should not duplicate this information.

All tests pass.
2026-03-05 11:06:28 -05:00
Jeremie Fraeys
8ae0875800
feat(cli): add synced column support to experiment commands
Update experiment creation to track sync status:
- Insert experiments with synced=0 (not synced to server)
- Align with schema update in db.zig for offline tracking

Part of offline run synchronization feature.
2026-03-04 20:23:06 -05:00
Jeremie Fraeys
a1988de8b1
style(cli): Standardize printUsage() formatting with tabs and ASCII symbols
Replace space-padding with consistent tab (\t) alignment in all printUsage() functions.

Add ligature-friendly ASCII symbols:

  - => for results/outcomes (renders as ⇒ with ligatures)

  - ~> for modifications/changes (renders as ~> with ligatures)

  - -> for state transitions (renders as → with ligatures)

  - [OK] / [FAIL] for status indicators

All symbols use ASCII 32-126 for xargs-safe, copy-pasteable output.
2026-02-23 14:09:49 -05:00
Jeremie Fraeys
1a35c54300
feat: integrate native library into Zig CLI
- Add cli/src/native/hash.zig - C ABI wrapper for dataset_hash
- Update cli/src/commands/dataset.zig - Add 'hash' subcommand
- Update cli/build.zig - Link against libdataset_hash.so
- Fix pre-existing CLI errors in experiment.zig (errorMsg signatures, columnInt64)

Usage: ml dataset hash <path>

Note: Additional pre-existing CLI errors remain in sync.zig
2026-02-21 14:08:07 -05:00
Jeremie Fraeys
7efe8bbfbf
native: security hardening, research trustworthiness, and CVE mitigations
Security Fixes:
- CVE-2024-45339: Add O_EXCL flag to temp file creation in storage_write_entries()
  Prevents symlink attacks on predictable .tmp file paths
- CVE-2025-47290: Use openat_nofollow() in storage_open()
  Closes TOCTOU race condition via path_sanitizer infrastructure
- CVE-2025-0838: Add MAX_BATCH_SIZE=10000 to add_tasks()
  Prevents integer overflow in batch operations

Research Trustworthiness (dataset_hash):
- Deterministic file ordering: std::sort after collect_files()
- Recursive directory traversal: depth-limited with cycle detection
- Documented exclusions: hidden files and special files noted in API

Bug Fixes:
- R1: storage_init path validation for non-existent directories
- R2: safe_strncpy return value check before strcat
- R3: parallel_hash 256-file cap replaced with std::vector
- R4: wire qi_compact_index/qi_rebuild_index stubs
- R5: CompletionLatch race condition fix (hold mutex during decrement)
- R6: ARMv8 SHA256 transform fix (save abcd_pre before vsha256hq_u32)
- R7: fuzz_index_storage header format fix
- R8: enforce null termination in add_tasks/update_tasks
- R9: use 64 bytes (not 65) in combined hash to exclude null terminator
- R10: status field persistence in save()

New Tests:
- test_recursive_dataset.cpp: Verify deterministic recursive hashing
- test_storage_symlink_resistance.cpp: Verify CVE-2024-45339 fix
- test_queue_index_batch_limit.cpp: Verify CVE-2025-0838 fix
- test_sha256_arm_kat.cpp: ARMv8 known-answer tests
- test_storage_init_new_dir.cpp: F1 verification
- test_parallel_hash_large_dir.cpp: F3 verification
- test_queue_index_compact.cpp: F4 verification

All 8 native tests passing. Library ready for research lab deployment.
2026-02-21 13:33:45 -05:00
Jeremie Fraeys
a3b957dcc0
refactor(cli): Update build system and core infrastructure
- Makefile: Update build targets for native library integration
- build.zig: Add SQLite linking and native hash library support
- scripts/build_rsync.sh: Update rsync embedded binary build process
- scripts/build_sqlite.sh: Add SQLite constants generation script
- src/assets/README.md: Document embedded asset structure
- src/utils/rsync_embedded_binary.zig: Update for new build layout
2026-02-20 21:39:51 -05:00
Jeremie Fraeys
04ac745b01
refactor(cli): Rename note to annotate and re-add experiment command
- Renamed note.zig to annotate.zig (preserves user's preferred naming)
- Updated all references from 'ml note' to 'ml annotate'
- Re-added experiment.zig with create/list/show subcommands
- Updated main.zig dispatch: 'a' for annotate, 'e' for experiment
- Updated printUsage and test block to reflect changes
2026-02-20 21:32:01 -05:00
Jeremie Fraeys
adf4c2a834
refactor(cli): Update main.zig and remove deprecated commands
- main.zig: Update command dispatch and usage text
  - Wire up new commands: note, logs, sync, cancel, watch
  - Remove deprecated command references
  - Updated usage reflects unified command structure
- Delete deprecated command files:
  - annotate.zig (replaced by note.zig)
  - experiment.zig (functionality in run/note/logs)
  - logs.zig (old version, replaced)
  - monitor.zig (unused)
  - narrative.zig (replaced by note --hypothesis/context)
  - outcome.zig (replaced by note --outcome)
  - privacy.zig (replaced by note --privacy)
  - requeue.zig (functionality merged into queue --rerun)
2026-02-20 21:28:42 -05:00
Jeremie Fraeys
7ce0fd251e
feat(cli): unified commands and local mode support
- Update experiment.zig with unified commands (local + server modes)
- Add init.zig for local project initialization
- Update sync.zig for project synchronization
- Update main.zig to route new local mode commands (experiment, run, log)
- Support automatic mode detection from config (sqlite:// vs wss://)
2026-02-20 15:51:04 -05:00
Jeremie Fraeys
f2f645fe21
refactor(cli): use packet.deinit() in remaining command files
Replace manual defer blocks with packet.deinit() for consistent cleanup:
- jupyter.zig: 5 locations updated (restoreJupyter, startJupyter, stopJupyter, removeJupyter, listServices, packageCommands)
- experiment.zig: 3 locations updated (executeLog, executeShow, executeDelete)
- queue.zig: 1 location updated (queueSingleJob)
- logs.zig: 1 location updated (run)

Eliminates 44 manual cleanup blocks across 4 files.
Build verified: zig build --release=fast
2026-02-18 13:33:10 -05:00
Jeremie Fraeys
34186675dc
refactor(cli): use config.getWebSocketUrl() helper across all commands
Replace inline WebSocket URL construction with Config.getWebSocketUrl()
helper method in all command files. This eliminates code duplication
and ensures consistent URL formatting across the CLI.

Files updated:
- annotate.zig, dataset.zig, experiment.zig, logs.zig
- narrative.zig, prune.zig, queue.zig, requeue.zig
- sync.zig, validate.zig, watch.zig

The helper properly handles ws:// vs wss:// based on port (443).
2026-02-18 13:05:43 -05:00
Jeremie Fraeys
8e3fa94322
feat(cli): enhance Zig CLI with new commands and improved networking
- Add new commands: annotate, narrative, requeue
- Refactor WebSocket client into modular components (net/ws/)
- Add rsync embedded binary support
- Improve error handling and response packet processing
- Update build.zig and completions
2026-02-12 12:05:10 -05:00
Jeremie Fraeys
5ef24e4c6d feat(cli): add validate/info commands and improve protocol handling 2026-01-05 12:31:20 -05:00
Jeremie Fraeys
ea15af1833 Fix multi-user authentication and clean up debug code
- Fix YAML tags in auth config struct (json -> yaml)
- Update CLI configs to use pre-hashed API keys
- Remove double hashing in WebSocket client
- Fix port mapping (9102 -> 9103) in CLI commands
- Update permission keys to use jobs:read, jobs:create, etc.
- Clean up all debug logging from CLI and server
- All user roles now authenticate correctly:
  * Admin: Can queue jobs and see all jobs
  * Researcher: Can queue jobs and see own jobs
  * Analyst: Can see status (read-only access)

Multi-user authentication is now fully functional.
2025-12-06 12:35:32 -05:00
Jeremie Fraeys
d225ea1f00 feat: implement Zig CLI with comprehensive ML experiment management
- Add modern CLI interface built with Zig for performance
- Include TUI (Terminal User Interface) with bubbletea-like features
- Implement ML experiment commands (run, status, manage)
- Add configuration management and validation
- Include shell completion scripts for bash and zsh
- Add comprehensive CLI testing framework
- Support for multiple ML frameworks and project types

CLI provides fast, efficient interface for ML experiment management
with modern terminal UI and comprehensive feature set.
2025-12-04 16:53:58 -05:00