fetch_ml

jfraeysd/fetch_ml

Fork 0

Commit graph

Author	SHA1	Message	Date
Jeremie Fraeys	a36a5e4522	feat(cli): add execution_mode config setting for local/remote/auto preference Add execution_mode enum (local/remote/auto) to config for persistent control over command execution behavior. Removes --local/--remote flags from commands to simplify user workflow - no need to check server connection status manually. Changes: - config.zig: Add ExecutionMode enum, execution_mode field, parsing/serialization - mode.zig: Update detect() to check execution_mode == .local - init.zig: Add --mode flag (local/remote/auto) for setting during init - info.zig: Use config execution_mode, removed --local/--remote flags - run.zig: Use config execution_mode, removed --local/--remote flags - exec/mod.zig: Use config execution_mode, removed --local/--remote flags Priority order for determining execution mode: 1. Config setting (execution_mode: local/remote/auto) 2. Auto-detect only if config is 'auto' Users set mode once during init: ml init --mode=local # Always use local ml init --mode=remote # Always use remote ml init --mode=auto # Auto-detect (default)	2026-03-05 12:18:30 -05:00
Jeremie Fraeys	7efe8bbfbf	native: security hardening, research trustworthiness, and CVE mitigations Security Fixes: - CVE-2024-45339: Add O_EXCL flag to temp file creation in storage_write_entries() Prevents symlink attacks on predictable .tmp file paths - CVE-2025-47290: Use openat_nofollow() in storage_open() Closes TOCTOU race condition via path_sanitizer infrastructure - CVE-2025-0838: Add MAX_BATCH_SIZE=10000 to add_tasks() Prevents integer overflow in batch operations Research Trustworthiness (dataset_hash): - Deterministic file ordering: std::sort after collect_files() - Recursive directory traversal: depth-limited with cycle detection - Documented exclusions: hidden files and special files noted in API Bug Fixes: - R1: storage_init path validation for non-existent directories - R2: safe_strncpy return value check before strcat - R3: parallel_hash 256-file cap replaced with std::vector - R4: wire qi_compact_index/qi_rebuild_index stubs - R5: CompletionLatch race condition fix (hold mutex during decrement) - R6: ARMv8 SHA256 transform fix (save abcd_pre before vsha256hq_u32) - R7: fuzz_index_storage header format fix - R8: enforce null termination in add_tasks/update_tasks - R9: use 64 bytes (not 65) in combined hash to exclude null terminator - R10: status field persistence in save() New Tests: - test_recursive_dataset.cpp: Verify deterministic recursive hashing - test_storage_symlink_resistance.cpp: Verify CVE-2024-45339 fix - test_queue_index_batch_limit.cpp: Verify CVE-2025-0838 fix - test_sha256_arm_kat.cpp: ARMv8 known-answer tests - test_storage_init_new_dir.cpp: F1 verification - test_parallel_hash_large_dir.cpp: F3 verification - test_queue_index_compact.cpp: F4 verification All 8 native tests passing. Library ready for research lab deployment.	2026-02-21 13:33:45 -05:00
Jeremie Fraeys	551597b5df	feat(cli): Add core infrastructure for local mode support - mode.zig: Automatic online/offline mode detection with API ping - manifest.zig: Run manifest read/write/update operations - core/: Common flags, output formatting, and context management - local.zig + local/: Local mode experiment operations - server.zig + server/: Server mode API client - db.zig: Add pid column to ml_runs table for process tracking - config.zig: Add force_local, [experiment] section with name/entrypoint - utils/native_bridge.zig: Native library integration	2026-02-20 21:28:06 -05:00

Author

SHA1

Message

Date

Jeremie Fraeys

a36a5e4522

feat(cli): add execution_mode config setting for local/remote/auto preference

Add execution_mode enum (local/remote/auto) to config for persistent
control over command execution behavior. Removes --local/--remote flags
from commands to simplify user workflow - no need to check server
connection status manually.

Changes:
- config.zig: Add ExecutionMode enum, execution_mode field, parsing/serialization
- mode.zig: Update detect() to check execution_mode == .local
- init.zig: Add --mode flag (local/remote/auto) for setting during init
- info.zig: Use config execution_mode, removed --local/--remote flags
- run.zig: Use config execution_mode, removed --local/--remote flags
- exec/mod.zig: Use config execution_mode, removed --local/--remote flags

Priority order for determining execution mode:
1. Config setting (execution_mode: local/remote/auto)
2. Auto-detect only if config is 'auto'

Users set mode once during init:
  ml init --mode=local     # Always use local
  ml init --mode=remote    # Always use remote
  ml init --mode=auto      # Auto-detect (default)

2026-03-05 12:18:30 -05:00

Jeremie Fraeys

7efe8bbfbf

native: security hardening, research trustworthiness, and CVE mitigations

Security Fixes:
- CVE-2024-45339: Add O_EXCL flag to temp file creation in storage_write_entries()
  Prevents symlink attacks on predictable .tmp file paths
- CVE-2025-47290: Use openat_nofollow() in storage_open()
  Closes TOCTOU race condition via path_sanitizer infrastructure
- CVE-2025-0838: Add MAX_BATCH_SIZE=10000 to add_tasks()
  Prevents integer overflow in batch operations

Research Trustworthiness (dataset_hash):
- Deterministic file ordering: std::sort after collect_files()
- Recursive directory traversal: depth-limited with cycle detection
- Documented exclusions: hidden files and special files noted in API

Bug Fixes:
- R1: storage_init path validation for non-existent directories
- R2: safe_strncpy return value check before strcat
- R3: parallel_hash 256-file cap replaced with std::vector
- R4: wire qi_compact_index/qi_rebuild_index stubs
- R5: CompletionLatch race condition fix (hold mutex during decrement)
- R6: ARMv8 SHA256 transform fix (save abcd_pre before vsha256hq_u32)
- R7: fuzz_index_storage header format fix
- R8: enforce null termination in add_tasks/update_tasks
- R9: use 64 bytes (not 65) in combined hash to exclude null terminator
- R10: status field persistence in save()

New Tests:
- test_recursive_dataset.cpp: Verify deterministic recursive hashing
- test_storage_symlink_resistance.cpp: Verify CVE-2024-45339 fix
- test_queue_index_batch_limit.cpp: Verify CVE-2025-0838 fix
- test_sha256_arm_kat.cpp: ARMv8 known-answer tests
- test_storage_init_new_dir.cpp: F1 verification
- test_parallel_hash_large_dir.cpp: F3 verification
- test_queue_index_compact.cpp: F4 verification

All 8 native tests passing. Library ready for research lab deployment.

2026-02-21 13:33:45 -05:00

Jeremie Fraeys

551597b5df

feat(cli): Add core infrastructure for local mode support

- mode.zig: Automatic online/offline mode detection with API ping
- manifest.zig: Run manifest read/write/update operations
- core/: Common flags, output formatting, and context management
- local.zig + local/: Local mode experiment operations
- server.zig + server/: Server mode API client
- db.zig: Add pid column to ml_runs table for process tracking
- config.zig: Add force_local, [experiment] section with name/entrypoint
- utils/native_bridge.zig: Native library integration

2026-02-20 21:28:06 -05:00

3 commits