Jeremie Fraeys
cb142213fa
chore(build): update build system, Dockerfiles, and dependencies
...
Build and deployment improvements:
Makefile:
- Native library build targets with ASan support
- Cross-platform compilation helpers
- Performance benchmark targets
- Security scan integration
Docker:
- secure-prod.Dockerfile: Hardened production image (non-root, minimal surface)
- simple.Dockerfile: Lightweight development image
Scripts:
- build/: Go and native library build scripts, cross-platform builds
- ci/: checks.sh, test.sh, verify-paths.sh for validation
- benchmarks/: Local performance testing and regression tracking
- dev/: Monitoring setup
Dependencies: Update to latest stable with security patches
Commands:
- api-server/main.go: Server initialization updates
- data_manager/data_sync.go: Data sync with visibility
- errors/main.go: Error handling improvements
- tui/: TUI improvements for group management
2026-03-08 13:03:48 -04:00
Jeremie Fraeys
c459285cab
chore(deploy): update deployment configs and TUI for scheduler
...
Update deployment and CLI tooling:
- TUI models (jobs, state) with scheduler data
- TUI store with scheduler endpoints
- TUI config with scheduler settings
- Deployment Makefile with scheduler targets
- Deploy script with scheduler registration
- Docker Compose files with scheduler services
- Remove obsolete Dockerfiles (api-server, full-prod, test)
- Update remaining Dockerfiles with scheduler integration
2026-02-26 12:08:31 -05:00
Jeremie Fraeys
7efe8bbfbf
native: security hardening, research trustworthiness, and CVE mitigations
...
Security Fixes:
- CVE-2024-45339: Add O_EXCL flag to temp file creation in storage_write_entries()
Prevents symlink attacks on predictable .tmp file paths
- CVE-2025-47290: Use openat_nofollow() in storage_open()
Closes TOCTOU race condition via path_sanitizer infrastructure
- CVE-2025-0838: Add MAX_BATCH_SIZE=10000 to add_tasks()
Prevents integer overflow in batch operations
Research Trustworthiness (dataset_hash):
- Deterministic file ordering: std::sort after collect_files()
- Recursive directory traversal: depth-limited with cycle detection
- Documented exclusions: hidden files and special files noted in API
Bug Fixes:
- R1: storage_init path validation for non-existent directories
- R2: safe_strncpy return value check before strcat
- R3: parallel_hash 256-file cap replaced with std::vector
- R4: wire qi_compact_index/qi_rebuild_index stubs
- R5: CompletionLatch race condition fix (hold mutex during decrement)
- R6: ARMv8 SHA256 transform fix (save abcd_pre before vsha256hq_u32)
- R7: fuzz_index_storage header format fix
- R8: enforce null termination in add_tasks/update_tasks
- R9: use 64 bytes (not 65) in combined hash to exclude null terminator
- R10: status field persistence in save()
New Tests:
- test_recursive_dataset.cpp: Verify deterministic recursive hashing
- test_storage_symlink_resistance.cpp: Verify CVE-2024-45339 fix
- test_queue_index_batch_limit.cpp: Verify CVE-2025-0838 fix
- test_sha256_arm_kat.cpp: ARMv8 known-answer tests
- test_storage_init_new_dir.cpp: F1 verification
- test_parallel_hash_large_dir.cpp: F3 verification
- test_queue_index_compact.cpp: F4 verification
All 8 native tests passing. Library ready for research lab deployment.
2026-02-21 13:33:45 -05:00
Jeremie Fraeys
a3b957dcc0
refactor(cli): Update build system and core infrastructure
...
- Makefile: Update build targets for native library integration
- build.zig: Add SQLite linking and native hash library support
- scripts/build_rsync.sh: Update rsync embedded binary build process
- scripts/build_sqlite.sh: Add SQLite constants generation script
- src/assets/README.md: Document embedded asset structure
- src/utils/rsync_embedded_binary.zig: Update for new build layout
2026-02-20 21:39:51 -05:00
Jeremie Fraeys
7c4a59012b
feat(tui): Add SQLite support for local mode
...
- store/store.go: New SQLite storage for TUI local mode
- Open() with WAL mode and NORMAL synchronous
- Schema initialization for ml_experiments, ml_runs, ml_metrics, ml_params, ml_tags
- GetUnsyncedRuns(), GetRunsByExperiment(), MarkRunSynced()
- GetRunMetrics(), GetRunParams() for run details
- config/config.go: Add local mode configuration fields
- DBPath, ForceLocal, ProjectRoot fields
- Experiment struct with Name and Entrypoint
- IsLocalMode() and GetDBPath() helper methods
- go.mod: Add modernc.org/sqlite v1.36.0 dependency
2026-02-20 21:28:49 -05:00
