jfraeysd/fetch_ml
0
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
245 commits 1 branch 0 tags 38 MiB
Commit graph

10 commits

Author SHA1 Message Date
Jeremie Fraeys
23e5f3d1dc
refactor(api): internal refactoring for TUI and worker modules 
- Refactor internal/worker and internal/queue packages
- Update cmd/tui for monitoring interface
- Update test configurations
 2026-02-20 15:51:23 -05:00
Jeremie Fraeys
6028779239
feat: update CLI, TUI, and security documentation 
- Add safety checks to Zig build
- Add TUI with job management and narrative views
- Add WebSocket support and export services
- Add smart configuration defaults
- Update API routes with security headers
- Update SECURITY.md with comprehensive policy
- Add Makefile security scanning targets
 2026-02-19 15:35:05 -05:00
Jeremie Fraeys
e4d286f2e5
feat: add security monitoring and validation framework 
- Implement anomaly detection monitor (brute force, path traversal, etc.)
- Add input validation framework with safety rules
- Add environment-based secrets manager with redaction
- Add security test suite for path traversal and injection
- Add CI security scanning workflow
 2026-02-19 15:34:25 -05:00
Jeremie Fraeys
4756348c48
feat: Worker sandboxing and security configuration 
Add security hardening features for worker execution:
- Worker config with sandboxing options (network_mode, read_only, secrets)
- Execution setup with security context propagation
- Podman container runtime security enhancements
- Security configuration management in config package
- Add homelab-sandbox.yaml example configuration

Supports running jobs in isolated, restricted environments.
 2026-02-18 21:27:59 -05:00
Jeremie Fraeys
e127f97442
chore: implement centralized path registry and file organization conventions 
Add PathRegistry for centralized path management:
- Create internal/config/paths.go with PathRegistry type
- Binary paths: BinDir(), APIServerBinary(), WorkerBinary(), etc.
- Data paths: DataDir(), JupyterStateDir(), ExperimentsDir()
- Config paths: ConfigDir(), APIServerConfig()
- Helper methods: EnsureDir(), EnsureDirSecure(), FileExists()
- Auto-detect repo root by looking for go.mod

Update .gitignore for root protection:
- Add explicit /api-server, /worker, /tui, /data_manager rules
- Add /coverage.out and .DS_Store to root protection
- Prevents accidental commits of binaries to root

Add path verification script:
- Create scripts/verify-paths.sh
- Checks for binaries in root directory
- Checks for .DS_Store files
- Checks for coverage.out in root
- Verifies data/ is gitignored
- Returns exit code 1 on violations

Cleaned .DS_Store files from repository
 2026-02-18 16:48:50 -05:00
Jeremie Fraeys
3775bc3ee0
refactor: replace panic with error returns and update maintenance 
- Replace 9 panic() calls in smart_defaults.go with error returns
- Add ErrUnknownProfile error type for better error handling
- Update all callers (worker/config.go, tui/config.go, tui/cli_config.go, tui/main.go)
- Update CHANGELOG.md with recent WebSocket handler improvements
- Add metrics persistence, dataset handlers, and test organization notes
- Config validation passes (make configlint)
- All tests pass (go test ./tests/unit/api/ws)
 2026-02-18 14:44:21 -05:00
Jeremie Fraeys
d1bef0a450
refactor: Phase 3 - fix config/storage boundaries 
Move schema ownership to infrastructure layer:

- Redis keys: config/constants.go -> queue/keys.go (TaskQueueKey, TaskPrefix, etc.)

- Filesystem paths: config/paths.go -> storage/paths.go (JobPaths)

- Create config/shared.go with RedisConfig, SSHConfig

- Update all imports: worker/, api/helpers, api/ws_jobs, api/ws_validate

- Clean up: remove duplicates from queue/task.go, queue/queue.go, config/paths.go

Build status: Compiles successfully
 2026-02-17 12:49:53 -05:00
Jeremie Fraeys
add4a90e62 feat(api): refactor websocket handlers; add health and prometheus middleware 2026-01-05 12:31:07 -05:00
Jeremie Fraeys
ea15af1833 Fix multi-user authentication and clean up debug code 
- Fix YAML tags in auth config struct (json -> yaml)
- Update CLI configs to use pre-hashed API keys
- Remove double hashing in WebSocket client
- Fix port mapping (9102 -> 9103) in CLI commands
- Update permission keys to use jobs:read, jobs:create, etc.
- Clean up all debug logging from CLI and server
- All user roles now authenticate correctly:
  * Admin: Can queue jobs and see all jobs
  * Researcher: Can queue jobs and see own jobs
  * Analyst: Can see status (read-only access)

Multi-user authentication is now fully functional.
 2025-12-06 12:35:32 -05:00
Jeremie Fraeys
803677be57 feat: implement Go backend with comprehensive API and internal packages 
- Add API server with WebSocket support and REST endpoints
- Implement authentication system with API keys and permissions
- Add task queue system with Redis backend and error handling
- Include storage layer with database migrations and schemas
- Add comprehensive logging, metrics, and telemetry
- Implement security middleware and network utilities
- Add experiment management and container orchestration
- Include configuration management with smart defaults
 2025-12-04 16:53:53 -05:00