fetch_ml

jfraeysd/fetch_ml

Fork 0

Commit graph

Author	SHA1	Message	Date
Jeremie Fraeys	7880ea8d79	refactor: reorganize podman directory structure Organize podman/ directory into logical subdirectories: New structure: - docs/ - ML_TOOLS_GUIDE.md, jupyter_workflow.md - configs/ - environment.yml, security_policy.json - containers/ - .dockerfile, .podfile - scripts/ - .sh, *.py (secure_runner, cli_integration, etc.) - jupyter/ - jupyter_cookie_secret (flattened from jupyter_runtime/runtime/) - workspace/ - Example projects (cleaned of temp files) Cleaned workspace: - Removed .DS_Store, mlflow.db, cache/ - Removed duplicate cli_integration.py Removed unnecessary nesting: - Flattened jupyter_runtime/runtime/ to just jupyter/ Improves maintainability by grouping files by purpose and eliminating root directory clutter.	2026-02-18 16:40:46 -05:00
Jeremie Fraeys	8271277dc3	feat: implement research-grade maintainability phases 2, 5, 8, 10 Phase 2: Deterministic Manifests - Add manifest.Validator with required field checking - Support Validate() and ValidateStrict() modes - Integrate validation into worker executor before execution - Block execution if manifest missing commit_id or deps_manifest_sha256 Phase 5: Pinned Dependencies - Add hermetic.dockerfile template with pinned system deps - Frozen package versions: libblas3, libcudnn8, etc. - Support for deps_manifest.json and requirements.txt with hashes - Image tagging strategy: deps-<first-8-of-sha256> Phase 8: Tests as Specifications - Add queue_spec_test.go with executable scheduler specs - Document priority ordering (higher first) - Document FIFO tiebreaker for same priority - Test cases for negative/zero priorities Phase 10: Local Dev Parity - Create root-level docker-compose.dev.yml - Simplified from deployments/ for quick local dev - Redis + API server + Worker with hot reload volumes - Debug ports: 9101 (API), 6379 (Redis)	2026-02-18 15:34:28 -05:00

Author

SHA1

Message

Date

Jeremie Fraeys

7880ea8d79

refactor: reorganize podman directory structure

Organize podman/ directory into logical subdirectories:

New structure:
- docs/          - ML_TOOLS_GUIDE.md, jupyter_workflow.md
- configs/       - environment*.yml, security_policy.json
- containers/    - *.dockerfile, *.podfile
- scripts/       - *.sh, *.py (secure_runner, cli_integration, etc.)
- jupyter/       - jupyter_cookie_secret (flattened from jupyter_runtime/runtime/)
- workspace/     - Example projects (cleaned of temp files)

Cleaned workspace:
- Removed .DS_Store, mlflow.db, cache/
- Removed duplicate cli_integration.py

Removed unnecessary nesting:
- Flattened jupyter_runtime/runtime/ to just jupyter/

Improves maintainability by grouping files by purpose and eliminating root directory clutter.

2026-02-18 16:40:46 -05:00

Jeremie Fraeys

8271277dc3

feat: implement research-grade maintainability phases 2, 5, 8, 10

Phase 2: Deterministic Manifests
- Add manifest.Validator with required field checking
- Support Validate() and ValidateStrict() modes
- Integrate validation into worker executor before execution
- Block execution if manifest missing commit_id or deps_manifest_sha256

Phase 5: Pinned Dependencies
- Add hermetic.dockerfile template with pinned system deps
- Frozen package versions: libblas3, libcudnn8, etc.
- Support for deps_manifest.json and requirements.txt with hashes
- Image tagging strategy: deps-<first-8-of-sha256>

Phase 8: Tests as Specifications
- Add queue_spec_test.go with executable scheduler specs
- Document priority ordering (higher first)
- Document FIFO tiebreaker for same priority
- Test cases for negative/zero priorities

Phase 10: Local Dev Parity
- Create root-level docker-compose.dev.yml
- Simplified from deployments/ for quick local dev
- Redis + API server + Worker with hot reload volumes
- Debug ports: 9101 (API), 6379 (Redis)

2026-02-18 15:34:28 -05:00

2 commits