Jeremie Fraeys
|
3248279c01
|
refactor: Phase 3 - Extract data integrity layer
Created integrity package with extracted data utilities:
1. internal/worker/integrity/hash.go (113 lines)
- FileSHA256Hex() - SHA256 hash of single file
- NormalizeSHA256ChecksumHex() - Checksum normalization
- DirOverallSHA256Hex() - Directory hash (sequential)
- DirOverallSHA256HexParallel() - Directory hash (parallel workers)
2. internal/worker/integrity/validate.go (76 lines)
- DatasetVerifier type for dataset validation
- VerifyDatasetSpecs() method for checksum validation
- ProvenanceCalculator type for provenance computation
- ComputeProvenance() method for task provenance
Note: Used 'integrity' instead of 'data' due to .gitignore conflict
(data/ directory is ignored for experiment artifacts)
Functions extracted from data_integrity.go:
- fileSHA256Hex → FileSHA256Hex
- normalizeSHA256ChecksumHex → NormalizeSHA256ChecksumHex
- dirOverallSHA256HexGo → DirOverallSHA256Hex
- dirOverallSHA256HexParallel → DirOverallSHA256HexParallel
- verifyDatasetSpecs logic → DatasetVerifier
- computeTaskProvenance logic → ProvenanceCalculator
Build status: Compiles successfully
|
2026-02-17 14:20:41 -05:00 |
|