jfraeysd/fetch_ml
0
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

docs: add Known Limitations section and testing structure updates

Browse source 
Add Known Limitations section to AGENTS.md documenting:
- AMD GPU not implemented (use NVIDIA, Apple Silicon, or CPU)
- 100+ node gang allocation stress testing not yet implemented
- Podman-in-Docker CI requires privileged mode, not yet automated
- Error handling patterns for unimplemented features
- Container usage rules (Docker for testing/deployments, Podman for experiments)
- Error codes table (NOT_IMPLEMENTED, NOT_FOUND, INVALID_CONFIGURATION)

Update testing documentation to reflect new test locations:
- Unit tests moved from tests/unit/ to internal/ (Go convention)
- Update all test file path references in security testing docs
This commit is contained in:
Jeremie Fraeys 2026-03-12 16:33:19 -04:00
parent 6646f3a382
commit b00fa236db
No known key found for this signature in database
10 changed files with 109 additions and 73 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
.forgejo/workflows/ci.yml
View file

@ -417,32 +417,32 @@ jobs:
- name: Run tests - ${{ matrix.build_config.name }}
run: |
echo "=== Testing ${{ matrix.build_config.name }} build (CGO_ENABLED=${{ matrix.build_config.cgo_enabled }}, tags=${{ matrix.build_config.tags }}) ==="
CGO_ENABLED=${{ matrix.build_config.cgo_enabled }} go test -tags "${{ matrix.build_config.tags }}" -v ./tests/unit/... || true
CGO_ENABLED=${{ matrix.build_config.cgo_enabled }} go test -tags "${{ matrix.build_config.tags }}" -v ./internal/... || true
- name: Run plugin quota tests
run: |
echo "=== Running Plugin GPU Quota tests ==="
go test -v ./tests/unit/scheduler/... -run TestPluginQuota
go test -v ./internal/scheduler/... -run TestPluginQuota
- name: Run service templates tests
run: |
echo "=== Running Service Templates tests ==="
go test -v ./tests/unit/scheduler/... -run TestServiceTemplate
go test -v ./internal/scheduler/... -run TestServiceTemplate
- name: Run scheduler tests
run: |
echo "=== Running Scheduler tests ==="
go test -v ./tests/unit/scheduler/... -run TestScheduler
go test -v ./internal/scheduler/... -run TestScheduler
- name: Run vLLM plugin tests
run: |
echo "=== Running vLLM Plugin tests ==="
go test -v ./tests/unit/worker/plugins/... -run TestVLLM
go test -v ./internal/worker/plugins/... -run TestVLLM
- name: Run audit tests
run: |
echo "=== Running Audit Logging tests ==="
go test -v ./tests/unit/security/... -run TestAudit
go test -v ./internal/security/... -run TestAudit
go test -v ./tests/integration/audit/...
build-trigger:

20
.forgejo/workflows/security-modes-test.yml
View file

@ -78,36 +78,36 @@ jobs:
if: matrix.security_mode == 'hipaa'
run: |
echo "=== Running HIPAA-specific validation tests ==="
go test -v ./tests/unit/security/... -run TestHIPAAValidation
go test -v ./internal/security/... -run TestHIPAAValidation
- name: Run PHI denylist tests
if: matrix.security_mode == 'hipaa'
run: |
echo "=== Running PHI denylist validation tests ==="
go test -v ./tests/unit/security/... -run TestPHIDenylist
go test -v ./internal/security/... -run TestPHIDenylist
- name: Run artifact ingestion cap tests
if: matrix.security_mode == 'hipaa'
run: |
echo "=== Running artifact ingestion cap tests ==="
go test -v ./tests/unit/security/... -run TestArtifactIngestionCaps
go test -v ./internal/security/... -run TestArtifactIngestionCaps
- name: Run config hash tests
if: matrix.security_mode == 'hipaa'
run: |
echo "=== Running config hash computation tests ==="
go test -v ./tests/unit/security/... -run TestConfigHash
go test -v ./internal/security/... -run TestConfigHash
- name: Run inline credential rejection tests
if: matrix.security_mode == 'hipaa'
run: |
echo "=== Running inline credential rejection tests ==="
go test -v ./tests/unit/security/... -run TestHIPAAValidation_InlineCredentials
go test -v ./internal/security/... -run TestHIPAAValidation_InlineCredentials
- name: Test config validation for ${{ matrix.security_mode }} mode
run: |
echo "=== Testing config validation for ${{ matrix.security_mode }} mode ==="
go test -v ./tests/unit/security/... || true
go test -v ./internal/security/... || true
- name: Verify compliance mode in config
run: |
@ -157,19 +157,19 @@ EOF
# which is tested by the unit tests above
# Check that the test covers all required validations
if grep -r "compliance_mode" tests/unit/security/hipaa*.go 2>/dev/null; then
if grep -r "compliance_mode" internal/security/hipaa*.go 2>/dev/null; then
echo "✓ compliance_mode validation is tested"
fi
if grep -r "network_mode" tests/unit/security/hipaa*.go 2>/dev/null; then
if grep -r "network_mode" internal/security/hipaa*.go 2>/dev/null; then
echo "✓ network_mode validation is tested"
fi
if grep -r "no_new_privileges" tests/unit/security/hipaa*.go 2>/dev/null; then
if grep -r "no_new_privileges" internal/security/hipaa*.go 2>/dev/null; then
echo "✓ no_new_privileges validation is tested"
fi
if grep -r "seccomp_profile" tests/unit/security/hipaa*.go 2>/dev/null; then
if grep -r "seccomp_profile" internal/security/hipaa*.go 2>/dev/null; then
echo "✓ seccomp_profile validation is tested"
fi

2
.forgejo/workflows/test-matrix.yml
View file

@ -32,7 +32,7 @@ jobs:
- name: Run unit tests
run: |
export FETCHML_NATIVE_LIBS=$([ "${{ matrix.build_type }}" = "native" ] && echo "1" || echo "0")
go test -v ./tests/unit/...
go test -v ./internal/...
- name: Run integration tests
run: |

2
.forgejo/workflows/verification.yml
View file

@ -78,7 +78,7 @@ jobs:
go-version: '1.25'
- name: Run audit chain verifier tests
run: go test ./tests/unit/audit/... -run TestChainVerifier -v
run: go test -v ./internal/audit/... -run TestChainVerifier -v
- name: Build audit verifier tool
run: go build -o bin/audit-verifier ./cmd/audit-verifier/

36
AGENTS.md
View file

@ -124,3 +124,39 @@ cd cli && zig fmt .
- Go 1.25+, Zig 0.15+, Python 3.11+
- Redis (integration tests), Docker/Podman (container tests)
---
## Known Limitations
See `docs/known-limitations.md` for full details.
**Key items**:
- **AMD GPU**: Not implemented. Use NVIDIA, Apple Silicon, or CPU. Mock available for testing.
- **100+ node gang allocation**: Stress testing not yet implemented.
- **Podman-in-Docker CI**: Requires privileged mode, not yet automated.
**Error Handling**:
```go
// For unimplemented features:
return apierrors.NewNotImplemented("feature name")
// Validation:
if err := detectionResult.Validate(); err != nil {
return err // Clear error message for user
}
```
**Container Rule Reminder**:
- Docker = testing & deployments
- Podman = experiment isolation only
---
## Error Codes
| Code | HTTP Status | Use Case |
|------|-------------|----------|
| `NOT_IMPLEMENTED` | 501 | Feature planned but not available |
| `NOT_FOUND` | 404 | Resource doesn't exist |
| `INVALID_CONFIGURATION` | 400 | Bad config (e.g., AMD GPU in production) |

2
DEVELOPMENT.md
View file

@ -89,7 +89,7 @@ make test
We maintain comprehensive test coverage across multiple categories:
- **Unit tests**: Fast tests for individual components (`tests/unit/`)
- **Unit tests**: Fast tests for individual components (`internal/`)
- **Integration tests**: Test component interactions (`tests/integration/`)
- **Property-based tests**: Verify invariants using gopter (`tests/property/`)
- **Fault injection tests**: Test failure scenarios (`tests/fault/`)

86
docs/TEST_COVERAGE_MAP.md
View file

@ -27,17 +27,17 @@ This document is a companion to the Security Plan and Verification Plan. It does
| Requirement | Test | Location | Status |
|---|---|---|---|
| Config file integrity / signature verification | `TestConfigIntegrityVerification` | `tests/unit/security/config_integrity_test.go` | `✓ Exists` — Tests config loading, signing, and tamper detection (lines 14-127) |
| `compliance_mode: hipaa` enforces network_mode | `TestHIPAAValidation_NetworkMode` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| `compliance_mode: hipaa` enforces no_new_privileges | `TestHIPAAValidation_NoNewPrivileges` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| `compliance_mode: hipaa` enforces seccomp_profile | `TestHIPAAValidation_SeccompProfile` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| `compliance_mode: hipaa` rejects inline credentials | `TestHIPAAValidation_InlineCredentials` | `tests/unit/security/hipaa_test.go` | `✓ Exists` — Now includes env var expansion verification for RedisPassword (lines 132-140) |
| `AllowedSecrets` PHI denylist enforced at `Validate()` | `TestPHIDenylist_Validation` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| Manifest filename includes nonce | `TestManifestFilenameNonce` | `tests/unit/security/manifest_filename_test.go` | `✓ Exists` — Verifies cryptographic nonce generation and filename pattern (lines 17-140) |
| Artifact ingestion file count cap | `TestArtifactIngestionCaps` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| Artifact ingestion total size cap | `TestArtifactIngestionCaps` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| GPU detection method logged at startup | `TestGPUDetectionAudit` | `tests/unit/security/gpu_audit_test.go` | `✓ Exists` — Verifies structured logging of GPU detection at startup (lines 14-160) |
| Resource env vars bounded by quota enforcement | `TestResourceEnvVarParsing` | `tests/unit/security/resource_quota_test.go` | `✓ Exists` — Tests env var parsing and override behavior (lines 11-183) |
| Config file integrity / signature verification | `TestConfigIntegrityVerification` | `internal/security/config_integrity_test.go` | `✓ Exists` — Tests config loading, signing, and tamper detection (lines 14-127) |
| `compliance_mode: hipaa` enforces network_mode | `TestHIPAAValidation_NetworkMode` | `internal/security/hipaa_test.go` | `✓ Exists` |
| `compliance_mode: hipaa` enforces no_new_privileges | `TestHIPAAValidation_NoNewPrivileges` | `internal/security/hipaa_test.go` | `✓ Exists` |
| `compliance_mode: hipaa` enforces seccomp_profile | `TestHIPAAValidation_SeccompProfile` | `internal/security/hipaa_test.go` | `✓ Exists` |
| `compliance_mode: hipaa` rejects inline credentials | `TestHIPAAValidation_InlineCredentials` | `internal/security/hipaa_test.go` | `✓ Exists` — Now includes env var expansion verification for RedisPassword (lines 132-140) |
| `AllowedSecrets` PHI denylist enforced at `Validate()` | `TestPHIDenylist_Validation` | `internal/security/hipaa_test.go` | `✓ Exists` |
| Manifest filename includes nonce | `TestManifestFilenameNonce` | `internal/security/manifest_filename_test.go` | `✓ Exists` — Verifies cryptographic nonce generation and filename pattern (lines 17-140) |
| Artifact ingestion file count cap | `TestArtifactIngestionCaps` | `internal/security/hipaa_test.go` | `✓ Exists` |
| Artifact ingestion total size cap | `TestArtifactIngestionCaps` | `internal/security/hipaa_test.go` | `✓ Exists` |
| GPU detection method logged at startup | `TestGPUDetectionAudit` | `internal/security/gpu_audit_test.go` | `✓ Exists` — Verifies structured logging of GPU detection at startup (lines 14-160) |
| Resource env vars bounded by quota enforcement | `TestResourceEnvVarParsing` | `internal/security/resource_quota_test.go` | `✓ Exists` — Tests env var parsing and override behavior (lines 11-183) |
---
@ -45,20 +45,20 @@ This document is a companion to the Security Plan and Verification Plan. It does
| Requirement | Test | Location | Status |
|---|---|---|---|
| R.1 — `manifest.Artifacts.Environment` populated on every scan | `TestManifestEnvironmentCapture` | `tests/unit/reproducibility/environment_capture_test.go` | `✓ Exists` — Tests Environment population with ConfigHash and DetectionMethod (lines 15-127) |
| R.1 — `Environment.ConfigHash` non-empty | `TestManifestEnvironmentCapture` | `tests/unit/reproducibility/environment_capture_test.go` | `✓ Exists` — Verified in EnvironmentPopulatedInManifest subtest (line 58) |
| R.1 — `Environment.DetectionMethod` non-empty | `TestManifestEnvironmentCapture` | `tests/unit/reproducibility/environment_capture_test.go` | `✓ Exists` — Verified in EnvironmentPopulatedInManifest subtest (line 63) |
| R.2 — Resolved config hash stable (same input → same hash) | `TestConfigHash_Computation` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| R.2 — Resolved config hash differs on changed input | `TestConfigHash_Computation` | `tests/unit/security/hipaa_test.go` | `✓ Exists` |
| R.2 — Hash computed after defaults and env expansion, not raw file | `TestConfigHashPostDefaults` | `tests/unit/reproducibility/config_hash_test.go` | `✓ Exists` — Tests hash computation after env expansion and defaults (lines 14-118) |
| R.3 — `CreateDetectorWithInfo` result written to manifest | `TestGPUDetectionWrittenToManifest` | `tests/unit/reproducibility/` | `✓ Exists` — **Covered by:** `TestAMDAliasManifestRecord` in `tests/unit/gpu/gpu_detector_test.go` tests GPU detection and manifest recording (lines 87-138) |
| R.3 — AMD alias recorded as `configured_vendor` in manifest | `TestAMDAliasManifestRecord` | `tests/unit/gpu/gpu_detector_test.go` | `✓ Exists` — Test renamed and enhanced with manifest recording validation (line 87-138) |
| R.4 — `ProvenanceBestEffort=false` fails on incomplete environment | `TestProvenanceBestEffortEnforcement` | `tests/unit/reproducibility/` | `✓ Exists` — Covered by `TestEnforceTaskProvenance_StrictMissingOrMismatchFails` in `tests/unit/worker/worker_test.go` |
| R.4 — `ProvenanceBestEffort=true` succeeds on incomplete environment | `TestProvenanceBestEffortPermissive` | `tests/unit/reproducibility/` | `✓ Exists` — Covered by `TestEnforceTaskProvenance_BestEffortOverwrites` in `tests/unit/worker/worker_test.go` |
| R.5 — Scan exclusions recorded in manifest | `TestScanExclusionsRecorded` | `tests/unit/worker/artifacts_test.go` | `✓ Exists` — Renamed from TestScanArtifacts_SkipsKnownPathsAndLogs, validates exclusions recorded with reasons (lines 71-116) |
| R.5 — `*.log` exclusion reason recorded | `TestScanExclusionsRecorded` | `tests/unit/worker/artifacts_test.go` | `✓ Exists` — Verified in exclusion reason check (line 85) |
| R.5 — `code/` exclusion reason recorded | `TestScanExclusionsRecorded` | `tests/unit/worker/artifacts_test.go` | `✓ Exists` — Verified in exclusion reason check (line 87) |
| R.5 — `snapshot/` exclusion reason recorded | `TestScanExclusionsRecorded` | `tests/unit/worker/artifacts_test.go` | `✓ Exists` — Verified in exclusion reason check (line 89) |
| R.1 — `manifest.Artifacts.Environment` populated on every scan | `TestManifestEnvironmentCapture` | `internal/tracking/environment_capture_test.go` | `✓ Exists` — Tests Environment population with ConfigHash and DetectionMethod (lines 15-127) |
| R.1 — `Environment.ConfigHash` non-empty | `TestManifestEnvironmentCapture` | `internal/tracking/environment_capture_test.go` | `✓ Exists` — Verified in EnvironmentPopulatedInManifest subtest (line 58) |
| R.1 — `Environment.DetectionMethod` non-empty | `TestManifestEnvironmentCapture` | `internal/tracking/environment_capture_test.go` | `✓ Exists` — Verified in EnvironmentPopulatedInManifest subtest (line 63) |
| R.2 — Resolved config hash stable (same input → same hash) | `TestConfigHash_Computation` | `internal/security/hipaa_test.go` | `✓ Exists` |
| R.2 — Resolved config hash differs on changed input | `TestConfigHash_Computation` | `internal/security/hipaa_test.go` | `✓ Exists` |
| R.2 — Hash computed after defaults and env expansion, not raw file | `TestConfigHashPostDefaults` | `internal/tracking/config_hash_test.go` | `✓ Exists` — Tests hash computation after env expansion and defaults (lines 14-118) |
| R.3 — `CreateDetectorWithInfo` result written to manifest | `TestGPUDetectionWrittenToManifest` | `internal/tracking/` | `✓ Exists` — **Covered by:** `TestAMDAliasManifestRecord` in `internal/resources/gpu_detector_test.go` tests GPU detection and manifest recording (lines 87-138) |
| R.3 — AMD alias recorded as `configured_vendor` in manifest | `TestAMDAliasManifestRecord` | `internal/resources/gpu_detector_test.go` | `✓ Exists` — Test renamed and enhanced with manifest recording validation (line 87-138) |
| R.4 — `ProvenanceBestEffort=false` fails on incomplete environment | `TestProvenanceBestEffortEnforcement` | `internal/tracking/` | `✓ Exists` — Covered by `TestEnforceTaskProvenance_StrictMissingOrMismatchFails` in `internal/worker/worker_test.go` |
| R.4 — `ProvenanceBestEffort=true` succeeds on incomplete environment | `TestProvenanceBestEffortPermissive` | `internal/tracking/` | `✓ Exists` — Covered by `TestEnforceTaskProvenance_BestEffortOverwrites` in `internal/worker/worker_test.go` |
| R.5 — Scan exclusions recorded in manifest | `TestScanExclusionsRecorded` | `internal/worker/artifacts_test.go` | `✓ Exists` — Renamed from TestScanArtifacts_SkipsKnownPathsAndLogs, validates exclusions recorded with reasons (lines 71-116) |
| R.5 — `*.log` exclusion reason recorded | `TestScanExclusionsRecorded` | `internal/worker/artifacts_test.go` | `✓ Exists` — Verified in exclusion reason check (line 85) |
| R.5 — `code/` exclusion reason recorded | `TestScanExclusionsRecorded` | `internal/worker/artifacts_test.go` | `✓ Exists` — Verified in exclusion reason check (line 87) |
| R.5 — `snapshot/` exclusion reason recorded | `TestScanExclusionsRecorded` | `internal/worker/artifacts_test.go` | `✓ Exists` — Verified in exclusion reason check (line 89) |
---
@ -68,7 +68,7 @@ This document is a companion to the Security Plan and Verification Plan. It does
|---|---|---|---|
| `manifest.Artifacts` schema matches committed version | `TestSchemaUnchanged` | `internal/manifest/schema_test.go` | `✓ Exists` |
| `Environment` field required in schema | `TestSchemaEnvironmentRequired` | `internal/manifest/` | `✓ Exists` — **Covered by:** `TestSchemaRejectsInvalidManifest` in `internal/manifest/schema_test.go` validates missing `environment.config_hash` is rejected |
| `DetectionMethod` constrained to enum values in schema | `TestSchemaDetectionMethodEnum` | `tests/unit/manifest/schema_test.go` | `✓ Exists` — **Covered by:** `TestSchemaRejectsInvalidManifest` validates `compliance_mode` enum; `gpu_detection_method` validated in environment capture tests |
| `DetectionMethod` constrained to enum values in schema | `TestSchemaDetectionMethodEnum` | `internal/manifest/schema_test.go` | `✓ Exists` — **Covered by:** `TestSchemaRejectsInvalidManifest` validates `compliance_mode` enum; `gpu_detection_method` validated in environment capture tests |
---
@ -92,7 +92,7 @@ Not tests themselves — packages and targets that must achieve >80% mutation ki
| `pkg/worker/config.go` | `ProvenanceBestEffort` enforcement branch, HIPAA hard-requirement checks, credential denylist |
| `pkg/worker/gpu_detector.go` | `CreateDetectorWithInfo` call site, `DetectionInfo` capture |
| `internal/manifest/` | `Environment` nil check, `Exclusions` population, schema version check |
| `tests/unit/security/` | PHI denylist logic, inline credential detection |
| `internal/security/` | PHI denylist logic, inline credential detection |
---
@ -113,8 +113,8 @@ Not tests — static analysis rules enforced at compile time in CI. All four mus
| Requirement | Test | Location | Status |
|---|---|---|---|
| Chained hash detects tampered entry | `TestAuditChainTamperDetection` | `tests/unit/security/audit_test.go` | `✓ Exists` — **Covered by:** `TestAuditLogger_VerifyChain` validates tamper detection (lines 89-100) |
| Chained hash detects deleted entry | `TestAuditChainDeletionDetection` | `tests/unit/security/audit_test.go` | `✓ Exists` — **Covered by:** `TestAuditLogger_VerifyChain` validates chain break detection via `prev_hash` mismatch (lines 102-113) |
| Chained hash detects tampered entry | `TestAuditChainTamperDetection` | `internal/security/audit_test.go` | `✓ Exists` — **Covered by:** `TestAuditLogger_VerifyChain` validates tamper detection (lines 89-100) |
| Chained hash detects deleted entry | `TestAuditChainDeletionDetection` | `internal/security/audit_test.go` | `✓ Exists` — **Covered by:** `TestAuditLogger_VerifyChain` validates chain break detection via `prev_hash` mismatch (lines 102-113) |
| Background verification job alerts on chain break | `TestAuditVerificationJob` | `tests/integration/audit/verification_test.go` | `✓ Exists` — Integration test for audit chain verification (lines 14-126) |
---
@ -165,15 +165,15 @@ The following tests exist but use different naming conventions than specified in
| Coverage Map Name | Actual Test Name | Location | Relationship |
|---|---|---|---|
| `TestGPUDetectionAudit` | `TestGPUDetectorEnvOverrides`, `TestGPUDetectorDetectionSources`, `TestGPUDetectorInfoFields` | `tests/unit/gpu/gpu_detector_test.go` | Tests GPU detection but not audit logging |
| `TestAMDAliasManifestRecord` | `TestGPUDetectorAMDVendorAlias` | `tests/unit/gpu/gpu_detector_test.go` | Tests AMD vendor aliasing but not manifest recording |
| `TestGPUDetectionAudit` | `TestGPUDetectorEnvOverrides`, `TestGPUDetectorDetectionSources`, `TestGPUDetectorInfoFields` | `internal/resources/gpu_detector_test.go` | Tests GPU detection but not audit logging |
| `TestAMDAliasManifestRecord` | `TestGPUDetectorAMDVendorAlias` | `internal/resources/gpu_detector_test.go` | Tests AMD vendor aliasing but not manifest recording |
| `TestGPUDetectionWrittenToManifest` | N/A - uses same tests as above | - | GPU detection tests don't verify manifest writing |
| `TestProvenanceBestEffortEnforcement` | `TestEnforceTaskProvenance_StrictMissingOrMismatchFails` | `tests/unit/worker/worker_test.go` | Tests strict provenance enforcement |
| `TestProvenanceBestEffortPermissive` | `TestEnforceTaskProvenance_BestEffortOverwrites` | `tests/unit/worker/worker_test.go` | Tests best-effort provenance behavior |
| `TestScanExclusionsRecorded` | `TestScanArtifacts_SkipsKnownPathsAndLogs` | `tests/unit/worker/artifacts_test.go` | Tests scan exclusions but not manifest recording |
| `TestProvenanceBestEffortEnforcement` | `TestEnforceTaskProvenance_StrictMissingOrMismatchFails` | `internal/worker/worker_test.go` | Tests strict provenance enforcement |
| `TestProvenanceBestEffortPermissive` | `TestEnforceTaskProvenance_BestEffortOverwrites` | `internal/worker/worker_test.go` | Tests best-effort provenance behavior |
| `TestScanExclusionsRecorded` | `TestScanArtifacts_SkipsKnownPathsAndLogs` | `internal/worker/artifacts_test.go` | Tests scan exclusions but not manifest recording |
| `TestSandboxSyscallBlocking` | `TestSandboxSeccompEnforcement` | `tests/integration/security/sandbox_escape_test.go` | Tests seccomp syscall blocking |
| `TestAuditChainTamperDetection` | `TestAuditLogger_VerifyChain` (tamper portion) | `tests/unit/security/audit_test.go` | Lines 89-100 test tamper detection |
| `TestAuditChainDeletionDetection` | `TestAuditLogger_VerifyChain` (chain break portion) | `tests/unit/security/audit_test.go` | Lines 102-113 test prev_hash mismatch |
| `TestAuditChainTamperDetection` | `TestAuditLogger_VerifyChain` (tamper portion) | `internal/security/audit_test.go` | Lines 89-100 test tamper detection |
| `TestAuditChainDeletionDetection` | `TestAuditLogger_VerifyChain` (chain break portion) | `internal/security/audit_test.go` | Lines 102-113 test prev_hash mismatch |
| `TestSchemaEnvironmentRequired` | `TestSchemaRejectsInvalidManifest` (portion) | `internal/manifest/schema_test.go` | Tests missing environment.config_hash rejection |
---
@ -184,11 +184,11 @@ These tests exist and provide related functionality testing, but don't fully cov
| Requirement Area | Related Tests | Location | Gap |
|---|---|---|---|
| GPU Detection | `TestGPUDetectorEnvOverrides`, `TestGPUDetectorAMDVendorAlias`, `TestGPUDetectorDetectionSources`, `TestGPUDetectorInfoFields`, `TestGPUDetectorEnvCountOverride` | `tests/unit/gpu/gpu_detector_test.go` | No manifest writing validation; no startup audit logging |
| Artifact Scanning | `TestScanArtifacts_SkipsKnownPathsAndLogs` | `tests/unit/worker/artifacts_test.go` | No `Environment` population check; no exclusion reason recording in manifest |
| Provenance | `TestEnforceTaskProvenance_StrictMissingOrMismatchFails`, `TestEnforceTaskProvenance_BestEffortOverwrites`, `TestComputeTaskProvenance` | `tests/unit/worker/worker_test.go` | Different test structure than coverage map specifies |
| GPU Detection | `TestGPUDetectorEnvOverrides`, `TestGPUDetectorAMDVendorAlias`, `TestGPUDetectorDetectionSources`, `TestGPUDetectorInfoFields`, `TestGPUDetectorEnvCountOverride` | `internal/resources/gpu_detector_test.go` | No manifest writing validation; no startup audit logging |
| Artifact Scanning | `TestScanArtifacts_SkipsKnownPathsAndLogs` | `internal/worker/artifacts_test.go` | No `Environment` population check; no exclusion reason recording in manifest |
| Provenance | `TestEnforceTaskProvenance_StrictMissingOrMismatchFails`, `TestEnforceTaskProvenance_BestEffortOverwrites`, `TestComputeTaskProvenance` | `internal/worker/worker_test.go` | Different test structure than coverage map specifies |
| Schema Validation | `TestSchemaValidatesExampleManifest`, `TestSchemaRejectsInvalidManifest` | `internal/manifest/schema_test.go` | Exist and provide good coverage |
| Manifest | `TestRunManifestWriteLoadAndMarkFinished`, `TestRunManifestApplyNarrativePatchPartialUpdate` | `tests/unit/manifest/run_manifest_test.go` | Basic manifest operations tested |
| Manifest | `TestRunManifestWriteLoadAndMarkFinished`, `TestRunManifestApplyNarrativePatchPartialUpdate` | `internal/manifest/run_manifest_test.go` | Basic manifest operations tested |
| Sandbox Security | `TestSandboxCapabilityDrop`, `TestSandboxNoNewPrivileges`, `TestSandboxSeccompEnforcement`, `TestSandboxNetworkIsolation`, `TestSandboxFilesystemEscape` | `tests/integration/security/sandbox_escape_test.go` | Comprehensive sandbox tests exist |
---
@ -252,9 +252,9 @@ Work through gaps in this order:
- `TestConfigHashPostDefaults` - Hash computation after env expansion and defaults
### Files Modified
- `tests/unit/gpu/gpu_detector_test.go`
- `tests/unit/worker/artifacts_test.go`
- `tests/unit/security/hipaa_validation_test.go`
- `internal/resources/gpu_detector_test.go`
- `internal/worker/artifacts_test.go`
- `internal/security/hipaa_validation_test.go`
- `internal/worker/artifacts.go` (added exclusions recording)
- `internal/manifest/run_manifest.go` (nonce-based filename support)
- 6 new test files created

2
docs/src/scheduler-architecture.md
View file

@ -146,7 +146,7 @@ All tests use shared fixtures in `tests/fixtures/`:
| Category | Count | Files |
|----------|-------|-------|
| Unit | 17+ | `tests/unit/scheduler/` |
| Unit | 17+ | `internal/scheduler/` |
| Integration | 6 | `tests/integration/scheduler/` |
| E2E | 6 | `tests/e2e/scheduler/` |

18
docs/src/testing.md
View file

@ -65,8 +65,8 @@ docker exec ml-experiments-redis redis-cli ping
make test-unit # Go unit tests only
cd cli && zig build test # Zig CLI tests
# Unit tests live under tests/unit/ (including tests that cover internal/ packages)
go test ./tests/unit/...
# Unit tests now live alongside source code in internal/ (following Go conventions)
go test ./internal/...
```
### Integration Tests
@ -479,7 +479,7 @@ FetchML implements comprehensive security testing with **100% coverage** across
make test
# Run security-specific unit tests
go test -v ./tests/unit/security/...
go test -v ./internal/security/...
# Run audit verification tests
go test -v ./tests/integration/audit/...
@ -493,13 +493,13 @@ go test -v ./tests/property/...
### Security Test Files
- `tests/unit/security/path_traversal_test.go` - Path traversal prevention
- `tests/unit/security/filetype_test.go` - Magic bytes validation
- `tests/unit/security/secrets_test.go` - Environment expansion & secret detection
- `tests/unit/security/audit_test.go` - Audit chain integrity
- `tests/unit/security/config_integrity_test.go` - Config validation
- `internal/security/path_traversal_test.go` - Path traversal prevention
- `internal/security/filetype_test.go` - Magic bytes validation
- `internal/security/secrets_test.go` - Environment expansion & secret detection
- `internal/security/audit_test.go` - Audit chain integrity
- `internal/security/config_integrity_test.go` - Config validation
- `tests/integration/security/cross_tenant_test.go` - Tenant isolation
- `tests/integration/audit/verification_test.go` - Audit verification
- `internal/audit/verifier_test.go` - Unit testsrification
- `tests/property/*_test.go` - Property-based testing
- `tests/fault/fault_test.go` - Fault injection scenarios

2
docs/src/verification.md
View file

@ -82,7 +82,7 @@ type Event struct {
**Components:**
- `internal/audit/verifier.go` - Chain verification logic
- `cmd/audit-verifier/main.go` - Standalone CLI tool
- `tests/unit/audit/verifier_test.go` - Unit tests
- `internal/audit/verifier_test.go` - Unit tests
**Features:**
- **Continuous verification:** Background job runs every 15 minutes (HIPAA) or hourly (other)