chore: add AI assistant files to .gitignore, update AGENTS.md

- Ignore AGENTS.md and .windsurf/* in .gitignore

- Expand AGENTS.md with native lib and Zig CLI build commands

.gitignore

@@ -263,7 +263,7 @@ Launch_*.trace/
 # Native C++ build artifacts
 native/build/
 
-# Build artifacts  
+# Build artifacts
 bin/
 cli/zig-out/
 cli/.zig-cache/
@@ -287,3 +287,8 @@ db/*.db
 ssl/
 *.pem
 *.key
+
+# AI assitant files
+AGENTS.md
+.windsurf/*
+

AGENTS.md

@@ -6,39 +6,56 @@ This document provides essential information for agentic coding assistants worki
 
 ### Go Components
 ```bash
-# Build all Go components
+# Build all components (Go binaries + native libs + Zig CLI)
 make build
 
-# Build for development (faster compilation)
+# Build for development (faster compilation, no LTO)
 make dev
 
-# Build for production (optimized)
+# Build production-optimized binaries
 make prod
 
+# Build production with native C++ libraries
+make prod-with-native
+
 # Cross-platform build for Linux/macOS/Windows
 make cross-platform
+
+# Build native C++ libraries
+make native-build
+make native-release
+make native-debug
 ```
 
 ### Zig CLI
 ```bash
-# Build CLI with release-small optimization
+# Build CLI with release-small optimization (smallest binary)
+make build-cli
 cd cli && make all
+cd cli && zig build --release=small
 
-# Build for development
-cd cli && make dev
-
-# Build for production
+# Build for production (fast, LTO)
 cd cli && make prod
+cd cli && zig build --release=fast
 
-# Alternative using zig build system
-cd cli && zig build
+# Build for development (fast compilation)
+cd cli && make dev
+cd cli && zig build --release=fast
+
+# Debug build (fastest compile, no optimizations)
+cd cli && make debug
+cd cli && zig build -Doptimize=Debug
+
+# Run Zig tests
+cd cli && make test
+cd cli && zig build test
 ```
 
 ## Test Commands
 
 ### Running Tests
 ```bash
-# Run all tests
+# Run all tests with Docker infrastructure
 make test
 
 # Run unit tests only
@@ -47,8 +64,9 @@ make test-unit
 # Run integration tests only
 make test-integration
 
-# Run end-to-end tests only
+# Run end-to-end tests only (Podman tests opt-in)
 make test-e2e
+FETCH_ML_E2E_PODMAN=1 go test ./tests/e2e/...
 
 # Run with coverage report
 make test-coverage
@@ -58,12 +76,20 @@ go test -v ./path/to/package -run TestName
 
 # Run tests with race detector
 go test -race ./path/to/package/...
+
+# Run specific test file
+go test -v ./tests/unit/auth/keychain_test.go
+
+# Run tests with logging
+LOG_LEVEL=debug go test -v ./path/to/package
 ```
 
 ### Performance Testing
 ```bash
 # Run performance benchmarks
 make benchmark
+make benchmark-local
+make benchmark-native
 
 # Run load testing suite
 make load-test
@@ -72,7 +98,15 @@ make load-test
 make chaos-test
 
 # Run complete technical excellence suite
-make tech-excellence
+make complete-suite
+
+# Detect performance regressions
+make detect-regressions
+make profile-tools
+
+# CPU profiling
+make profile-load
+make profile-ws-queue
 ```
 
 ## Lint Commands
@@ -81,21 +115,35 @@ make tech-excellence
 # Run all linters and formatters
 make lint
 
-# Run Go linters
+# Run Go linters with golangci-lint
 golangci-lint run
+golangci-lint run ./path/to/package
 
 # Run Go formatters
 gofmt -w ./cmd ./internal ./tests
+goimports -w ./cmd ./internal ./tests
 
 # Run Go vet
 go vet ./...
 
 # Format Zig code
 cd cli && zig fmt .
+cd cli && make dev  # includes formatting
 
 # Validate YAML configs against schema
 make configlint
 make worker-configlint
+
+# Security scans
+make security-scan
+make gosec
+make govulncheck
+make check-unsafe
+
+# OpenAPI validation
+make openapi-validate
+make openapi-generate
+make openapi-generate-server
 ```
 
 ## Code Style Guidelines
@@ -103,56 +151,89 @@ make worker-configlint
 ### Go Code Style
 
 1. **Formatting**:
-   - Use `gofmt` for formatting
-   - Line length: 100 characters (configured in .golangci.yml)
+   - Use `gofmt` for formatting (configured in .golangci.yml)
+   - Line length: 100 characters
    - Use `goimports` for import organization
+   - Run `make lint` to format all code
 
 2. **Naming Conventions**:
    - Use camelCase for variables and functions
-   - Use PascalCase for exported identifiers
+   - Use PascalCase for exported identifiers (types, interfaces, structs)
    - Use clear, descriptive names
-   - Acronyms should be all caps (e.g., HTTP, URL)
+   - Acronyms should be all caps (e.g., HTTP, URL, API, JSON)
+   - Interface names: typically end in "er" (e.g., Manager, Handler)
 
 3. **Imports**:
    - Group imports in order: standard library, third-party, internal
-   - Use meaningful import aliases when needed
-   - Remove unused imports
+   - Use meaningful import aliases when needed (e.g., `jupyter "github.com/..."`)
+   - Remove unused imports (golangci-lint will catch these)
+   - Internal imports: `github.com/jfraeys/fetch_ml/internal/...`
 
 4. **Error Handling**:
    - Always handle errors explicitly
    - Use `errors.Is` and `errors.As` for error checking
    - Wrap errors with context using `fmt.Errorf("context: %w", err)`
+   - Don't ignore errors with `_` unless explicitly documented
+   - Return errors early when appropriate (guard clauses)
 
 5. **Documentation**:
    - All exported functions, types, and variables must have doc comments
    - Comments should explain "why" not just "what"
    - Use godoc formatting conventions
+   - Include examples for complex functions
+   - Document error conditions and edge cases
+
+6. **Testing**:
+   - Use `t.Parallel()` for independent tests
+   - Skip tests conditionally with `t.Skip()` rather than failing
+   - Test files: `*_test.go` in same package directory
+   - Unit tests: `tests/unit/`
+   - Integration tests: `tests/integration/`
+   - E2E tests: `tests/e2e/`
+   - Benchmarks: `tests/benchmarks/`
 
 ### Zig Code Style
 
 1. **Formatting**:
-   - Use built-in Zig formatter (`zig fmt`)
+   - Use built-in Zig formatter (`zig fmt .` in cli/)
    - Consistent indentation with 4 spaces
+   - Run `make lint` to format all Zig code
 
 2. **Naming Conventions**:
    - Use camelCase for variables and functions
-   - Use PascalCase for types
-   - Use snake_case for file names
+   - Use PascalCase for types (structs, enums, unions)
+   - Use snake_case for file names (e.g., `utils.zig`, `colors.zig`)
+   - Public functions: `pub fn`
+   - Internal functions: `fn`
 
 3. **Error Handling**:
-   - Use Zig's error union types (`!T`)
+   - Use Zig's error union types (`!T` or `Error!T`)
    - Handle errors with `try`, `catch`, or `if` expressions
    - Return errors early when appropriate
+   - Use `catch unreachable` only when error is truly impossible
+   - Document error returns in function docs
+
+4. **Imports**:
+   - Use `@import("module")` for modules
+   - Organize imports logically
+   - Remove unused imports
+
+5. **Testing**:
+   - Tests in `cli/tests/` directory
+   - Use `@import("std").testing` for test utilities
+   - Run with `cd cli && zig build test`
 
 ### Python Code Style
 
 1. **Formatting**:
    - Use Black formatter with 80 character line limit
    - Use isort for import organization (Google style)
+   - Run `black .` and `isort .` in project root
 
 2. **Type Checking**:
    - Use mypy with strict settings
    - All functions should have type annotations
+   - Use `from __future__ import annotations` for forward references
 
 3. **Testing**:
    - Use pytest framework
@@ -161,9 +242,9 @@ make worker-configlint
 ### YAML/Configuration Style
 
 1. **Formatting**:
-   - Use yamllint for validation
-   - Consistent indentation with 2 spaces
+   - Use 2 spaces for indentation
    - No trailing whitespace
+   - Use yamllint for validation
 
 2. **Validation**:
    - All configs must pass schema validation
@@ -172,16 +253,21 @@ make worker-configlint
 ## Development Guidelines
 
 ### Testing Requirements (from .windsurf/rules/test-new-features.md)
-- Every new feature MUST include corresponding tests
+- MANDATORY: Every new feature MUST include corresponding tests
+- Write tests BEFORE implementing complex features (TDD approach)
 - Test coverage for new code should be >80%
 - Include both unit tests and integration tests where applicable
 - Test edge cases, error paths, and boundary conditions
 
 ### Code Organization
-- Clean up as you go - no orphaned files or dead code
-- Remove commented-out code blocks
+- CRITICAL: Clean up as you go - no orphaned files or dead code
+- Remove commented-out code blocks (use git history instead)
 - Delete unused imports, functions, and variables immediately
 - Consolidate duplicate code into reusable functions
+- Move TODO items from loose files into:
+  - Code comments with `// TODO(context):` for implementation tasks
+  - GitHub Issues for larger features
+  - NEVER create standalone .md files for tracking
 
 ### Documentation Standards
 - Update relevant documentation IN THE SAME COMMIT as code changes
@@ -190,6 +276,63 @@ make worker-configlint
   - CHANGELOG.md: All changes, following Keep a Changelog format
   - Code comments: Complex logic, non-obvious decisions, API contracts
   - Function/struct docs: Public APIs must have doc comments
+- Use concrete examples in documentation
+- Keep docs concise but complete
+
+### When Making Changes
+For EVERY significant change, complete ALL of these:
+
+1. Write/update tests
+2. Update documentation (README, CHANGELOG, code comments)
+3. Update build scripts if dependencies/build process changed
+4. Remove any temporary/debug code added during development
+5. Delete unused files created during exploration
+6. Verify no dead code remains (unused functions, imports, variables)
+
+### Cleanup Checklist (Run BEFORE committing)
+- [ ] Removed all debug print statements
+- [ ] Deleted temporary test files
+- [ ] Removed commented-out code
+- [ ] Cleaned up unused imports
+- [ ] Deleted exploratory/spike code
+- [ ] Consolidated duplicate logic
+- [ ] Removed obsolete scripts/configs
+
+### Communication Style
+- Report what you've done: "Added feature X with tests in test/x_test.go"
+- Highlight what needs attention: "WARNING: Manual testing needed for edge case Y"
+- Ask questions directly: "Should we support Z? Trade-offs are..."
+- NEVER say "I'll track this in a markdown file" - use code comments or tell me directly
+
+### Script/Build System Updates
+- Update Makefile/build.zig when adding new targets or commands
+- Modify CI/CD configs (.github/workflows) if build/test process changes
+- Update package.json/Cargo.toml/go.mod when dependencies change
+- Document new scripts in README under "Development" section
+
+## Anti-Patterns to AVOID
+- Creating notes.md, todo.md, tasks.md, ideas.md files
+- Leaving commented-out code "for reference"
+- Keeping old implementation files with .old or .backup suffixes
+- Adding features without tests
+- Updating code without updating docs
+- Leaving TODO comments without context or assignee
+
+## Preferred Patterns
+- Inline TODO comments: `// TODO(user): Add caching layer for better performance`
+- Self-documenting code with clear names
+- Tests that serve as usage examples
+- Incremental, complete commits (code + tests + docs)
+- Direct communication about tasks and priorities
+
+## Definition of Done
+A task is complete ONLY when:
+1. Code is written and working
+2. Tests are written and passing
+3. Documentation is updated
+4. All temporary/dead code is removed
+5. Build scripts are updated if needed
+6. Changes are committed with clear message
 
 ### Commit Message Convention
 Use conventional commits:
@@ -235,4 +378,11 @@ make docs-build    # Build static documentation
 # Cleanup
 make clean         # Remove build artifacts
 make clean-docs    # Remove documentation artifacts
-```
+```
+
+## Additional Resources
+
+- **README.md**: Project overview and quick start guide
+- **DEVELOPMENT.md**: Detailed development setup and workflow
+- **docs/src/testing.md**: Comprehensive testing documentation
+- **cli/README.md**: Zig CLI documentation