diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 0000000..90bb66e --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,302 @@ +# Development Guide + +This guide helps developers set up their environment and contribute effectively to FetchML. + +## Quick Setup + +```bash +# Clone the repository +git clone +cd fetch_ml + +# Install dependencies +make setup-dev + +# Start development environment +make dev-start + +# Run tests +make test +``` + +## Development Environment + +### Prerequisites + +- Go 1.25+ +- Zig 0.11+ +- Python 3.11+ +- Docker & Docker Compose +- Redis +- Node.js (for some tools) + +### Local Development Setup + +1. **Install Go dependencies** + ```bash + go mod download + go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest + ``` + +2. **Install Zig tools** + ```bash + # Install Zig language server + zig build --install zls + ``` + +3. **Setup Python environment** + ```bash + python -m venv venv + source venv/bin/activate # or venv\Scripts\activate on Windows + pip install -r requirements-dev.txt + ``` + +4. **Optional: Install pre-commit hooks** + ```bash + # If you want automated code quality checks + pre-commit install + pre-commit install --hook-type commit-msg + ``` + +### Development Workflow + +#### Making Changes + +1. Create a feature branch: + ```bash + git checkout -b feature/your-feature-name + ``` + +2. Make your changes with live feedback: + ```bash + # Go development with hot reload + make dev-go + + # Zig development with build on save + make dev-zig + + # Run specific tests + make test-unit + make test-integration + ``` + +3. Run quality checks: + ```bash + # Lint and format (if you have tools configured) + make lint + make format + + # Full test suite + make test-all + + # Optional: Pre-commit checks + pre-commit run --all-files + ``` + +#### Testing Strategy + +- **Unit tests**: Fast tests for individual components +- **Integration tests**: Test component interactions +- **E2E tests**: Full workflow validation +- **Performance tests**: Load and stress testing + +```bash +# Run tests by type +make test-unit # Unit tests only +make test-integration # Integration tests only +make test-e2e # End-to-end tests only +make test-performance # Performance tests only + +# Run with coverage +make test-coverage + +# Watch mode for development +make test-watch +``` + +## Code Quality + +### Formatting + +- **Go**: `gofmt` and `goimports` +- **Zig**: Built-in formatter +- **Python**: `black` +- **YAML**: `yamllint` +- **Shell**: `shellcheck` + +### Linting + +- **Go**: `golangci-lint` +- **Python**: `pylint` +- **Docker**: `hadolint` +- **Shell**: `shellcheck` + +### Conventional Commits + +We use conventional commits for clear commit messages: + +``` +feat: add new feature +fix: bug fix +docs: documentation changes +style: formatting changes +refactor: code refactoring +test: add or update tests +chore: maintenance tasks +``` + +## Debugging + +### Go Debugging + +```bash +# Debug with delve +dlv debug cmd/api-server/main.go + +# Debug tests +dlv test ./internal/... + +# Profile with pprof +go tool pprof http://localhost:6060/debug/pprof/profile +``` + +### Zig Debugging + +```bash +# Debug build +zig build-exe -O Debug -fstrip=false your_file.zig + +# Test with debugging +zig test --gdb your_file.zig +``` + +### Container Debugging + +```bash +# Debug containers +docker-compose exec api-server bash +docker-compose logs -f api-server + +# Inspect running processes +docker-compose exec api-server ps aux +``` + +## Performance Monitoring + +### Local Monitoring + +```bash +# Start monitoring stack +make monitoring-start + +# View metrics +open http://localhost:3000 # Grafana +open http://localhost:9090 # Prometheus +``` + +### Performance Testing + +```bash +# Load test API +make load-test-api + +# Performance benchmarks +make benchmark + +# Memory profiling +make profile-memory +``` + +## Database Management + +### Development Database + +```bash +# Reset database +make db-reset + +# Run migrations +make db-migrate + +# Access database +docker-compose exec redis redis-cli +``` + +### Schema Changes + +1. Update migration files in `internal/storage/` +2. Test with `make db-test-migration` +3. Update documentation + +## Configuration + +### Environment Variables + +Copy `.env.example` to `.env.local` and adjust: + +```bash +cp .env.example .env.local +``` + +Key variables for development: +- `REDIS_URL`: Redis connection string +- `LOG_LEVEL`: Set to `debug` for verbose logging +- `API_PORT`: API server port (default: 9101) + +### Configuration Files + +- `configs/config-dev.yaml`: Development configuration +- `configs/config-local.yaml`: Local overrides +- `configs/config-prod.yaml`: Production settings + +## IDE Setup + +### Recommended Extensions + +**VS Code:** +- Go (golang.go) +- Zig (ziglang.vscode-zig) +- Python (ms-python.python) +- Docker (ms-azuretools.vscode-docker) +- YAML (redhat.vscode-yaml) + +**Neovim:** +- Go language server (gopls) +- Zig language server (zls) +- Python LSP (pylsp) +- Docker integration + +### Configuration + +Set up your preferred editor with: +- Go formatting (gofmt, goimports) +- Zig formatting (built-in) +- Python formatting (black, autopep8) +- YAML linting (yamllint) + +Configure these tools according to your workflow preferences. + +## Troubleshooting + +### Common Issues + +1. **Go modules not found**: Run `go mod download` +2. **Zig build fails**: Check Zig version with `zig version` +3. **Docker containers not starting**: Check Docker daemon +4. **Redis connection failed**: Start Redis with `docker-compose up redis` + +### Getting Help + +- Check logs: `make logs` +- Run diagnostics: `make doctor` +- Review troubleshooting guide: [docs/src/troubleshooting.md](docs/src/troubleshooting.md) + +## Contributing + +1. Follow the code of conduct +2. Use conventional commits +3. Add tests for new features +4. Update documentation +5. Ensure all checks pass + +See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.