# ADR-007: Simplified Streaming I/O Without io_uring

## Status
Proposed

## Context
The streaming_io library handles large artifact uploads/downloads with gzip/tar decompression. Initial plans included io_uring for asynchronous I/O, which promised maximum performance. However, io_uring adds significant operational complexity:

- Kernel version requirements (Linux 5.10+)
- Container compatibility issues (Docker/Podman restrictions)
- Security surface area (ring buffer attacks)
- Fallback complexity (must handle unsupported kernels)
- macOS incompatibility (io_uring is Linux-only)

The 95% syscall reduction from streaming_io primarily comes from:
1. Memory-mapped I/O (mmap) for reading archive headers
2. Thread pool for parallel gzip decompression
3. Direct I/O (O_DIRECT) for large writes

io_uring adds only marginal benefit beyond these techniques.

## Decision
Use **simplified streaming I/O** without io_uring. The implementation uses:

1. **mmap** for archive header inspection (gzip/tar structure)
2. **Thread pool** for parallel decompression of independent gzip blocks
3. **Direct I/O (O_DIRECT)** for large file writes (portable, simple fallback to buffered)
4. **Standard pread/pwrite** for random access (no io_uring)

**Revised Implementation:**
```cpp
// streaming_io.h - REVISED (no io_uring)

struct streaming_ctx {
    // Memory mapping
    void* mmap_base;
    size_t mmap_size;
    
    // Thread pool
    ThreadPool thread_pool{4};  // Configurable
    
    // Direct I/O for large writes
    bool use_direct_io;
    int output_fd;
};

// Main entry point - single CGo call
extern "C" int extract_tar_gz(
    const char* archive_path,
    const char* output_dir,
    uint32_t num_threads
);
```

**Thread Pool Decompression:**
```cpp
void decompress_blocks_parallel(
    const std::vector<CompressedBlock>& blocks,
    const std::string& output_dir
) {
    // Each block independent - perfect for thread pool
    thread_pool.parallel_for(blocks.size(), [&](size_t i) {
        auto decompressed = gzip_decompress(blocks[i].data);
        direct_write(output_dir + "/" + blocks[i].filename, decompressed);
    });
}
```

## Consequences

### Positive
- Works on all platforms (Linux, macOS, containers)
- No kernel version dependencies
- Simpler code (no ring buffer management)
- Easier debugging (standard syscalls)
- 95% of syscall win retained

### Negative
- ~5% performance loss vs io_uring for bulk I/O
- Thread pool overhead for small archives
- O_DIRECT alignment requirements (platform-specific)

## Options Considered

### Option 1: Full io_uring Implementation
**Pros:** Maximum performance on modern Linux  
**Cons:** Container issues, kernel requirements, macOS incompatible, security surface  
**Verdict:** Rejected - complexity exceeds benefit

### Option 2: Hybrid (io_uring when available, fallback otherwise)
**Pros:** Best of both worlds  
**Cons:** Double the code, double the testing, runtime detection complexity  
**Verdict:** Rejected - fallback paths often have bugs

### Option 3: Simplified (mmap + thread pool) (Chosen)
**Pros:** 95% of benefit, works everywhere  
**Cons:** Slightly lower peak performance  
**Verdict:** Accepted - pragmatic performance

## Performance Comparison

| Approach | Syscall Reduction | Complexity | Portability |
|----------|-------------------|------------|-------------|
| Go stdlib | Baseline | Low | Universal |
| Simplified C++ | 95% | Medium | Universal |
| io_uring C++ | 98% | High | Linux 5.10+ only |

**3% difference not worth the complexity cost.**

## Enforcement

### Code Review Checklist
- [ ] No io_uring headers included
- [ ] Standard POSIX I/O only (pread/pwrite, mmap)
- [ ] Thread pool size configurable
- [ ] O_DIRECT with automatic fallback

### Testing Requirements
- CI on Linux (containerized and native)
- CI on macOS (Intel and ARM)
- Large file tests (>1GB archives)
- Concurrent extraction tests

### Benchmarks
```bash
# Verify 95% syscall reduction
make benchmark-streaming-io

# Compare with Go implementation
go test -bench=BenchmarkStreaming -benchmem ./tests/benchmarks/
```

## Rationale

The 95/5 rule applies: 95% of the syscall win comes from parallel decompression and mmap, only 5% from io_uring. The operational cost of io_uring (kernel requirements, container restrictions, macOS incompatibility) exceeds the marginal benefit.

The simplified approach is **correct first, fast second**. A universal 95% win beats a fragmented 98% win.

## Future Considerations

If kernel requirements change (e.g., minimum becomes Linux 6.0+), io_uring can be reconsidered. The simplified architecture is forward-compatible: io_uring can be added as an optional backend without changing the Go API.

```cpp
// Future: pluggable backend
class IOBackend {
    virtual ssize_t read(int fd, void* buf, size_t count, off_t offset) = 0;
    virtual ssize_t write(int fd, const void* buf, size_t count, off_t offset) = 0;
};

class PosixBackend : public IOBackend { /* current impl */ };
// class UringBackend : public IOBackend { /* future */ };
```

Until then, the simplified implementation is the correct choice.
