fetch_ml/cmd/tui

FetchML TUI - Terminal User Interface

An interactive terminal dashboard for managing ML experiments, monitoring system resources, and controlling job execution.

Features

📊 Real-time Monitoring

• Job Status - Track pending, running, finished, and failed jobs
• GPU Metrics - Monitor GPU utilization, memory, and temperature
• Container Status - View running Podman/Docker containers
• Task Queue - See queued tasks with priorities and status

🎮 Interactive Controls

• Queue Jobs - Submit jobs with custom arguments and priorities
• View Logs - Real-time log viewing for running jobs
• Cancel Tasks - Stop running tasks
• Delete Jobs - Remove pending jobs
• Mark Failed - Manually mark stuck jobs as failed

⚙️ Settings Management

• API Key Configuration - Set and update API keys on the fly
• In-memory Storage - Settings persist for the session

🎨 Modern UI

• Clean Design - Dark-mode friendly with adaptive colors
• Responsive Layout - Adjusts to terminal size
• Context-aware Help - Shows relevant shortcuts for each view
• Mouse Support - Optional mouse navigation

Quick Start

Running the TUI

# Using make (recommended)
make tui-dev          # Dev mode (remote server)
make tui              # Local mode

# Direct execution with CLI config (TOML)
./bin/tui --config ~/.ml/config.toml

# With custom TOML config
./bin/tui --config path/to/config.toml

First Time Setup

1. Build the binary

   make build
   

2. Get your API key

   ./bin/user_manager --config configs/config_dev.yaml --cmd generate-key --username your_name
   

3. Launch the TUI

   make tui-dev
   

Keyboard Shortcuts

Navigation

Key	Action
1	Switch to Job List view
g	Switch to GPU Status view
l	View logs for selected job
v	Switch to Task Queue view
o	Switch to Container Status view
s	Open Settings
h or ?	Toggle help screen

Job Management

Key	Action
t	Queue selected job (default args)
a	Queue job with custom arguments
c	Cancel running task
d	Delete pending job
f	Mark running job as failed

System

Key	Action
r	Refresh all data
G	Refresh GPU status only
q or Ctrl+C	Quit

Settings View

Key	Action
↑/↓ or j/k	Navigate options
Enter	Select/Save
Esc	Exit settings

Views

Job List (Default)

• Shows all jobs across all statuses
• Filter with / key
• Navigate with arrow keys or j/k
• Select and press l to view logs

GPU Status

• Real-time GPU metrics (nvidia-smi)
• macOS GPU info (system_profiler)
• Utilization, memory, temperature

Container Status

• Running Podman/Docker containers
• Container health and status
• System info (Podman/Docker version)

Task Queue

• All queued tasks with priorities
• Task status and creation time
• Running duration for active tasks

Logs

• Last 200 lines of job output
• Auto-scroll to bottom
• Refreshes with job status

Settings

• View current API key status
• Update API key
• Save configuration (in-memory)

Terminal Compatibility

The TUI is built with Bubble Tea and works on all modern terminals:

✅ Fully Supported

• WezTerm (recommended)
• Alacritty
• Kitty
• iTerm2 (macOS)
• Terminal.app (macOS)
• Windows Terminal
• GNOME Terminal
• Konsole

✅ Multiplexers

• tmux
• screen

Features

• ✅ 256 colors
• ✅ True color (24-bit)
• ✅ Mouse support
• ✅ Alt screen buffer
• ✅ Adaptive colors (light/dark themes)

Key Components

• Model - Pure data structures (State, Job, Task)
• View - Rendering functions (no business logic)
• Controller - Message handling and state updates
• Services - SSH/Redis communication

Configuration

The TUI uses TOML configuration format for CLI settings:

# ~/.ml/config.toml
worker_host = "localhost"
worker_user = "your_user"
worker_base = "~/ml_jobs"
worker_port = 22
api_key = "your_api_key_here"

For CLI usage, run ml init to create a default configuration file.

See Configuration Documentation for details.

Troubleshooting

TUI doesn't start

# Check if binary exists
ls -la bin/tui

# Rebuild if needed
make build

# Check CLI config
cat ~/.ml/config.toml

Authentication errors

# Verify CLI config exists
ls -la ~/.ml/config.toml

# Initialize CLI config if needed
ml init

# Test connection
./bin/tui --config ~/.ml/config.toml

Display issues

# Check terminal type
echo $TERM

# Should be xterm-256color or similar
# If not, set it:
export TERM=xterm-256color

Connection issues

# Test SSH connection
ssh your_user@your_server

# Test Redis connection
redis-cli ping

Development

Building

# Build TUI only
go build -o bin/tui ./cmd/tui

# Build all binaries
make build

Testing

# Run with verbose logging
./bin/tui --config ~/.ml/config.toml 2>tui.log

# Check logs
tail -f tui.log

Code Organization

• Keep files under 300 lines
• Separate concerns (MVC pattern)
• Use descriptive function names
• Add comments for complex logic

Tips & Tricks

Efficient Workflow

1. Keep TUI open in one terminal
2. Edit code in another terminal
3. Use r to refresh after changes
4. Use h to quickly reference shortcuts

Custom Arguments

When queuing jobs with a:

--epochs 100 --lr 0.001 --priority 5

Monitoring

• Use G for quick GPU refresh (faster than r)
• Check queue with v before queuing new jobs
• Use l to debug failed jobs

Settings

• Update API key without restarting
• Changes are in-memory only
• Restart TUI to reset

See Also

• Main Documentation
• Worker Documentation
• Configuration Guide
• Bubble Tea Documentation