7880ea8d79
Organize podman/ directory into logical subdirectories: New structure: - docs/ - ML_TOOLS_GUIDE.md, jupyter_workflow.md - configs/ - environment*.yml, security_policy.json - containers/ - *.dockerfile, *.podfile - scripts/ - *.sh, *.py (secure_runner, cli_integration, etc.) - jupyter/ - jupyter_cookie_secret (flattened from jupyter_runtime/runtime/) - workspace/ - Example projects (cleaned of temp files) Cleaned workspace: - Removed .DS_Store, mlflow.db, cache/ - Removed duplicate cli_integration.py Removed unnecessary nesting: - Flattened jupyter_runtime/runtime/ to just jupyter/ Improves maintainability by grouping files by purpose and eliminating root directory clutter.
50 lines
1.2 KiB
Markdown
50 lines
1.2 KiB
Markdown
# CLI-Jupyter Integration Workflow
|
|
## Workflow Overview
|
|
|
|
This workflow integrates the FetchML CLI with Jupyter notebooks for seamless data science experiments.
|
|
|
|
### Step 1: Start Jupyter Server
|
|
```bash
|
|
# Build the container with ML tools
|
|
podman build -f ml-tools-runner.podfile -t ml-tools-runner .
|
|
|
|
# Start Jupyter server
|
|
podman run -d -p 8888:8888 --name ml-jupyter \
|
|
-v "$(pwd)/workspace:/workspace:Z" \
|
|
--entrypoint conda localhost/ml-tools-runner \
|
|
run -n ml_env jupyter notebook --no-browser --ip=0.0.0.0 --port=8888 \
|
|
--NotebookApp.token='' --NotebookApp.password='' --allow-root
|
|
|
|
# Access at http://localhost:8888
|
|
```
|
|
|
|
### Step 2: Use CLI to Sync Projects
|
|
```bash
|
|
# From another terminal, sync your project
|
|
cd cli && ./zig-out/bin/ml sync ./my_project --queue
|
|
|
|
# Check status
|
|
./zig-out/bin/ml status
|
|
```
|
|
|
|
### Step 3: Run Experiments in Jupyter
|
|
```python
|
|
# In your Jupyter notebook
|
|
import mlflow
|
|
import wandb
|
|
import pandas as pd
|
|
|
|
# Start MLflow tracking
|
|
mlflow.start_run()
|
|
mlflow.log_param("model", "random_forest")
|
|
mlflow.log_metric("accuracy", 0.95)
|
|
```
|
|
|
|
### Step 4: Monitor with CLI
|
|
```bash
|
|
# Monitor jobs from CLI
|
|
./zig-out/bin/ml monitor
|
|
|
|
# View logs
|
|
./zig-out/bin/ml experiment log my_experiment
|
|
```
|