This commit is contained in:
commit
4cd7e72e2b
23 changed files with 1010 additions and 0 deletions
3
.env.example
Normal file
3
.env.example
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
CONFIG_PATH=/etc/infra-controller/config.yml
|
||||
ACTIVE_APPS_DIR=/var/run/active-apps
|
||||
LOG_LEVEL=INFO
|
||||
55
.forgejo/workflows/deploy.yml
Normal file
55
.forgejo/workflows/deploy.yml
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
name: Deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: docker
|
||||
steps:
|
||||
# Checkout code
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
# Setup SSH for services server
|
||||
- name: Setup SSH
|
||||
shell: bash
|
||||
env:
|
||||
SERVICE_SSH_KEY: ${{ secrets.SERVICE_SSH_KEY }}
|
||||
SERVICE_HOST: ${{ secrets.SERVICE_HOST }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -p ~/.ssh
|
||||
printf '%s\n' "$SERVICE_SSH_KEY" > ~/.ssh/id_ed25519
|
||||
chmod 600 ~/.ssh/id_ed25519
|
||||
ssh-keyscan -H "$SERVICE_HOST" >> ~/.ssh/known_hosts
|
||||
|
||||
# Deploy app locally on the runner host
|
||||
- name: Deploy App (Docker Compose)
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
APP_NAME="${{ github.event.repository.name }}"
|
||||
APP_PATH="/srv/apps/$APP_NAME"
|
||||
echo "Deploying $APP_NAME from $APP_PATH..."
|
||||
cd "$APP_PATH"
|
||||
docker compose pull
|
||||
docker compose up -d
|
||||
|
||||
# Register app on the services server (triggers infra-controller.path)
|
||||
- name: Register App Requirements
|
||||
shell: bash
|
||||
env:
|
||||
SERVICE_HOST: ${{ secrets.SERVICE_HOST }}
|
||||
SERVICE_USER: ${{ secrets.SERVICE_USER }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
APP_NAME="${{ github.event.repository.name }}"
|
||||
echo "Registering app $APP_NAME with infra-controller..."
|
||||
test -f .infra.toml
|
||||
ssh -i ~/.ssh/id_ed25519 "$SERVICE_USER@$SERVICE_HOST" \
|
||||
"cat > /var/run/active-apps/$APP_NAME.toml.tmp && mv /var/run/active-apps/$APP_NAME.toml.tmp /var/run/active-apps/$APP_NAME.toml" \
|
||||
< .infra.toml
|
||||
21
LICENSE
Normal file
21
LICENSE
Normal file
|
|
@ -0,0 +1,21 @@
|
|||
MIT License
|
||||
|
||||
Copyright (c) 2026
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
18
Makefile
Normal file
18
Makefile
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
.PHONY: install dev test clean build
|
||||
|
||||
install:
|
||||
pip install -e .
|
||||
|
||||
dev:
|
||||
pip install -e ".[dev]"
|
||||
|
||||
test:
|
||||
pytest tests/
|
||||
|
||||
clean:
|
||||
find . -type d -name __pycache__ -exec rm -rf {} +
|
||||
find . -type f -name "*.pyc" -delete
|
||||
rm -rf build/ dist/ *.egg-info
|
||||
|
||||
build:
|
||||
python -m build
|
||||
53
README.md
Normal file
53
README.md
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
# infra-controller
|
||||
|
||||
Python-based controller that discovers active apps and ensures required infrastructure services are present.
|
||||
|
||||
Services are expected to be managed as Docker Compose projects on the services server (e.g. `/opt/grafana`, `/opt/prometheus`).
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.11+
|
||||
- Docker and Docker Compose installed on the services server
|
||||
|
||||
## Config
|
||||
|
||||
Preferred config file:
|
||||
|
||||
- `/etc/infra-controller/config.toml`
|
||||
|
||||
Copy the example config:
|
||||
|
||||
- `config/controller.toml.example` -> `/etc/infra-controller/config.toml`
|
||||
|
||||
Optional YAML config:
|
||||
|
||||
- `config/controller.yml.example` -> `/etc/infra-controller/config.yml`
|
||||
|
||||
## Run
|
||||
|
||||
- `infra-controller --once`
|
||||
|
||||
## systemd (event-driven)
|
||||
|
||||
To avoid running a daemon or polling timer, you can trigger a one-shot run whenever deployments update the active apps directory:
|
||||
|
||||
- enable path trigger: `sudo systemctl enable --now infra-controller.path`
|
||||
- view logs: `journalctl -u infra-controller-once.service -f`
|
||||
|
||||
## Remote app registration
|
||||
|
||||
Run `infra-controller` on the service server. When you deploy, create/update a registration file in `/var/run/active-apps/` (this triggers the path unit).
|
||||
|
||||
Recommended (Forgejo runner on the web/app server):
|
||||
|
||||
- deploy app locally on the web/app server (docker compose or bare-metal)
|
||||
- register app on the service server by streaming `.infra.toml` over SSH (no scp)
|
||||
|
||||
Example (from web/app server runner):
|
||||
|
||||
```bash
|
||||
APP_NAME=my-app
|
||||
ssh infractl@service-host \
|
||||
"cat > /var/run/active-apps/$APP_NAME.toml.tmp && mv /var/run/active-apps/$APP_NAME.toml.tmp /var/run/active-apps/$APP_NAME.toml" \
|
||||
< .infra.toml
|
||||
```
|
||||
23
config/controller.toml.example
Normal file
23
config/controller.toml.example
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
[discovery.file_based]
|
||||
enabled = true
|
||||
path = "/var/run/active-apps"
|
||||
|
||||
[discovery.scan]
|
||||
enabled = true
|
||||
paths = ["/home", "/opt/apps"]
|
||||
exclude_patterns = ["**/node_modules/**", "**/.venv/**", "**/venv/**"]
|
||||
|
||||
[docker]
|
||||
base_dir = "/opt"
|
||||
compose_file = "docker-compose.yml"
|
||||
|
||||
[services]
|
||||
grace_period_minutes = 15
|
||||
check_interval_seconds = 60
|
||||
state_file = "/var/lib/infra-controller/state.json"
|
||||
|
||||
[logging]
|
||||
level = "INFO"
|
||||
file = "/var/log/infra-controller/controller.log"
|
||||
max_bytes = 10485760
|
||||
backup_count = 5
|
||||
29
config/controller.yml.example
Normal file
29
config/controller.yml.example
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
discovery:
|
||||
file_based:
|
||||
enabled: true
|
||||
path: /var/run/active-apps
|
||||
|
||||
scan:
|
||||
enabled: true
|
||||
paths:
|
||||
- /home
|
||||
- /opt/apps
|
||||
exclude_patterns:
|
||||
- "**/node_modules/**"
|
||||
- "**/.venv/**"
|
||||
- "**/venv/**"
|
||||
|
||||
docker:
|
||||
base_dir: /opt
|
||||
compose_file: docker-compose.yml
|
||||
|
||||
services:
|
||||
grace_period_minutes: 15
|
||||
check_interval_seconds: 60
|
||||
state_file: /var/lib/infra-controller/state.json
|
||||
|
||||
logging:
|
||||
level: INFO
|
||||
file: /var/log/infra-controller/controller.log
|
||||
max_bytes: 10485760
|
||||
backup_count: 5
|
||||
54
install.sh
Normal file
54
install.sh
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
echo "Installing Infrastructure Controller..."
|
||||
|
||||
python_version=$(python3 --version | awk '{print $2}')
|
||||
required_version="3.11"
|
||||
|
||||
if [ "$(printf '%s\n' "$required_version" "$python_version" | sort -V | head -n1)" != "$required_version" ]; then
|
||||
echo "Error: Python 3.11+ required, found $python_version"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! id infractl >/dev/null 2>&1; then
|
||||
echo "Creating infractl user..."
|
||||
sudo useradd -r -s /bin/false -d /opt/infra-controller infractl
|
||||
fi
|
||||
|
||||
sudo mkdir -p /opt/infra-controller
|
||||
sudo mkdir -p /etc/infra-controller
|
||||
sudo mkdir -p /var/run/active-apps
|
||||
sudo mkdir -p /var/lib/infra-controller
|
||||
sudo mkdir -p /var/log/infra-controller
|
||||
|
||||
echo "Installing Python package..."
|
||||
sudo python3 -m venv /opt/infra-controller/venv
|
||||
sudo /opt/infra-controller/venv/bin/pip install --upgrade pip
|
||||
sudo /opt/infra-controller/venv/bin/pip install -e .
|
||||
|
||||
if [ ! -f /etc/infra-controller/config.toml ]; then
|
||||
echo "Installing default configuration..."
|
||||
sudo cp config/controller.toml.example /etc/infra-controller/config.toml
|
||||
fi
|
||||
|
||||
if [ ! -f /etc/infra-controller/controller.env ]; then
|
||||
sudo cp systemd/infra-controller.env /etc/infra-controller/controller.env
|
||||
fi
|
||||
|
||||
echo "Installing systemd service..."
|
||||
sudo cp systemd/infra-controller.service /etc/systemd/system/
|
||||
sudo cp systemd/infra-controller-once.service /etc/systemd/system/
|
||||
sudo cp systemd/infra-controller.path /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
echo "Enabling infra-controller path trigger..."
|
||||
sudo systemctl enable infra-controller.path
|
||||
|
||||
sudo chown -R infractl:infractl /opt/infra-controller
|
||||
sudo chown -R infractl:infractl /var/lib/infra-controller
|
||||
sudo chown -R infractl:infractl /var/log/infra-controller
|
||||
sudo chown -R infractl:infractl /var/run/active-apps
|
||||
sudo chmod 755 /var/run/active-apps
|
||||
|
||||
echo "Installation complete!"
|
||||
48
pyproject.toml
Normal file
48
pyproject.toml
Normal file
|
|
@ -0,0 +1,48 @@
|
|||
[build-system]
|
||||
requires = ["setuptools>=65.0", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "infra-controller"
|
||||
version = "0.1.0"
|
||||
description = "Adaptive infrastructure controller for dynamic service management"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
license = {text = "MIT"}
|
||||
authors = [
|
||||
{name = "Your Name", email = "you@example.com"},
|
||||
]
|
||||
keywords = ["infrastructure", "automation", "docker", "docker-compose"]
|
||||
classifiers = [
|
||||
"Development Status :: 4 - Beta",
|
||||
"Intended Audience :: System Administrators",
|
||||
"License :: OSI Approved :: MIT License",
|
||||
"Programming Language :: Python :: 3",
|
||||
"Programming Language :: Python :: 3.11",
|
||||
"Topic :: System :: Systems Administration",
|
||||
]
|
||||
|
||||
dependencies = [
|
||||
"pyyaml>=6.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=7.0",
|
||||
"ruff>=0.1.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
infra-controller = "infra_controller.__main__:main"
|
||||
infra-register = "infra_controller.cli:register"
|
||||
infra-deregister = "infra_controller.cli:deregister"
|
||||
infra-status = "infra_controller.cli:status"
|
||||
|
||||
[tool.setuptools]
|
||||
package-dir = {"" = "src"}
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
1
requirements.txt
Normal file
1
requirements.txt
Normal file
|
|
@ -0,0 +1 @@
|
|||
pyyaml>=6.0
|
||||
3
setup.py
Normal file
3
setup.py
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
from setuptools import setup
|
||||
|
||||
setup()
|
||||
3
src/infra_controller/__init__.py
Normal file
3
src/infra_controller/__init__.py
Normal file
|
|
@ -0,0 +1,3 @@
|
|||
__all__ = ["__version__"]
|
||||
|
||||
__version__ = "0.1.0"
|
||||
45
src/infra_controller/__main__.py
Normal file
45
src/infra_controller/__main__.py
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from infra_controller.config import load_config
|
||||
from infra_controller.controller import InfraController
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(prog="infra-controller")
|
||||
parser.add_argument("config", nargs="?", help="Path to config (.toml/.yml)")
|
||||
parser.add_argument("--config", dest="config_flag", help="Path to config (.toml/.yml)")
|
||||
parser.add_argument(
|
||||
"--once",
|
||||
action="store_true",
|
||||
help="Run one discovery/apply cycle and exit (for systemd timers)",
|
||||
)
|
||||
|
||||
args = parser.parse_args(sys.argv[1:])
|
||||
|
||||
config_path: Path | None = None
|
||||
if args.config_flag:
|
||||
config_path = Path(args.config_flag)
|
||||
elif args.config:
|
||||
config_path = Path(args.config)
|
||||
|
||||
cfg = load_config(config_path)
|
||||
|
||||
logging.basicConfig(
|
||||
level=cfg.logging.level,
|
||||
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
|
||||
)
|
||||
|
||||
controller = InfraController(cfg)
|
||||
try:
|
||||
if args.once:
|
||||
controller.run_once()
|
||||
return
|
||||
controller.run()
|
||||
except KeyboardInterrupt:
|
||||
logging.info("Shutting down...")
|
||||
raise SystemExit(0)
|
||||
72
src/infra_controller/cli.py
Normal file
72
src/infra_controller/cli.py
Normal file
|
|
@ -0,0 +1,72 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
from infra_controller.config import load_config
|
||||
from infra_controller.controller import InfraController
|
||||
from infra_controller.discovery import InfraMetadata
|
||||
|
||||
|
||||
def register() -> None:
|
||||
parser = argparse.ArgumentParser(prog="infra-register")
|
||||
parser.add_argument("metadata_file", help="Path to .infra.toml/.infra.yml (or any metadata file)")
|
||||
parser.add_argument(
|
||||
"--name",
|
||||
help="Override project name (otherwise read from metadata_file)",
|
||||
)
|
||||
args = parser.parse_args(sys.argv[1:])
|
||||
|
||||
cfg = load_config()
|
||||
src = Path(args.metadata_file)
|
||||
if not src.exists():
|
||||
print(f"metadata file not found: {src}", file=sys.stderr)
|
||||
raise SystemExit(2)
|
||||
|
||||
md = InfraMetadata.from_file(src)
|
||||
name = args.name or md.project
|
||||
|
||||
dst_dir = cfg.discovery.file_based_path
|
||||
dst_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
ext = src.suffix if src.suffix in {".toml", ".yml", ".yaml"} else ".toml"
|
||||
dst = dst_dir / f"{name}{ext}"
|
||||
tmp = dst.with_suffix(dst.suffix + ".tmp")
|
||||
shutil.copyfile(src, tmp)
|
||||
tmp.replace(dst)
|
||||
|
||||
|
||||
def deregister() -> None:
|
||||
parser = argparse.ArgumentParser(prog="infra-deregister")
|
||||
parser.add_argument("name", help="Project name to deregister")
|
||||
args = parser.parse_args(sys.argv[1:])
|
||||
|
||||
cfg = load_config()
|
||||
dst_dir = cfg.discovery.file_based_path
|
||||
|
||||
removed = False
|
||||
for ext in (".toml", ".yml", ".yaml"):
|
||||
p = dst_dir / f"{args.name}{ext}"
|
||||
if p.exists():
|
||||
p.unlink()
|
||||
removed = True
|
||||
|
||||
if not removed:
|
||||
print(f"no registration found for: {args.name}", file=sys.stderr)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
def status() -> None:
|
||||
cfg = load_config()
|
||||
print(f"config={cfg}")
|
||||
|
||||
|
||||
def ensure_service_cli(argv: list[str] | None = None) -> None:
|
||||
parser = argparse.ArgumentParser(prog="infra-ensure")
|
||||
parser.add_argument("service")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
cfg = load_config()
|
||||
InfraController(cfg).ensure_service(args.service)
|
||||
189
src/infra_controller/config.py
Normal file
189
src/infra_controller/config.py
Normal file
|
|
@ -0,0 +1,189 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import tomllib
|
||||
|
||||
import yaml
|
||||
|
||||
|
||||
@dataclass
|
||||
class DiscoveryConfig:
|
||||
file_based_enabled: bool = True
|
||||
file_based_path: Path = Path("/var/run/active-apps")
|
||||
http_enabled: bool = True
|
||||
http_host: str = "127.0.0.1"
|
||||
http_port: int = 8080
|
||||
scan_enabled: bool = True
|
||||
scan_paths: list[Path] = field(default_factory=lambda: [Path("/home"), Path("/opt/apps")])
|
||||
exclude_patterns: list[str] = field(
|
||||
default_factory=lambda: ["**/node_modules/**", "**/.venv/**", "**/venv/**"]
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class DockerComposeConfig:
|
||||
base_dir: Path = Path("/opt")
|
||||
compose_file: str = "docker-compose.yml"
|
||||
|
||||
|
||||
@dataclass
|
||||
class ServicesConfig:
|
||||
grace_period_minutes: int
|
||||
check_interval_seconds: int
|
||||
state_file: Path
|
||||
|
||||
|
||||
@dataclass
|
||||
class LoggingConfig:
|
||||
level: str = "INFO"
|
||||
file: Path | None = None
|
||||
max_bytes: int = 10 * 1024 * 1024
|
||||
backup_count: int = 5
|
||||
|
||||
|
||||
@dataclass
|
||||
class ControllerConfig:
|
||||
discovery: DiscoveryConfig = field(default_factory=DiscoveryConfig)
|
||||
docker: DockerComposeConfig = field(default_factory=DockerComposeConfig)
|
||||
services: ServicesConfig = field(
|
||||
default_factory=lambda: ServicesConfig(
|
||||
grace_period_minutes=15,
|
||||
check_interval_seconds=60,
|
||||
state_file=Path("/var/lib/infra-controller/state.json"),
|
||||
)
|
||||
)
|
||||
logging: LoggingConfig = field(default_factory=LoggingConfig)
|
||||
|
||||
@classmethod
|
||||
def from_file(cls, path: Path | str) -> "ControllerConfig":
|
||||
path = Path(path)
|
||||
if not path.exists():
|
||||
return cls()
|
||||
|
||||
if path.suffix in {".toml", ".tml"}:
|
||||
with open(path, "rb") as f:
|
||||
data = tomllib.load(f)
|
||||
elif path.suffix in {".yml", ".yaml"}:
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
else:
|
||||
raise ValueError(f"Unsupported config format: {path.suffix}")
|
||||
|
||||
return cls._from_dict(data)
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> "ControllerConfig":
|
||||
config = cls()
|
||||
|
||||
if val := os.getenv("ACTIVE_APPS_DIR"):
|
||||
config.discovery.file_based_path = Path(val)
|
||||
if val := os.getenv("SCAN_PATHS"):
|
||||
config.discovery.scan_paths = [Path(p.strip()) for p in val.split(",") if p.strip()]
|
||||
if val := os.getenv("DOCKER_BASE_DIR"):
|
||||
config.docker.base_dir = Path(val)
|
||||
if val := os.getenv("DOCKER_COMPOSE_FILE"):
|
||||
config.docker.compose_file = val
|
||||
if val := os.getenv("CHECK_INTERVAL"):
|
||||
config.services.check_interval_seconds = int(val)
|
||||
if val := os.getenv("GRACE_PERIOD_MINUTES"):
|
||||
config.services.grace_period_minutes = int(val)
|
||||
if val := os.getenv("LOG_LEVEL"):
|
||||
config.logging.level = val
|
||||
|
||||
return config
|
||||
|
||||
@classmethod
|
||||
def _from_dict(cls, data: dict[str, Any]) -> "ControllerConfig":
|
||||
config = cls()
|
||||
|
||||
if isinstance(data.get("discovery"), dict):
|
||||
disc = data["discovery"]
|
||||
if isinstance(disc.get("file_based"), dict):
|
||||
fb = disc["file_based"]
|
||||
config.discovery.file_based_enabled = bool(fb.get("enabled", True))
|
||||
if fb.get("path"):
|
||||
config.discovery.file_based_path = Path(fb["path"])
|
||||
|
||||
if isinstance(disc.get("http"), dict):
|
||||
http = disc["http"]
|
||||
config.discovery.http_enabled = bool(http.get("enabled", True))
|
||||
config.discovery.http_host = str(http.get("host", config.discovery.http_host))
|
||||
if http.get("port") is not None:
|
||||
config.discovery.http_port = int(http["port"])
|
||||
|
||||
if isinstance(disc.get("scan"), dict):
|
||||
scan = disc["scan"]
|
||||
config.discovery.scan_enabled = bool(scan.get("enabled", True))
|
||||
if isinstance(scan.get("paths"), list):
|
||||
config.discovery.scan_paths = [Path(p) for p in scan.get("paths") or []]
|
||||
if isinstance(scan.get("exclude_patterns"), list):
|
||||
config.discovery.exclude_patterns = [str(p) for p in scan.get("exclude_patterns") or []]
|
||||
|
||||
if isinstance(data.get("docker"), dict):
|
||||
dk = data["docker"]
|
||||
if dk.get("base_dir"):
|
||||
config.docker.base_dir = Path(dk["base_dir"])
|
||||
if dk.get("compose_file"):
|
||||
config.docker.compose_file = str(dk["compose_file"])
|
||||
|
||||
if isinstance(data.get("services"), dict):
|
||||
svc = data["services"]
|
||||
if svc.get("grace_period_minutes") is not None:
|
||||
config.services.grace_period_minutes = int(svc["grace_period_minutes"])
|
||||
if svc.get("check_interval_seconds") is not None:
|
||||
config.services.check_interval_seconds = int(svc["check_interval_seconds"])
|
||||
if svc.get("state_file"):
|
||||
config.services.state_file = Path(svc["state_file"])
|
||||
|
||||
if isinstance(data.get("logging"), dict):
|
||||
lg = data["logging"]
|
||||
if lg.get("level"):
|
||||
config.logging.level = str(lg["level"])
|
||||
if lg.get("file"):
|
||||
config.logging.file = Path(lg["file"])
|
||||
if lg.get("max_bytes") is not None:
|
||||
config.logging.max_bytes = int(lg["max_bytes"])
|
||||
if lg.get("backup_count") is not None:
|
||||
config.logging.backup_count = int(lg["backup_count"])
|
||||
|
||||
return config
|
||||
|
||||
|
||||
def load_config(config_path: str | os.PathLike[str] | None = None) -> ControllerConfig:
|
||||
if config_path is None:
|
||||
config_path = os.getenv("CONFIG_PATH")
|
||||
if config_path:
|
||||
p = Path(str(config_path))
|
||||
if p.exists():
|
||||
cfg = ControllerConfig.from_file(p)
|
||||
else:
|
||||
cfg = ControllerConfig.from_env()
|
||||
else:
|
||||
default_toml = Path("/etc/infra-controller/config.toml")
|
||||
default_yaml = Path("/etc/infra-controller/config.yml")
|
||||
if default_toml.exists():
|
||||
cfg = ControllerConfig.from_file(default_toml)
|
||||
elif default_yaml.exists():
|
||||
cfg = ControllerConfig.from_file(default_yaml)
|
||||
else:
|
||||
cfg = ControllerConfig.from_env()
|
||||
|
||||
if val := os.getenv("ACTIVE_APPS_DIR"):
|
||||
cfg.discovery.file_based_path = Path(val)
|
||||
if val := os.getenv("SCAN_PATHS"):
|
||||
cfg.discovery.scan_paths = [Path(p.strip()) for p in val.split(",") if p.strip()]
|
||||
if val := os.getenv("DOCKER_BASE_DIR"):
|
||||
cfg.docker.base_dir = Path(val)
|
||||
if val := os.getenv("DOCKER_COMPOSE_FILE"):
|
||||
cfg.docker.compose_file = val
|
||||
if val := os.getenv("CHECK_INTERVAL"):
|
||||
cfg.services.check_interval_seconds = int(val)
|
||||
if val := os.getenv("GRACE_PERIOD_MINUTES"):
|
||||
cfg.services.grace_period_minutes = int(val)
|
||||
if val := os.getenv("LOG_LEVEL"):
|
||||
cfg.logging.level = val
|
||||
return cfg
|
||||
52
src/infra_controller/controller.py
Normal file
52
src/infra_controller/controller.py
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from infra_controller.config import ControllerConfig
|
||||
from infra_controller.discovery import AppRegistration, DiscoveryManager
|
||||
from infra_controller.service_manager import ServiceManager
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
class InfraController:
|
||||
def __init__(self, cfg: ControllerConfig):
|
||||
self._cfg = cfg
|
||||
self._discovery = DiscoveryManager(cfg.discovery)
|
||||
self._services = ServiceManager(cfg.docker)
|
||||
|
||||
def run(self) -> None:
|
||||
while True:
|
||||
self.run_once()
|
||||
time.sleep(self._cfg.services.check_interval_seconds)
|
||||
|
||||
def run_once(self) -> None:
|
||||
discovered = self._discovery.discover_all()
|
||||
required = self._required_services(discovered)
|
||||
|
||||
for service in sorted(required):
|
||||
logger.info("Ensuring service: %s", service)
|
||||
self.ensure_service(service)
|
||||
|
||||
def ensure_service(self, service_name: str) -> None:
|
||||
res = self._services.apply_service(service_name)
|
||||
if res.returncode != 0:
|
||||
raise RuntimeError(res.stderr or res.stdout)
|
||||
|
||||
def _required_services(self, apps: dict[str, AppRegistration]) -> set[str]:
|
||||
required: set[str] = set()
|
||||
|
||||
for reg in apps.values():
|
||||
requires = reg.metadata.requires
|
||||
services = requires.get("services")
|
||||
if services is None:
|
||||
continue
|
||||
|
||||
if isinstance(services, list):
|
||||
for s in services:
|
||||
if isinstance(s, str) and s.strip():
|
||||
required.add(s.strip())
|
||||
elif isinstance(services, str) and services.strip():
|
||||
required.add(services.strip())
|
||||
|
||||
return required
|
||||
182
src/infra_controller/discovery.py
Normal file
182
src/infra_controller/discovery.py
Normal file
|
|
@ -0,0 +1,182 @@
|
|||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
import logging
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
|
||||
import tomllib
|
||||
import yaml
|
||||
|
||||
from infra_controller.config import DiscoveryConfig
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class InfraMetadata:
|
||||
project: str
|
||||
requires: dict[str, object]
|
||||
metadata: dict[str, object] = field(default_factory=dict)
|
||||
|
||||
@classmethod
|
||||
def from_file(cls, path: Path) -> "InfraMetadata":
|
||||
if path.suffix == ".toml":
|
||||
with open(path, "rb") as f:
|
||||
data = tomllib.load(f)
|
||||
elif path.suffix in {".yml", ".yaml"}:
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
data = yaml.safe_load(f) or {}
|
||||
else:
|
||||
raise ValueError(f"Unsupported format: {path}")
|
||||
|
||||
project = data.get("project") or path.parent.name
|
||||
requires = data.get("requires") or {}
|
||||
metadata = {k: v for k, v in data.items() if k not in {"project", "requires"}}
|
||||
return cls(project=str(project), requires=dict(requires), metadata=metadata)
|
||||
|
||||
|
||||
@dataclass
|
||||
class AppRegistration:
|
||||
name: str
|
||||
metadata: InfraMetadata
|
||||
last_seen: datetime
|
||||
discovery_method: str
|
||||
|
||||
|
||||
class DiscoveryManager:
|
||||
def __init__(self, config: DiscoveryConfig):
|
||||
self.config = config
|
||||
|
||||
def discover_all(self) -> dict[str, AppRegistration]:
|
||||
discovered: dict[str, AppRegistration] = {}
|
||||
|
||||
if self.config.file_based_enabled:
|
||||
discovered.update(self._discover_file_based())
|
||||
|
||||
if self.config.scan_enabled:
|
||||
discovered.update(self._discover_scan())
|
||||
|
||||
return discovered
|
||||
|
||||
def _discover_file_based(self) -> dict[str, AppRegistration]:
|
||||
apps: dict[str, AppRegistration] = {}
|
||||
watch_path = self.config.file_based_path
|
||||
|
||||
if not watch_path.exists():
|
||||
watch_path.mkdir(parents=True, exist_ok=True)
|
||||
return apps
|
||||
|
||||
for link_file in watch_path.glob("*.toml"):
|
||||
try:
|
||||
target = link_file.resolve() if link_file.is_symlink() else link_file
|
||||
if not target.exists():
|
||||
continue
|
||||
|
||||
md = InfraMetadata.from_file(target)
|
||||
apps[md.project] = AppRegistration(
|
||||
name=md.project,
|
||||
metadata=md,
|
||||
last_seen=datetime.now(),
|
||||
discovery_method="file_based",
|
||||
)
|
||||
logger.info("Discovered app via file: %s", md.project)
|
||||
except Exception as e:
|
||||
logger.error("Error reading %s: %s", link_file, e)
|
||||
|
||||
for link_file in list(watch_path.glob("*.yml")) + list(watch_path.glob("*.yaml")):
|
||||
try:
|
||||
if link_file.stem in apps:
|
||||
continue
|
||||
|
||||
target = link_file.resolve() if link_file.is_symlink() else link_file
|
||||
if not target.exists():
|
||||
continue
|
||||
|
||||
md = InfraMetadata.from_file(target)
|
||||
apps[md.project] = AppRegistration(
|
||||
name=md.project,
|
||||
metadata=md,
|
||||
last_seen=datetime.now(),
|
||||
discovery_method="file_based",
|
||||
)
|
||||
logger.info("Discovered app via file (YAML): %s", md.project)
|
||||
except Exception as e:
|
||||
logger.error("Error reading %s: %s", link_file, e)
|
||||
|
||||
return apps
|
||||
|
||||
def _discover_scan(self) -> dict[str, AppRegistration]:
|
||||
apps: dict[str, AppRegistration] = {}
|
||||
|
||||
for base_path in self.config.scan_paths:
|
||||
if not base_path.exists():
|
||||
continue
|
||||
|
||||
for infra_file in base_path.rglob(".infra.toml"):
|
||||
if self._should_exclude(infra_file):
|
||||
continue
|
||||
if not self._is_active_project(infra_file.parent):
|
||||
continue
|
||||
|
||||
try:
|
||||
md = InfraMetadata.from_file(infra_file)
|
||||
apps[md.project] = AppRegistration(
|
||||
name=md.project,
|
||||
metadata=md,
|
||||
last_seen=datetime.now(),
|
||||
discovery_method="scan",
|
||||
)
|
||||
logger.info("Discovered app via scan: %s", md.project)
|
||||
except Exception as e:
|
||||
logger.error("Error reading %s: %s", infra_file, e)
|
||||
|
||||
for infra_file in base_path.rglob(".infra.yml"):
|
||||
if self._should_exclude(infra_file):
|
||||
continue
|
||||
try:
|
||||
md = InfraMetadata.from_file(infra_file)
|
||||
except Exception as e:
|
||||
logger.error("Error reading %s: %s", infra_file, e)
|
||||
continue
|
||||
|
||||
if md.project in apps:
|
||||
continue
|
||||
if not self._is_active_project(infra_file.parent):
|
||||
continue
|
||||
|
||||
apps[md.project] = AppRegistration(
|
||||
name=md.project,
|
||||
metadata=md,
|
||||
last_seen=datetime.now(),
|
||||
discovery_method="scan",
|
||||
)
|
||||
logger.info("Discovered app via scan (YAML): %s", md.project)
|
||||
|
||||
return apps
|
||||
|
||||
def _should_exclude(self, path: Path) -> bool:
|
||||
return any(path.match(pattern) for pattern in self.config.exclude_patterns)
|
||||
|
||||
def _is_active_project(self, project_dir: Path) -> bool:
|
||||
git_head = project_dir / ".git" / "HEAD"
|
||||
if git_head.exists():
|
||||
mtime = datetime.fromtimestamp(git_head.stat().st_mtime)
|
||||
if (datetime.now() - mtime).total_seconds() < 24 * 60 * 60:
|
||||
return True
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
["pgrep", "-f", str(project_dir)],
|
||||
capture_output=True,
|
||||
timeout=5,
|
||||
text=True,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
return False
|
||||
66
src/infra_controller/service_manager.py
Normal file
66
src/infra_controller/service_manager.py
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from infra_controller.config import DockerComposeConfig
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ServiceResult:
|
||||
returncode: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
|
||||
|
||||
class ServiceManager:
|
||||
def __init__(self, docker: DockerComposeConfig):
|
||||
self._docker = docker
|
||||
|
||||
def service_dir_for_service(self, service_name: str) -> Path:
|
||||
return self._docker.base_dir / service_name
|
||||
|
||||
def _resolve_compose_file(self, service_dir: Path) -> Path:
|
||||
configured = service_dir / self._docker.compose_file
|
||||
if configured.exists():
|
||||
return configured
|
||||
|
||||
candidates = list(service_dir.glob("docker-compose*.yml")) + list(service_dir.glob("docker-compose*.yaml"))
|
||||
candidates = [p for p in candidates if p.is_file()]
|
||||
|
||||
if not candidates:
|
||||
raise FileNotFoundError(
|
||||
f"Compose file not found in {service_dir} (expected {self._docker.compose_file} or docker-compose*.yml/.yaml)"
|
||||
)
|
||||
|
||||
def rank(p: Path) -> tuple[int, str]:
|
||||
name = p.name
|
||||
if name == "docker-compose.yml":
|
||||
return (0, name)
|
||||
if name == "docker-compose.yaml":
|
||||
return (1, name)
|
||||
if name.endswith(".yml"):
|
||||
return (2, name)
|
||||
return (3, name)
|
||||
|
||||
return sorted(candidates, key=rank)[0]
|
||||
|
||||
def apply_service(self, service_name: str) -> ServiceResult:
|
||||
service_dir = self.service_dir_for_service(service_name)
|
||||
if not service_dir.exists():
|
||||
raise FileNotFoundError(f"Service directory not found: {service_dir}")
|
||||
|
||||
compose_file = self._resolve_compose_file(service_dir)
|
||||
|
||||
cmd = [
|
||||
"docker",
|
||||
"compose",
|
||||
"-f",
|
||||
str(compose_file),
|
||||
"up",
|
||||
"-d",
|
||||
]
|
||||
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True, cwd=str(service_dir))
|
||||
return ServiceResult(returncode=proc.returncode, stdout=proc.stdout, stderr=proc.stderr)
|
||||
27
systemd/infra-controller-once.service
Normal file
27
systemd/infra-controller-once.service
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
[Unit]
|
||||
Description=Infrastructure Controller (one-shot)
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=infractl
|
||||
Group=infractl
|
||||
|
||||
EnvironmentFile=/etc/infra-controller/controller.env
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
|
||||
WorkingDirectory=/opt/infra-controller
|
||||
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/run/active-apps /var/lib/infra-controller
|
||||
CapabilityBoundingSet=
|
||||
|
||||
ExecStart=/opt/infra-controller/venv/bin/infra-controller --once
|
||||
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=infra-controller
|
||||
11
systemd/infra-controller.env
Normal file
11
systemd/infra-controller.env
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
CONFIG_PATH=/etc/infra-controller/config.toml
|
||||
|
||||
ACTIVE_APPS_DIR=/var/run/active-apps
|
||||
|
||||
DOCKER_BASE_DIR=/opt
|
||||
DOCKER_COMPOSE_FILE=docker-compose.yml
|
||||
|
||||
CHECK_INTERVAL=60
|
||||
GRACE_PERIOD_MINUTES=15
|
||||
|
||||
LOG_LEVEL=INFO
|
||||
11
systemd/infra-controller.path
Normal file
11
systemd/infra-controller.path
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
[Unit]
|
||||
Description=Run infra-controller when active app registrations change
|
||||
|
||||
[Path]
|
||||
PathExists=/var/run/active-apps
|
||||
PathModified=/var/run/active-apps
|
||||
DirectoryNotEmpty=/var/run/active-apps
|
||||
Unit=infra-controller-once.service
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
35
systemd/infra-controller.service
Normal file
35
systemd/infra-controller.service
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
[Unit]
|
||||
Description=Infrastructure Controller
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=infractl
|
||||
Group=infractl
|
||||
Restart=on-failure
|
||||
RestartSec=10s
|
||||
|
||||
EnvironmentFile=/etc/infra-controller/controller.env
|
||||
Environment=PYTHONUNBUFFERED=1
|
||||
|
||||
WorkingDirectory=/opt/infra-controller
|
||||
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=strict
|
||||
ProtectHome=true
|
||||
ReadWritePaths=/var/run/active-apps /var/lib/infra-controller
|
||||
CapabilityBoundingSet=
|
||||
|
||||
# Execution
|
||||
ExecStart=/opt/infra-controller/venv/bin/infra-controller
|
||||
ExecReload=/bin/kill -HUP $MAINPID
|
||||
|
||||
# Logging
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
SyslogIdentifier=infra-controller
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
9
tests/test_config.py
Normal file
9
tests/test_config.py
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
from infra_controller.config import load_config
|
||||
|
||||
|
||||
def test_load_config_defaults(tmp_path, monkeypatch):
|
||||
cfg_path = tmp_path / "cfg.yml"
|
||||
cfg_path.write_text("{}\n")
|
||||
monkeypatch.setenv("CONFIG_PATH", str(cfg_path))
|
||||
cfg = load_config()
|
||||
assert str(cfg.docker.base_dir).endswith("/opt")
|
||||
Loading…
Reference in a new issue