Compare commits
9 Commits
777926487b
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 98ba5004ad | |||
| c64cd4ca8b | |||
| f81d6186ab | |||
| f981ca1bf1 | |||
| baa1f58ad0 | |||
| 6f42d4c318 | |||
| 176c8fd71c | |||
| 0773f721ea | |||
| 48df10b17e |
@@ -23,5 +23,7 @@ SIMULATION_SEED=42
|
||||
SIMULATION_TIME_SCALE=10
|
||||
SIMULATOR_ADMIN_ENABLED=false
|
||||
SIMULATOR_ADMIN_TOKEN=replace-with-a-long-random-secret
|
||||
PROXMOXER_HOST=tls-gateway
|
||||
PROXMOXER_HOST=
|
||||
PROXMOXER_PORT=8443
|
||||
# For proxmoxer (HTTPS-only): docker compose --profile tls up -d
|
||||
# then PROXMOXER_HOST=tls-gateway PROXMOXER_PORT=8443
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: ci-${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
validate:
|
||||
name: Compose / Helm validate
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Validate Compose files
|
||||
run: |
|
||||
docker compose -f docker-compose.yml config --quiet
|
||||
docker compose -f docker-compose.release.yml config --quiet
|
||||
|
||||
- uses: azure/setup-helm@v4
|
||||
with:
|
||||
version: v3.16.4
|
||||
|
||||
- name: Lint and render Helm chart
|
||||
run: |
|
||||
make helm-lint
|
||||
make helm-template >/dev/null
|
||||
|
||||
quality:
|
||||
name: make ci
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 90
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Prepare lab env
|
||||
run: test -f .env || cp .env.example .env
|
||||
|
||||
- name: Build runtime and development images
|
||||
run: make install
|
||||
|
||||
- name: Ruff + mypy + offline pytest + API surface probe
|
||||
run: make ci
|
||||
|
||||
- name: Upload coverage XML
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: coverage-xml
|
||||
path: coverage.xml
|
||||
if-no-files-found: ignore
|
||||
@@ -67,9 +67,12 @@ hypothesis/
|
||||
*.sublime-project
|
||||
*.sublime-workspace
|
||||
|
||||
# Docker / local runtime
|
||||
# Docker / local runtime (`make up-local` writes port remaps here)
|
||||
*.log
|
||||
docker-compose.override.yml
|
||||
|
||||
# Lab probe output (regenerated by scripts/probe_api_surface.py)
|
||||
evidence/_api_surface_probe.json
|
||||
|
||||
# Pulumi suite reports (HTML/JSON/JUnit from make pulumi-tests)
|
||||
pulumi-tests/reports/*
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# Changelog
|
||||
|
||||
All notable changes to this project are documented in this file.
|
||||
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [0.1.0] - 2026-07-17
|
||||
|
||||
### Added
|
||||
|
||||
- Stateful Proxmox VE API simulator backed by PostgreSQL with durable UPID task
|
||||
workers.
|
||||
- Imported official API contracts for PVE majors **6.4-15**, **7.4-16**,
|
||||
**8.4.5**, and **9.2.3** with full handler registration for declared methods.
|
||||
- Interactive Web UI (catalog, runtime contract Apply, task monitor).
|
||||
- Development Compose stack with plain HTTP on host `:8006` (real PVE port; real
|
||||
PVE uses HTTPS). Optional TLS gateway via `--profile tls` on `:8443`.
|
||||
- Published runtime image workflow (`make release`) for
|
||||
[`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator).
|
||||
- `docker-compose.release.yml` + Helm chart with optional Ingress / cert-manager.
|
||||
- Bilingual documentation (English + Russian) and client cookbooks.
|
||||
- GitHub Actions CI (`make ci`, Compose/Helm validation).
|
||||
|
||||
### Security
|
||||
|
||||
- Explicit lab-only threat model: default secrets and open `/ui` / `/admin`
|
||||
helpers are for local/CI use. See [SECURITY.md](SECURITY.md).
|
||||
|
||||
[0.1.0]: https://github.com/sergeyantropoff/proxmox-api-simulator/releases/tag/v0.1.0
|
||||
@@ -0,0 +1,69 @@
|
||||
**Language / Язык:** [English](CONTRIBUTING.md) | [Русский](CONTRIBUTING.ru.md)
|
||||
|
||||
# Contributing
|
||||
|
||||
Thanks for helping improve the Proxmox VE API laboratory simulator.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker + Docker Compose
|
||||
- `make`
|
||||
- No local Python toolchain required for day-to-day work (tools run in Compose)
|
||||
|
||||
## Local loop
|
||||
|
||||
```bash
|
||||
cp -n .env.example .env
|
||||
make install
|
||||
make up
|
||||
make seed PROFILE=small
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
```
|
||||
|
||||
Primary HTTP endpoint (same port as real PVE): `http://localhost:8006/`
|
||||
|
||||
Optional HTTPS for proxmoxer: `docker compose --profile tls` →
|
||||
`https://localhost:8443/` (self-signed — use `curl -sk` or accept the browser
|
||||
warning).
|
||||
|
||||
## Quality gates (must pass before a PR)
|
||||
|
||||
```bash
|
||||
make ci # ruff format/check + mypy + offline pytest + surface probe
|
||||
make ci-all # also remaining integration + proxmoxer compatibility
|
||||
make helm-lint # chart lint (+ ingress example values)
|
||||
make pulumi-tests # Pulumi surface (majors 6–9) + pulumi-proxmoxve lifecycle
|
||||
```
|
||||
|
||||
GitHub Actions runs `make ci` plus Compose/Helm validation on every push and PR
|
||||
to `main`. Run `make ci-all`, `make helm-lint`, and `make pulumi-tests` locally
|
||||
before larger API or client-facing changes.
|
||||
|
||||
Suite layout, Make targets, and the latest recorded local results:
|
||||
[docs/testing.md](docs/testing.md).
|
||||
|
||||
## Project rules worth remembering
|
||||
|
||||
1. Mutations must **persist to PostgreSQL** (tables and/or jsonb metadata).
|
||||
2. Do **not** add user-facing “not supported in the simulator” style errors —
|
||||
see `.cursor/rules/durable-simulator.mdc`.
|
||||
3. Prefer matching Proxmox request/response shapes from the contract snapshot.
|
||||
4. Keep EN and RU docs in sync when you change operator-facing behaviour.
|
||||
|
||||
## Docs
|
||||
|
||||
- Index: [docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md)
|
||||
- Security / lab threat model: [SECURITY.md](SECURITY.md)
|
||||
|
||||
## Releases
|
||||
|
||||
Maintainers publish the runtime image with:
|
||||
|
||||
```bash
|
||||
docker login
|
||||
make release # pushes inecs/proxmox-api-simulator:<version> (+ :latest)
|
||||
```
|
||||
|
||||
After a public release, paste the overview from
|
||||
[docs/docker-hub-overview.md](docs/docker-hub-overview.md) into the Docker Hub
|
||||
repository description if it drifted.
|
||||
@@ -0,0 +1,66 @@
|
||||
**Language / Язык:** [English](CONTRIBUTING.md) | [Русский](CONTRIBUTING.ru.md)
|
||||
|
||||
# Участие в разработке
|
||||
|
||||
Спасибо за помощь с лабораторным симулятором Proxmox VE API.
|
||||
|
||||
## Требования
|
||||
|
||||
- Docker + Docker Compose
|
||||
- `make`
|
||||
- Локальный Python для повседневной работы не нужен (инструменты в Compose)
|
||||
|
||||
## Локальный цикл
|
||||
|
||||
```bash
|
||||
cp -n .env.example .env
|
||||
make install
|
||||
make up
|
||||
make seed PROFILE=small
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
```
|
||||
|
||||
Основной HTTPS-эндпоинт (как у реального PVE): `http://localhost:8006/`
|
||||
(self-signed — `curl -sk` или принять предупреждение в браузере).
|
||||
|
||||
## Quality gates (обязательны перед PR)
|
||||
|
||||
```bash
|
||||
make ci # ruff format/check + mypy + offline pytest + surface probe
|
||||
make ci-all # ещё integration + proxmoxer compatibility
|
||||
make helm-lint # lint chart (+ ingress example values)
|
||||
make pulumi-tests # Pulumi surface (majors 6–9) + pulumi-proxmoxve lifecycle
|
||||
```
|
||||
|
||||
GitHub Actions на каждый push/PR в `main` запускает `make ci` и проверку
|
||||
Compose/Helm. Перед крупными API/клиентскими изменениями локально прогоняйте
|
||||
`make ci-all`, `make helm-lint` и `make pulumi-tests`.
|
||||
|
||||
Карта наборов, Make-цели и последние записанные локальные результаты:
|
||||
[docs/ru/testing.md](docs/ru/testing.md).
|
||||
|
||||
## Важные правила проекта
|
||||
|
||||
1. Мутации должны **persist в PostgreSQL** (таблицы и/или jsonb metadata).
|
||||
2. Не добавляйте user-facing сообщения вроде «not supported in the simulator» —
|
||||
см. `.cursor/rules/durable-simulator.mdc`.
|
||||
3. Сохраняйте формы запросов/ответов Proxmox из снимка контракта.
|
||||
4. При изменении операторского поведения синхронизируйте EN и RU docs.
|
||||
|
||||
## Документация
|
||||
|
||||
- Индекс: [docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md)
|
||||
- Безопасность / модель угроз лаборатории: [SECURITY.md](SECURITY.md)
|
||||
|
||||
## Релизы
|
||||
|
||||
Maintainers публикуют runtime-образ так:
|
||||
|
||||
```bash
|
||||
docker login
|
||||
make release # пушит inecs/proxmox-api-simulator:<version> (+ :latest)
|
||||
```
|
||||
|
||||
После публичного релиза при необходимости вставьте overview из
|
||||
[docs/docker-hub-overview.md](docs/docker-hub-overview.md) в описание репозитория
|
||||
Docker Hub.
|
||||
@@ -16,7 +16,9 @@ FROM python:3.13-slim AS runtime
|
||||
ARG APP_VERSION=0.1.0
|
||||
LABEL org.opencontainers.image.title="proxmox-api-simulator" \
|
||||
org.opencontainers.image.version="$APP_VERSION" \
|
||||
org.opencontainers.image.source="https://github.com/example/proxmox-api-simulator"
|
||||
org.opencontainers.image.description="Stateful Proxmox VE API simulator for labs and CI" \
|
||||
org.opencontainers.image.source="https://github.com/sergeyantropoff/proxmox-api-simulator" \
|
||||
org.opencontainers.image.licenses="Apache-2.0"
|
||||
ENV PATH="/opt/venv/bin:$PATH" \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PYTHONUNBUFFERED=1 \
|
||||
@@ -25,6 +27,7 @@ ENV PATH="/opt/venv/bin:$PATH" \
|
||||
RUN groupadd --system --gid 10001 simulator \
|
||||
&& useradd --system --uid 10001 --gid simulator --home-dir /app --no-create-home simulator
|
||||
COPY --from=builder /opt/venv /opt/venv
|
||||
COPY VERSION /app/VERSION
|
||||
COPY contracts/e61a893e996d05d376579226e7dfbedbcfce8b71787adacffbc557e6e35901c1/snapshot.json /app/contracts/pve-9.2.3.json
|
||||
COPY evidence/ /app/evidence/
|
||||
WORKDIR /app
|
||||
|
||||
@@ -0,0 +1,205 @@
|
||||
// Proxmox API Simulator — CI/CD: multi-arch push (Harbor + Docker Hub) + deploy
|
||||
//
|
||||
// Один файл: Build (DinD) → Deploy (kubectl set image в релиз simulators).
|
||||
// Первый install lab — через DevOpsTools/K3S: make addon-simulators
|
||||
// (Ingress https://proxmox.devops.org.ru уже из addon values).
|
||||
//
|
||||
// Версия образа = VERSION из коммита. Локальный bump: make bump-patch / make push.
|
||||
// Теги: :{VERSION} и :latest (без sha).
|
||||
//
|
||||
// Credentials (Global):
|
||||
// harbor-devops-tools-push-pull-access — Harbor devops-tools (robot)
|
||||
// docker-hub — Docker Hub (inecs)
|
||||
// k3s-kubeconfig — kubeconfig к K3S
|
||||
// ssh-gitea-key — SCM checkout Gitea (в job)
|
||||
|
||||
pipeline {
|
||||
agent none
|
||||
|
||||
options {
|
||||
buildDiscarder(logRotator(numToKeepStr: '20'))
|
||||
disableConcurrentBuilds()
|
||||
timeout(time: 60, unit: 'MINUTES')
|
||||
timestamps()
|
||||
}
|
||||
|
||||
environment {
|
||||
HARBOR_REGISTRY = 'hub.antropoff.ru'
|
||||
HARBOR_IMAGE = 'hub.antropoff.ru/devops-tools/proxmox-api-simulator'
|
||||
DOCKERHUB_IMAGE = 'inecs/proxmox-api-simulator'
|
||||
BUILDX_BUILDER = "jenkins-proxmox-api-simulator-${env.BUILD_NUMBER}"
|
||||
HELM_NAMESPACE = 'simulators'
|
||||
DEPLOYMENT_NAME = 'simulators-proxmox'
|
||||
INGRESS_HOST = 'proxmox.devops.org.ru'
|
||||
TZ = 'Europe/Moscow'
|
||||
}
|
||||
|
||||
stages {
|
||||
stage('Build & push') {
|
||||
when {
|
||||
anyOf {
|
||||
branch 'main'
|
||||
branch 'master'
|
||||
}
|
||||
}
|
||||
agent { label 'docker' }
|
||||
stages {
|
||||
stage('Checkout') {
|
||||
steps {
|
||||
checkout scm
|
||||
}
|
||||
}
|
||||
|
||||
stage('Version') {
|
||||
steps {
|
||||
script {
|
||||
env.IMAGE_VERSION = readFile('VERSION').trim()
|
||||
if (!env.IMAGE_VERSION) {
|
||||
error('VERSION file is empty')
|
||||
}
|
||||
echo "IMAGE_VERSION from VERSION → ${env.IMAGE_VERSION}"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
stage('Buildx push') {
|
||||
steps {
|
||||
withCredentials([
|
||||
usernamePassword(
|
||||
credentialsId: 'harbor-devops-tools-push-pull-access',
|
||||
usernameVariable: 'HARBOR_USER',
|
||||
passwordVariable: 'HARBOR_PASS'
|
||||
),
|
||||
usernamePassword(
|
||||
credentialsId: 'docker-hub',
|
||||
usernameVariable: 'DOCKERHUB_USER',
|
||||
passwordVariable: 'DOCKERHUB_PASS'
|
||||
)
|
||||
]) {
|
||||
container('docker') {
|
||||
sh '''
|
||||
set -eux
|
||||
|
||||
test -n "${IMAGE_VERSION}"
|
||||
echo "Building tags: ${IMAGE_VERSION}, latest (amd64+arm64)"
|
||||
|
||||
echo "$HARBOR_PASS" | docker login "$HARBOR_REGISTRY" -u "$HARBOR_USER" --password-stdin
|
||||
echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USER" --password-stdin
|
||||
|
||||
docker buildx rm "$BUILDX_BUILDER" 2>/dev/null || true
|
||||
docker buildx create --name "$BUILDX_BUILDER" --driver docker-container --use
|
||||
docker buildx inspect --bootstrap >/dev/null
|
||||
|
||||
docker buildx build \
|
||||
--platform linux/amd64,linux/arm64 \
|
||||
--target runtime \
|
||||
--build-arg "APP_VERSION=${IMAGE_VERSION}" \
|
||||
--provenance=false --sbom=false --push \
|
||||
-t "${HARBOR_IMAGE}:${IMAGE_VERSION}" \
|
||||
-t "${HARBOR_IMAGE}:latest" \
|
||||
-t "${DOCKERHUB_IMAGE}:${IMAGE_VERSION}" \
|
||||
-t "${DOCKERHUB_IMAGE}:latest" \
|
||||
-f Dockerfile .
|
||||
|
||||
echo "--- Harbor ---"
|
||||
docker buildx imagetools inspect "${HARBOR_IMAGE}:${IMAGE_VERSION}" | sed -n '1,40p'
|
||||
echo "--- Docker Hub ---"
|
||||
docker buildx imagetools inspect "${DOCKERHUB_IMAGE}:${IMAGE_VERSION}" | sed -n '1,40p'
|
||||
|
||||
docker buildx rm "$BUILDX_BUILDER" || true
|
||||
'''
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
stage('Deploy') {
|
||||
steps {
|
||||
withCredentials([
|
||||
file(credentialsId: 'k3s-kubeconfig', variable: 'KUBECONFIG')
|
||||
]) {
|
||||
container('docker') {
|
||||
sh '''
|
||||
set -eux
|
||||
|
||||
if ! command -v kubectl >/dev/null 2>&1; then
|
||||
apk add --no-cache curl >/dev/null
|
||||
ARCH="$(uname -m)"
|
||||
case "$ARCH" in
|
||||
x86_64) ARCH=amd64 ;;
|
||||
aarch64|arm64) ARCH=arm64 ;;
|
||||
esac
|
||||
KVER="$(curl -fsSL https://dl.k8s.io/release/stable.txt)"
|
||||
curl -fsSLo /usr/local/bin/kubectl \
|
||||
"https://dl.k8s.io/release/${KVER}/bin/linux/${ARCH}/kubectl"
|
||||
chmod +x /usr/local/bin/kubectl
|
||||
fi
|
||||
|
||||
kubectl version --client || true
|
||||
|
||||
if ! kubectl -n "$HELM_NAMESPACE" get deploy "$DEPLOYMENT_NAME" >/dev/null 2>&1; then
|
||||
echo "Deployment ${HELM_NAMESPACE}/${DEPLOYMENT_NAME} not found." >&2
|
||||
echo "Сначала: make addon-simulators (Ingress ${INGRESS_HOST})." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Rolling ${DOCKERHUB_IMAGE}:${IMAGE_VERSION} → ${HELM_NAMESPACE}/${DEPLOYMENT_NAME}"
|
||||
echo "Public URL: https://${INGRESS_HOST}/"
|
||||
|
||||
# POSIX sh (Alpine ash): no bash arrays. One set-image when migrate exists.
|
||||
INITS="$(kubectl -n "$HELM_NAMESPACE" get deploy "$DEPLOYMENT_NAME" \
|
||||
-o jsonpath='{.spec.template.spec.initContainers[*].name}' || true)"
|
||||
if printf '%s\n' $INITS | grep -qx migrate; then
|
||||
kubectl -n "$HELM_NAMESPACE" set image \
|
||||
"deployment/${DEPLOYMENT_NAME}" \
|
||||
"simulator=${DOCKERHUB_IMAGE}:${IMAGE_VERSION}" \
|
||||
"migrate=${DOCKERHUB_IMAGE}:${IMAGE_VERSION}"
|
||||
else
|
||||
kubectl -n "$HELM_NAMESPACE" set image \
|
||||
"deployment/${DEPLOYMENT_NAME}" \
|
||||
"simulator=${DOCKERHUB_IMAGE}:${IMAGE_VERSION}"
|
||||
fi
|
||||
|
||||
kubectl -n "$HELM_NAMESPACE" rollout status \
|
||||
"deployment/${DEPLOYMENT_NAME}" --timeout=1200s
|
||||
kubectl -n "$HELM_NAMESPACE" get deploy,po,ing -o wide
|
||||
'''
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
post {
|
||||
always {
|
||||
script {
|
||||
try {
|
||||
container('docker') {
|
||||
sh 'docker buildx rm "$BUILDX_BUILDER" 2>/dev/null || true'
|
||||
}
|
||||
} catch (Ignored) {
|
||||
// pod may already be gone
|
||||
}
|
||||
}
|
||||
}
|
||||
success {
|
||||
echo "✓ ${DOCKERHUB_IMAGE}:{${IMAGE_VERSION},latest} → https://${INGRESS_HOST}/"
|
||||
}
|
||||
failure {
|
||||
echo "✗ Build/push/deploy failed — см. лог"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
post {
|
||||
success {
|
||||
echo "✓ Version: ${IMAGE_VERSION}"
|
||||
echo "✓ Harbor: ${HARBOR_IMAGE}:{${IMAGE_VERSION},latest}"
|
||||
echo "✓ Hub: ${DOCKERHUB_IMAGE}:{${IMAGE_VERSION},latest}"
|
||||
echo "✓ Live: https://${INGRESS_HOST}/"
|
||||
}
|
||||
failure {
|
||||
echo "✗ Pipeline failed"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,19 +4,22 @@ SERVICE_SIM := simulator
|
||||
PYTEST_OFFLINE := -m "not integration and not compatibility"
|
||||
|
||||
# Docker Hub release image (runtime target only — not the local bind-mount "dev" image).
|
||||
# SemVer source of truth: VERSION (synced to pyproject + Helm via bump targets).
|
||||
DOCKERHUB_USER ?= inecs
|
||||
IMAGE_NAME ?= proxmox-api-simulator
|
||||
VERSION ?= $(shell sed -n 's/^version = "\(.*\)"/\1/p' pyproject.toml)
|
||||
VERSION ?= $(shell cat VERSION 2>/dev/null || echo 0.1.0)
|
||||
DOCKER_IMAGE ?= $(DOCKERHUB_USER)/$(IMAGE_NAME)
|
||||
PUSH_LATEST ?= 1
|
||||
|
||||
COMPOSE_RELEASE ?= $(COMPOSE) -f docker-compose.release.yml
|
||||
HELM_CHART ?= ./helm/proxmox-api-simulator
|
||||
|
||||
.PHONY: help install format lint typecheck test test-unit test-integration test-contract test-compatibility test-surface evidence coverage run dev up down restart logs docker-build docker-up docker-down docker-logs docker-restart db-up db-down db-migrate db-reset api-import api-diff seed clean ci ci-all shell release release-build release-up release-down release-seed helm-deps helm-template
|
||||
.PHONY: help install format lint typecheck test test-unit test-integration test-contract test-compatibility test-surface evidence coverage run dev up up-local down down-local restart logs docker-build docker-up docker-down docker-logs docker-restart db-up db-down db-migrate db-reset api-import api-diff seed clean ci ci-all shell release release-build release-up release-down release-seed helm-deps helm-template helm-lint pulumi-tests push bump-patch bump-minor version-commit
|
||||
|
||||
help: ## Show available commands
|
||||
@awk 'BEGIN {FS = ":.*## "}; /^[a-zA-Z0-9_-]+:.*## / {printf "%-22s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
|
||||
@echo
|
||||
@echo "Version: $$(cat VERSION 2>/dev/null || echo 0.1.0) (make bump-patch | bump-minor)"
|
||||
|
||||
install: ## Build runtime and development images
|
||||
@test -f .env || cp .env.example .env
|
||||
@@ -45,11 +48,16 @@ test-integration: ## Run tests that require PostgreSQL
|
||||
test-contract: ## Run offline API contract tests
|
||||
$(COMPOSE) run --rm --no-deps $(SERVICE_DEV) pytest -m contract
|
||||
|
||||
test-compatibility: ## Run proxmoxer smoke flow against the Compose stack
|
||||
test-compatibility: ## Run proxmoxer smoke flow (needs --profile tls; proxmoxer is HTTPS-only)
|
||||
@test -f .env || cp .env.example .env
|
||||
$(COMPOSE) up -d --build --wait
|
||||
$(COMPOSE) run --rm --entrypoint python $(SERVICE_SIM) -m app.simulation.seed_cli
|
||||
$(COMPOSE) run --rm $(SERVICE_DEV) pytest -m compatibility
|
||||
$(COMPOSE) --profile tls up -d --build --wait
|
||||
# Medium profile provides pve1/pve2/pve3 required by the proxmoxer migration smoke.
|
||||
$(COMPOSE) run --rm -e SEED_PROFILE=medium --entrypoint python $(SERVICE_SIM) \
|
||||
-m app.simulation.seed_cli
|
||||
$(COMPOSE) run --rm \
|
||||
-e PROXMOXER_HOST=tls-gateway \
|
||||
-e PROXMOXER_PORT=8443 \
|
||||
$(SERVICE_DEV) pytest -m compatibility
|
||||
|
||||
test-surface: ## Probe every declared method on majors 6-9 (0x501 / 0xexception)
|
||||
@test -f .env || cp .env.example .env
|
||||
@@ -66,13 +74,21 @@ run: ## Run the application in the foreground
|
||||
@test -f .env || cp .env.example .env
|
||||
$(COMPOSE) up --build
|
||||
|
||||
up: ## Start PostgreSQL, simulator, and TLS gateway
|
||||
up: ## Start PostgreSQL and simulator (plain HTTP :8006)
|
||||
@test -f .env || cp .env.example .env
|
||||
$(COMPOSE) up -d --build --wait
|
||||
|
||||
up-local: ## Start locally; remap busy ports via gitignored docker-compose.override.yml
|
||||
@COMPOSE="$(COMPOSE)" ./scripts/up-local.sh
|
||||
|
||||
down: ## Stop local services
|
||||
$(COMPOSE) down
|
||||
|
||||
down-local: ## Stop local stack and remove gitignored docker-compose.override.yml
|
||||
@$(COMPOSE) down
|
||||
@rm -f docker-compose.override.yml
|
||||
@echo "Stopped local stack and removed docker-compose.override.yml"
|
||||
|
||||
restart: ## Rebuild and restart the stack
|
||||
@test -f .env || cp .env.example .env
|
||||
$(COMPOSE) up -d --build --force-recreate --wait
|
||||
@@ -90,8 +106,8 @@ docker-build: ## Build runtime and development images
|
||||
|
||||
docker-up: up ## Alias for up
|
||||
|
||||
docker-restart: ## Rebuild and recreate simulator and TLS gateway only
|
||||
$(COMPOSE) up -d --build --force-recreate simulator tls-gateway
|
||||
docker-restart: ## Rebuild and recreate the simulator
|
||||
$(COMPOSE) up -d --build --force-recreate simulator
|
||||
|
||||
docker-down: down ## Alias for down
|
||||
|
||||
@@ -122,7 +138,11 @@ api-diff: ## Compare API snapshots
|
||||
|
||||
seed: ## Seed simulation data
|
||||
@test -f .env || cp .env.example .env
|
||||
SEED_PROFILE="$${PROFILE:-small}" $(COMPOSE) run --rm --entrypoint python $(SERVICE_SIM) -m app.simulation.seed_cli
|
||||
$(COMPOSE) run --rm \
|
||||
-e SEED_PROFILE="$${PROFILE:-small}" \
|
||||
-e SEED_LARGE_NODES="$${SEED_LARGE_NODES:-10}" \
|
||||
-e SEED_LARGE_RESOURCES="$${SEED_LARGE_RESOURCES:-1000}" \
|
||||
--entrypoint python $(SERVICE_SIM) -m app.simulation.seed_cli
|
||||
|
||||
shell: ## Open an interactive shell in the development container
|
||||
$(COMPOSE) run --rm --no-deps $(SERVICE_DEV) bash
|
||||
@@ -170,14 +190,64 @@ release-down: ## Stop the published Hub stack
|
||||
$(COMPOSE_RELEASE) down
|
||||
|
||||
release-seed: ## Seed the published Hub stack (PROFILE=small by default)
|
||||
SEED_PROFILE="$${PROFILE:-small}" IMAGE_TAG="$${IMAGE_TAG:-$(VERSION)}" DOCKER_IMAGE="$(DOCKER_IMAGE)" \
|
||||
$(COMPOSE_RELEASE) run --rm --entrypoint python simulator -m app.simulation.seed_cli
|
||||
IMAGE_TAG="$${IMAGE_TAG:-$(VERSION)}" DOCKER_IMAGE="$(DOCKER_IMAGE)" \
|
||||
$(COMPOSE_RELEASE) run --rm \
|
||||
-e SEED_PROFILE="$${PROFILE:-small}" \
|
||||
-e SEED_LARGE_NODES="$${SEED_LARGE_NODES:-10}" \
|
||||
-e SEED_LARGE_RESOURCES="$${SEED_LARGE_RESOURCES:-1000}" \
|
||||
--entrypoint python simulator -m app.simulation.seed_cli
|
||||
|
||||
helm-deps: ## No-op placeholder (chart has no OCI dependencies)
|
||||
@echo "Chart $(HELM_CHART) vendors PostgreSQL templates; no helm dependency update required."
|
||||
|
||||
helm-lint: ## Lint the Helm chart (requires helm)
|
||||
helm lint $(HELM_CHART)
|
||||
helm lint $(HELM_CHART) -f $(HELM_CHART)/values-ingress-example.yaml \
|
||||
--set certManager.email=docs@example.com \
|
||||
--set secret.ticketSigningKey=docs-only-signing-key
|
||||
|
||||
helm-template: ## Render Helm manifests locally (requires helm)
|
||||
helm template pve-sim $(HELM_CHART) \
|
||||
-f $(HELM_CHART)/values-ingress-example.yaml \
|
||||
--set certManager.email=docs@example.com \
|
||||
--set secret.ticketSigningKey=docs-only-signing-key
|
||||
|
||||
pulumi-tests: ## Full Pulumi suite (surface majors 6–9 + lifecycle, HTML report)
|
||||
$(MAKE) -C pulumi-tests up
|
||||
$(MAKE) -C pulumi-tests test
|
||||
|
||||
bump-patch: ## VERSION +0.0.1 (pyproject + Helm)
|
||||
@PYTHONPATH=. python3 scripts/bump_version.py patch
|
||||
@echo "VERSION → $$(cat VERSION)"
|
||||
|
||||
bump-minor: ## VERSION +0.1.0 (pyproject + Helm)
|
||||
@PYTHONPATH=. python3 scripts/bump_version.py minor
|
||||
@echo "VERSION → $$(cat VERSION)"
|
||||
|
||||
version-commit: ## Stage version files and commit "Bump version to …"
|
||||
@git add VERSION pyproject.toml helm/proxmox-api-simulator/Chart.yaml helm/proxmox-api-simulator/values.yaml
|
||||
@if git diff --cached --quiet; then \
|
||||
echo "Nothing to commit for version files."; \
|
||||
else \
|
||||
git commit -m "Bump version to $$(cat VERSION)."; \
|
||||
fi
|
||||
|
||||
push: ## bump-patch + commit version + push origin (GitHub + antropoff.ru)
|
||||
@$(MAKE) bump-patch
|
||||
@$(MAKE) version-commit
|
||||
@set -e; \
|
||||
git add .; \
|
||||
echo "=== staged ==="; \
|
||||
git status --short; \
|
||||
echo; \
|
||||
if git diff --cached --quiet; then \
|
||||
echo "Nothing else to commit — pushing current branch."; \
|
||||
else \
|
||||
echo "Enter commit message for remaining changes, then Ctrl-D:"; \
|
||||
msg=$$(cat </dev/tty); \
|
||||
if [ -z "$$msg" ]; then echo "Empty commit message, aborting." >&2; exit 1; fi; \
|
||||
git commit -m "$$msg"; \
|
||||
fi; \
|
||||
echo "Pushing to both remotes:"; \
|
||||
git remote get-url --push --all origin | sed 's/^/ - /'; \
|
||||
git push origin HEAD
|
||||
|
||||
@@ -1,9 +1,16 @@
|
||||
**Language / Язык:** [English](README.md) | [Русский](README.ru.md)
|
||||
|
||||
# proxmox-api-simulator
|
||||
|
||||
Stateful asynchronous [Proxmox VE](https://www.proxmox.com/) API simulator for
|
||||
testing API clients and infrastructure tooling without a real hypervisor
|
||||
cluster.
|
||||
|
||||
> **Laboratory / CI only.** Default credentials, signing keys, and open UI/admin
|
||||
> helpers are intentional lab defaults. Do **not** expose this stack to the
|
||||
> public Internet without replacing secrets and adding your own controls.
|
||||
> See [SECURITY.md](SECURITY.md) and [Security](docs/security.md).
|
||||
|
||||
The simulator is backed by PostgreSQL, driven by imported official API
|
||||
contracts, and exposes the same `/api2/json` and `/api2/extjs` surfaces as
|
||||
Proxmox VE. Semantic handlers persist mutations; long-running work returns
|
||||
@@ -37,13 +44,16 @@ Image: [`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-ap
|
||||
|
||||
### Docker Compose
|
||||
|
||||
Needs a checkout that includes `docker-compose.release.yml`. Do not publish host
|
||||
`:8006` beyond a trusted lab without rotating `TICKET_SIGNING_KEY` / DB password.
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.release.yml up -d
|
||||
docker compose -f docker-compose.release.yml run --rm --entrypoint python \
|
||||
simulator -m app.simulation.seed_cli
|
||||
|
||||
curl http://localhost:8006/health/ready
|
||||
curl http://localhost:8006/api2/json/version
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
curl -sS http://localhost:8006/api2/json/version
|
||||
```
|
||||
|
||||
Or: `make release-up && make release-seed PROFILE=small`
|
||||
@@ -57,10 +67,14 @@ helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
|
||||
--set certManager.email=you@example.com \
|
||||
--set ingress.hosts[0].host=pve-sim.example.com \
|
||||
--set ingress.tls[0].hosts[0]=pve-sim.example.com \
|
||||
--set secret.ticketSigningKey="$(openssl rand -hex 32)"
|
||||
--set secret.ticketSigningKey="$(openssl rand -hex 32)" \
|
||||
--set postgresql.auth.password="$(openssl rand -hex 16)"
|
||||
```
|
||||
|
||||
Requires an Ingress controller and cert-manager. Details:
|
||||
Requires an Ingress controller and cert-manager. The chart Service speaks
|
||||
**HTTP** `:8006`; TLS terminates at Ingress. Compose also serves plain HTTP on
|
||||
`:8006`; optional HTTPS for proxmoxer-style clients is
|
||||
`docker compose --profile tls` on host `:8443`. Details:
|
||||
[Kubernetes / Helm](docs/kubernetes.md).
|
||||
|
||||
- HTTP API and Web UI (Compose): [http://localhost:8006/](http://localhost:8006/)
|
||||
@@ -76,29 +90,34 @@ make install
|
||||
make up
|
||||
make seed PROFILE=small
|
||||
|
||||
curl http://localhost:8006/health/ready
|
||||
curl http://localhost:8006/api2/json/version
|
||||
curl -X POST -d 'username=root@pam&password=secret' \
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
curl -sS http://localhost:8006/api2/json/version
|
||||
curl -sS -X POST -d 'username=root@pam&password=secret' \
|
||||
http://localhost:8006/api2/json/access/ticket
|
||||
```
|
||||
|
||||
- HTTP API and Web UI: [http://localhost:8006/](http://localhost:8006/)
|
||||
- HTTPS gateway (self-signed, development only): `https://localhost:8007`
|
||||
— checked-in `docker/tls/server.key` is a **lab-only** localhost cert; do not
|
||||
reuse it outside local Compose.
|
||||
(real PVE uses **HTTPS** on `:8006`; this lab uses plain HTTP on the same
|
||||
port. Optional TLS for proxmoxer: `docker compose --profile tls` →
|
||||
`https://localhost:8443/`)
|
||||
- Port map and TLS notes: [Ports and TLS](docs/configuration.md#ports-and-tls).
|
||||
- FastAPI schema docs: [http://localhost:8006/docs](http://localhost:8006/docs)
|
||||
|
||||
### Web UI
|
||||
|
||||
Interactive console with light/dark themes, endpoint catalog for PVE 6–9, and
|
||||
runtime contract hot-swap. More detail: [Web UI](docs/web-ui.md).
|
||||
Interactive console with light/dark themes, endpoint catalog for PVE 6–9,
|
||||
runtime contract hot-swap, and a UPID task monitor.
|
||||
|
||||

|
||||

|
||||
|
||||

|
||||
More screens and details: [Web UI](docs/web-ui.md).
|
||||
|
||||
## Documentation
|
||||
|
||||
Documentation is bilingual. Use the **Language / Язык** switcher at the top of
|
||||
each page, or open the Russian root [README.ru.md](README.ru.md). Index:
|
||||
[docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md).
|
||||
|
||||
| Guide | Description |
|
||||
|---|---|
|
||||
| [Getting started](docs/getting-started.md) | First successful lab session |
|
||||
@@ -120,6 +139,8 @@ runtime contract hot-swap. More detail: [Web UI](docs/web-ui.md).
|
||||
| [Compatibility](docs/compatibility.md) | Evidence model and release matrix |
|
||||
|
||||
Runnable cookbooks live under [`examples/`](examples/README.md).
|
||||
Pulumi integration suite (contract surface majors 6–9 + lifecycle, HTML report):
|
||||
[`pulumi-tests/`](pulumi-tests/README.md) (`make pulumi-tests`).
|
||||
|
||||
## proxmoxer (HTTPS gateway)
|
||||
|
||||
@@ -128,7 +149,7 @@ from proxmoxer import ProxmoxAPI
|
||||
|
||||
proxmox = ProxmoxAPI(
|
||||
"localhost",
|
||||
port=8007,
|
||||
port=8006,
|
||||
user="root@pam",
|
||||
password="secret",
|
||||
verify_ssl=False, # local self-signed development certificate only
|
||||
@@ -169,9 +190,20 @@ make release-build # build/tag only, no push
|
||||
make release-up && make release-seed # run the published stack locally
|
||||
```
|
||||
|
||||
## Contributing / security / changelog
|
||||
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) · [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md)
|
||||
- [Testing](docs/testing.md) · [Тесты](docs/ru/testing.md) — suites, Make targets, latest results
|
||||
- [SECURITY.md](SECURITY.md)
|
||||
- [CHANGELOG.md](CHANGELOG.md)
|
||||
|
||||
## What this is not
|
||||
|
||||
- Not a hypervisor: no KVM/LXC execution on bare metal or nested hosts.
|
||||
- Not a drop-in multi-tenant production Proxmox replacement.
|
||||
- Remote IdP / LDAP / live Ceph / live ACME directories are simulated locally;
|
||||
they do not call real external systems.
|
||||
|
||||
## See also
|
||||
|
||||
- [Web UI](docs/web-ui.md) — interactive console, catalogs, DATA panel, and screenshots
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
**Language / Язык:** [English](README.md) | [Русский](README.ru.md)
|
||||
|
||||
# proxmox-api-simulator
|
||||
|
||||
Stateful-асинхронный симулятор API [Proxmox VE](https://www.proxmox.com/) для
|
||||
тестирования API-клиентов и инфраструктурных инструментов без реального
|
||||
гипервизорного кластера.
|
||||
|
||||
> **Только лаборатория / CI.** Учётные данные, signing keys и открытые UI/admin
|
||||
> helpers по умолчанию — намеренные лабораторные значения. **Не** выставляйте
|
||||
> стек в публичный Интернет без замены секретов и своих сетевых ограничений.
|
||||
> См. [SECURITY.md](SECURITY.md) и [Безопасность](docs/ru/security.md).
|
||||
|
||||
Симулятор работает на PostgreSQL, опирается на импортированные официальные
|
||||
контракты API и предоставляет те же поверхности `/api2/json` и `/api2/extjs`,
|
||||
что и Proxmox VE. Семантические обработчики сохраняют мутации; длительные
|
||||
операции возвращают устойчивые UPID, которые выполняют воркеры с арендой задач.
|
||||
|
||||
## Проверенное покрытие API
|
||||
|
||||
Реестр обработчиков и верифицированные ledger поверхности — **100%** для каждого
|
||||
включённого major:
|
||||
|
||||
| Контракт | Declared | Implemented | Verified |
|
||||
|---|---:|---:|---:|
|
||||
| PVE 6.4-15 | 504 | 504 | 504 |
|
||||
| PVE 7.4-16 | 540 | 540 | 540 |
|
||||
| PVE 8.4.5 | 605 | 605 | 605 |
|
||||
| PVE 9.2.3 | 675 | 675 | 675 |
|
||||
|
||||
Переключение активного контракта в runtime — из Web UI (**Apply as runtime**) или
|
||||
`POST /ui/api/contract/apply?major=N` — каждый Apply загружает
|
||||
`evidence/pve-{version}.json`, чтобы observed/verified следовали выбранному
|
||||
major. После импорта нового контракта перегенерируйте ledger: `make evidence`.
|
||||
Живые отчёты: `/admin/compatibility` (также `.md` / `.html`). См.
|
||||
[Совместимость](docs/ru/compatibility.md) и [Версии API](docs/ru/api-versions.md).
|
||||
|
||||
> Это измеримое покрытие контракта и обработчиков лабораторного симулятора —
|
||||
> не утверждение, что каждый краевой случай Proxmox или удалённая интеграция
|
||||
> ведёт себя идентично продакшен-железу.
|
||||
|
||||
## Быстрый старт (опубликованный образ)
|
||||
|
||||
Образ: [`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator)
|
||||
|
||||
### Docker Compose
|
||||
|
||||
Нужен checkout с `docker-compose.release.yml`. Не публикуйте хост `:8006` за
|
||||
пределы доверенной лаборатории без ротации `TICKET_SIGNING_KEY` / пароля БД.
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.release.yml up -d
|
||||
docker compose -f docker-compose.release.yml run --rm --entrypoint python \
|
||||
simulator -m app.simulation.seed_cli
|
||||
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
curl -sS http://localhost:8006/api2/json/version
|
||||
```
|
||||
|
||||
Или: `make release-up && make release-seed PROFILE=small`
|
||||
|
||||
### Helm (Kubernetes + Ingress + Let's Encrypt)
|
||||
|
||||
```bash
|
||||
helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
|
||||
-n proxmox-sim --create-namespace \
|
||||
-f ./helm/proxmox-api-simulator/values-ingress-example.yaml \
|
||||
--set certManager.email=you@example.com \
|
||||
--set ingress.hosts[0].host=pve-sim.example.com \
|
||||
--set ingress.tls[0].hosts[0]=pve-sim.example.com \
|
||||
--set secret.ticketSigningKey="$(openssl rand -hex 32)" \
|
||||
--set postgresql.auth.password="$(openssl rand -hex 16)"
|
||||
```
|
||||
|
||||
Нужны Ingress-контроллер и cert-manager. Service чарта говорит по **HTTP**
|
||||
`:8006`; TLS — на Ingress. Compose тоже отдаёт plain HTTP на `:8006`; опциональный
|
||||
HTTPS для proxmoxer — `docker compose --profile tls` на хосте `:8443`.
|
||||
Подробности: [Kubernetes / Helm](docs/ru/kubernetes.md).
|
||||
|
||||
- HTTP API и Web UI (Compose): [http://localhost:8006/](http://localhost:8006/)
|
||||
- Схема FastAPI: [http://localhost:8006/docs](http://localhost:8006/docs)
|
||||
- Админ по умолчанию после seed: `root@pam` / `secret`
|
||||
|
||||
## Быстрый старт (разработка из репозитория)
|
||||
|
||||
Сборка и запуск development-стека с bind-mount из этого репозитория:
|
||||
|
||||
```bash
|
||||
make install
|
||||
make up
|
||||
make seed PROFILE=small
|
||||
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
curl -sS http://localhost:8006/api2/json/version
|
||||
curl -sS -X POST -d 'username=root@pam&password=secret' \
|
||||
http://localhost:8006/api2/json/access/ticket
|
||||
```
|
||||
|
||||
- HTTP API и Web UI: [http://localhost:8006/](http://localhost:8006/)
|
||||
(реальный PVE — **HTTPS** на `:8006`; лаборатория — plain HTTP на том же
|
||||
порту. Опциональный TLS для proxmoxer: `docker compose --profile tls` →
|
||||
`https://localhost:8443/`)
|
||||
- Карта портов и TLS: [Порты и TLS](docs/ru/configuration.md#порты-и-tls).
|
||||
- Схема FastAPI: [http://localhost:8006/docs](http://localhost:8006/docs)
|
||||
|
||||
### Web UI
|
||||
|
||||
Интерактивная консоль со светлой/тёмной темой, каталогом эндпоинтов PVE 6–9,
|
||||
горячей сменой runtime-контракта и монитором задач UPID.
|
||||
|
||||

|
||||
|
||||
Больше экранов и подробностей: [Web UI](docs/ru/web-ui.md).
|
||||
|
||||
## Документация
|
||||
|
||||
Документация двуязычная. Переключатель **Language / Язык** — в шапке каждой
|
||||
страницы; английский корень — [README.md](README.md). Индекс:
|
||||
[docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md).
|
||||
|
||||
| Руководство | Описание |
|
||||
|---|---|
|
||||
| [Начало работы](docs/ru/getting-started.md) | Первая успешная лабораторная сессия |
|
||||
| [Конфигурация](docs/ru/configuration.md) | Переменные окружения и Compose |
|
||||
| [Аутентификация](docs/ru/authentication.md) | Тикеты, CSRF, API-токены, ACL |
|
||||
| [Версии API](docs/ru/api-versions.md) | Контракты 6–9 и hot-swap |
|
||||
| [Клиенты и примеры](docs/ru/clients.md) | Python, Go, Java, Perl, Ansible, Terraform, Pulumi |
|
||||
| [Профили seed](docs/ru/seed-profiles.md) | Детерминированные фикстуры кластера |
|
||||
| [Поверхность API](docs/ru/api-surface.md) | Маршрутизация, обработчики, fallback |
|
||||
| [Домены](docs/ru/domains/README.md) | QEMU, LXC, storage, HA, SDN, … |
|
||||
| [Web UI](docs/ru/web-ui.md) | Интерактивная консоль и каталоги |
|
||||
| [Эксплуатация](docs/ru/operations.md) | Миграции, reseed, обновления |
|
||||
| [Kubernetes / Helm](docs/ru/kubernetes.md) | Образ Hub + Ingress + Let's Encrypt |
|
||||
| [Безопасность](docs/ru/security.md) | Модель угроз лаборатории и учётные данные |
|
||||
| [Наблюдаемость](docs/ru/observability.md) | Health-эндпоинты и логирование |
|
||||
| [Устранение неполадок](docs/ru/troubleshooting.md) | Типичные сбои |
|
||||
| [FAQ](docs/ru/faq.md) | Краткие ответы |
|
||||
| [Архитектура](docs/ru/architecture.md) | Границы компонентов |
|
||||
| [Совместимость](docs/ru/compatibility.md) | Модель evidence и матрица релиза |
|
||||
|
||||
Индекс гайдов: [`docs/ru/README.md`](docs/ru/README.md).
|
||||
Запускаемые cookbook: [`examples/`](examples/README.ru.md).
|
||||
Интеграционный набор Pulumi (surface majors 6–9 + lifecycle, HTML-отчёт):
|
||||
[`pulumi-tests/`](pulumi-tests/README.ru.md) (`make pulumi-tests`).
|
||||
|
||||
## proxmoxer (HTTPS-шлюз)
|
||||
|
||||
```python
|
||||
from proxmoxer import ProxmoxAPI
|
||||
|
||||
proxmox = ProxmoxAPI(
|
||||
"localhost",
|
||||
port=8006,
|
||||
user="root@pam",
|
||||
password="secret",
|
||||
verify_ssl=False, # только локальный self-signed сертификат разработки
|
||||
)
|
||||
print(proxmox.version.get())
|
||||
print(proxmox.nodes("pve01").qemu.get())
|
||||
```
|
||||
|
||||
Пример API-токена: пользователь `root@pam`, `token_name="automation"`,
|
||||
`token_value="automation-secret"`. Запросам с токеном CSRF не нужен; мутациям
|
||||
по тикету — нужен.
|
||||
|
||||
## Частые цели Make
|
||||
|
||||
```bash
|
||||
make up / make down / make logs / make dev
|
||||
make test # unit + contract (включая verified surface)
|
||||
make test-integration # с PostgreSQL
|
||||
make test-surface # все глаголы × majors 6-9 (0x501 / 0xexception)
|
||||
make test-compatibility # proxmoxer против Compose
|
||||
make evidence # перегенерация evidence/pve-*.json
|
||||
make seed PROFILE=small
|
||||
make db-migrate
|
||||
make shell
|
||||
make ci # ruff + mypy + offline pytest + surface probe
|
||||
make release # сборка + push runtime-образа в Docker Hub
|
||||
make release-up # pull/start docker-compose.release.yml
|
||||
make release-seed PROFILE=small
|
||||
```
|
||||
|
||||
Релиз в Docker Hub (нужен `docker login` владельца Hub; см.
|
||||
[Эксплуатация](docs/ru/operations.md)):
|
||||
|
||||
```bash
|
||||
make release # inecs/proxmox-api-simulator:<версия pyproject> + :latest
|
||||
make release VERSION=0.2.0 # переопределить тег
|
||||
make release-build # только build/tag, без push
|
||||
make release-up && make release-seed # запустить опубликованный стек локально
|
||||
```
|
||||
|
||||
## Участие / безопасность / changelog
|
||||
|
||||
- [CONTRIBUTING.md](CONTRIBUTING.md) · [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md)
|
||||
- [Testing](docs/testing.md) · [Тесты](docs/ru/testing.md) — наборы, Make-цели, последние результаты
|
||||
- [SECURITY.md](SECURITY.md)
|
||||
- [CHANGELOG.md](CHANGELOG.md)
|
||||
|
||||
## Чем это не является
|
||||
|
||||
- Не гипервизор: нет выполнения KVM/LXC на железе или nested-хостах.
|
||||
- Не drop-in замена мультиарендного продакшен-Proxmox.
|
||||
- Удалённые IdP / LDAP / live Ceph / live ACME эндпоинты симулируются локально;
|
||||
к реальным внешним системам они не обращаются.
|
||||
|
||||
## См. также
|
||||
|
||||
- [Web UI](docs/ru/web-ui.md) — интерактивная консоль, каталоги, панель DATA и скриншоты
|
||||
@@ -0,0 +1,52 @@
|
||||
**Language / Язык:** [English](SECURITY.md) | [Русский](docs/ru/security.md)
|
||||
|
||||
# Security policy
|
||||
|
||||
## Supported versions
|
||||
|
||||
| Version | Supported |
|
||||
|---|---|
|
||||
| `0.1.x` (latest) | Yes — security reports accepted |
|
||||
| older / untagged | Best-effort only |
|
||||
|
||||
## Threat model (read this first)
|
||||
|
||||
This repository is a **local / CI laboratory simulator**, not a hardened
|
||||
multi-tenant public Proxmox deployment.
|
||||
|
||||
Default Compose and Helm values intentionally ship convenient lab secrets,
|
||||
seeded passwords, open Web UI helper routes (`/ui/api/*`), and compatibility
|
||||
endpoints (`/admin/compatibility*`). Treat network reachability as the trust
|
||||
boundary.
|
||||
|
||||
**Do not** expose host port `8006` (or a public Ingress) to untrusted networks
|
||||
without replacing every default secret and adding controls you own.
|
||||
|
||||
Full lab notes: [docs/security.md](docs/security.md) ·
|
||||
[docs/ru/security.md](docs/ru/security.md).
|
||||
|
||||
## Reporting a vulnerability
|
||||
|
||||
Please **do not** open a public GitHub issue for sensitive reports.
|
||||
|
||||
Email the maintainer privately (account that owns the GitHub repository /
|
||||
Docker Hub `inecs` namespace), or use GitHub
|
||||
[ privately reported vulnerabilities](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability)
|
||||
if enabled on the repository.
|
||||
|
||||
Include:
|
||||
|
||||
- Affected version / image tag (`inecs/proxmox-api-simulator:…`)
|
||||
- Reproduction steps against a **local** lab (not third-party instances)
|
||||
- Impact assessment (auth bypass, secret leak, RCE, etc.)
|
||||
|
||||
You should receive an acknowledgement within a few business days.
|
||||
|
||||
## Lab secrets that must be rotated outside toy labs
|
||||
|
||||
| Secret | Where |
|
||||
|---|---|
|
||||
| `TICKET_SIGNING_KEY` | Compose / Helm |
|
||||
| PostgreSQL password | Compose / Helm |
|
||||
| Seeded `root@pam` / API tokens | After `seed` |
|
||||
| `docker/tls/server.key` | Checked-in self-signed material — never reuse outside local Compose |
|
||||
@@ -17,7 +17,7 @@ from app.api.errors import ApiError, ContractValidationError
|
||||
from app.api.openapi import contract_openapi_tags
|
||||
from app.config import Settings
|
||||
from app.contracts.examples import schema_example
|
||||
from app.contracts.model import Method, Schema, Snapshot
|
||||
from app.contracts.model import Method, Parameter, Schema, Snapshot
|
||||
from app.db.pool import AsyncpgDatabase
|
||||
from app.security.acl import AclEntry, CapabilityRequirement, authorize, requirement_from_contract
|
||||
from app.security.auth import parse_api_token, verify_csrf, verify_secret, verify_ticket
|
||||
@@ -137,6 +137,64 @@ def register_legacy_handler_routes(
|
||||
return seen
|
||||
|
||||
|
||||
_PATH_PARAM_RE = re.compile(r"\{([^{}]+)\}")
|
||||
|
||||
|
||||
def _unbound_method(path: str, verb: str) -> Method:
|
||||
"""Minimal contract method so handler-only paths can be mounted."""
|
||||
|
||||
parameters = tuple(
|
||||
Parameter(name=name, definition=Schema(type="string", optional=True))
|
||||
for name in _PATH_PARAM_RE.findall(path)
|
||||
)
|
||||
return Method(
|
||||
verb=verb.upper(),
|
||||
name=f"unbound-{verb.lower()}-{path}",
|
||||
description="Handler-backed route not present in the active contract snapshot",
|
||||
parameters=parameters,
|
||||
returns=Schema(),
|
||||
permissions=None,
|
||||
checksum="0" * 64,
|
||||
)
|
||||
|
||||
|
||||
def register_unbound_handler_routes(
|
||||
app: FastAPI,
|
||||
handlers: HandlerRegistry,
|
||||
fallback: FallbackMode = "error",
|
||||
*,
|
||||
existing: set[tuple[str, str, str]] | None = None,
|
||||
) -> set[tuple[str, str, str]]:
|
||||
"""Mount semantic handlers that no cached contract currently declares.
|
||||
|
||||
Keeps DB-backed handlers reachable (no FastAPI 404) when Proxmox schema
|
||||
omits a path that the simulator still implements.
|
||||
"""
|
||||
|
||||
seen = existing if existing is not None else set()
|
||||
for path, verb in sorted(handlers.keys()):
|
||||
for renderer in ("json", "extjs"):
|
||||
route = f"/api2/{renderer}{path}"
|
||||
key = (route, verb, renderer)
|
||||
if key in seen:
|
||||
continue
|
||||
method = _unbound_method(path, verb)
|
||||
seen.add(key)
|
||||
app.add_api_route(
|
||||
route,
|
||||
_endpoint(path, method, renderer, handlers, fallback),
|
||||
methods=[verb],
|
||||
name=f"unbound:{renderer}:{verb}:{path}",
|
||||
tags=cast(list[str | Enum], contract_openapi_tags(path, renderer)),
|
||||
openapi_extra={
|
||||
"x-proxmox-method-checksum": method.checksum,
|
||||
"x-proxmox-implementation": "implemented",
|
||||
"x-proxmox-unbound-handler": True,
|
||||
},
|
||||
)
|
||||
return seen
|
||||
|
||||
|
||||
def _endpoint(
|
||||
semantic_path: str,
|
||||
method: Method,
|
||||
@@ -302,8 +360,14 @@ async def _parse_inputs(request: Request, method: Method) -> dict[str, Any]:
|
||||
continue
|
||||
if name not in supplied:
|
||||
if definition.optional:
|
||||
# Proxmox schema ``default`` is often UI documentation text (e.g.
|
||||
# bwlimit default = "restore limit from datacenter…"), not a value
|
||||
# to materialize. Only inject defaults that coerce to the type.
|
||||
if definition.default is not None:
|
||||
parsed[name] = definition.default
|
||||
try:
|
||||
parsed[name] = _coerce(definition.default, definition)
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
continue
|
||||
errors[name] = "property is missing and it is not optional"
|
||||
continue
|
||||
|
||||
@@ -1,7 +1,15 @@
|
||||
"""Generate example values from Proxmox contract schemas."""
|
||||
"""Generate example values from Proxmox contract schemas.
|
||||
|
||||
Examples follow the official API-viewer dialect: named ``format`` tokens,
|
||||
property-string ``format`` objects (``key=value,key2=value2`` / bare
|
||||
``default_key``), and common path-parameter placeholders.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping
|
||||
from typing import Any
|
||||
|
||||
from app.contracts.model import Schema
|
||||
|
||||
_PATH_PARAM_EXAMPLES: dict[str, object] = {
|
||||
@@ -22,6 +30,53 @@ _PATH_PARAM_EXAMPLES: dict[str, object] = {
|
||||
"key": "cpu",
|
||||
"digest": "00000000",
|
||||
"name": "example",
|
||||
"clustername": "example",
|
||||
}
|
||||
|
||||
_FORMAT_EXAMPLES: dict[str, object] = {
|
||||
"pve-node": "pve01",
|
||||
"pve-node-list": "pve01,pve02",
|
||||
"pve-vmid": 100,
|
||||
"pve-vmid-list": "100,101",
|
||||
"pve-storage-id": "local",
|
||||
"pve-storage-server": "192.168.0.10",
|
||||
"pve-poolid": "testpool",
|
||||
"pve-userid": "root@pam",
|
||||
"pve-userid-list": "root@pam,admin@pve",
|
||||
"pve-realm": "pam",
|
||||
"pve-groupid": "admins",
|
||||
"pve-groupid-list": "admins,ops",
|
||||
"pve-roleid": "Administrator",
|
||||
"pve-roleid-list": "Administrator,PVEAdmin",
|
||||
"pve-tokenid": "automation",
|
||||
"pve-tokenid-list": "automation",
|
||||
"pve-priv-list": "VM.Allocate,VM.Config.Options",
|
||||
"pve-iface": "net0",
|
||||
"pve-iface-list": "net0,net1",
|
||||
"pve-configid": "default",
|
||||
"pve-configid-list": "default",
|
||||
"pve-sdn-vnet-id": "vnet0",
|
||||
"pve-replication-job-id": "100-0",
|
||||
"pve-ha-resource-or-vm-id": "vm:100",
|
||||
"pve-ha-group-node-list": "pve01:1,pve02:1",
|
||||
"pve-day-of-week-list": "mon,tue,wed",
|
||||
"pve-fw-addr-spec": "192.168.0.0/24",
|
||||
"pve-fw-dport-spec": "22",
|
||||
"pve-fw-sport-spec": "1024:65535",
|
||||
"pve-fw-protocol-spec": "tcp",
|
||||
"pve-fw-icmp-type-spec": "echo-request",
|
||||
"address": "192.168.0.1",
|
||||
"ip": "192.168.0.1",
|
||||
"IPorCIDR": "192.168.0.0/24",
|
||||
"IPorCIDRorAlias": "192.168.0.0/24",
|
||||
"dns-name": "example.local",
|
||||
"email-opt": "admin@example.local",
|
||||
"email-list": "admin@example.local",
|
||||
"email-or-username-list": "admin@example.local",
|
||||
"ldap-simple-attr": "uid",
|
||||
"ldap-simple-attr-list": "posixGroup,groupOfNames",
|
||||
"string-alist": "/var/lib/vz",
|
||||
"pve-tfa-secret": "ABCDEFGHIJKLMNOP",
|
||||
}
|
||||
|
||||
|
||||
@@ -31,6 +86,14 @@ def path_param_example(name: str) -> object | None:
|
||||
return _PATH_PARAM_EXAMPLES.get(name)
|
||||
|
||||
|
||||
def wire_param_name(name: str, *, index: int = 0) -> str:
|
||||
"""Map contract ``foo[n]`` names to a concrete wire key (``foo0``)."""
|
||||
|
||||
if "[n]" in name:
|
||||
return name.replace("[n]", str(index))
|
||||
return name
|
||||
|
||||
|
||||
def schema_example(schema: Schema, *, name: str | None = None) -> object:
|
||||
"""Build a representative example value for a contract schema."""
|
||||
|
||||
@@ -47,6 +110,19 @@ def schema_example(schema: Schema, *, name: str | None = None) -> object:
|
||||
hinted = path_param_example(indexed.rstrip("0123456789"))
|
||||
if hinted is not None:
|
||||
return hinted
|
||||
base = indexed.rstrip("0123456789")
|
||||
if base in _PATH_PARAM_EXAMPLES:
|
||||
return _PATH_PARAM_EXAMPLES[base]
|
||||
if isinstance(schema.format, Mapping):
|
||||
return property_string_example(schema.format)
|
||||
if isinstance(schema.format, str):
|
||||
formatted = _named_format_example(schema.format, name=name)
|
||||
if formatted is not None:
|
||||
return formatted
|
||||
if schema.format == "email":
|
||||
return "user@example.com"
|
||||
if schema.format == "uri":
|
||||
return "https://example.com"
|
||||
if schema.type == "array":
|
||||
if schema.items is not None:
|
||||
return [schema_example(schema.items)]
|
||||
@@ -68,9 +144,105 @@ def schema_example(schema: Schema, *, name: str | None = None) -> object:
|
||||
return float(schema.minimum)
|
||||
return 1.0
|
||||
if schema.type == "string" or schema.type is None:
|
||||
if schema.format == "email":
|
||||
return "user@example.com"
|
||||
if schema.format == "uri":
|
||||
return "https://example.com"
|
||||
return "example"
|
||||
return None
|
||||
|
||||
|
||||
def property_string_example(fmt: Mapping[str, Any]) -> str:
|
||||
"""Render a Proxmox property-string example from a ``format`` object.
|
||||
|
||||
Mirrors the API viewer dialect: a ``default_key`` may appear as a bare
|
||||
value; other required keys use ``key=value``; optional keys are omitted
|
||||
from the minimal example (they appear in ``typetext`` as ``[,key=…]``).
|
||||
"""
|
||||
|
||||
bare: str | None = None
|
||||
keyed: list[str] = []
|
||||
for key, raw in sorted(fmt.items()):
|
||||
if not isinstance(raw, Mapping):
|
||||
continue
|
||||
value = _format_key_example(key, raw)
|
||||
rendered = _stringify_property_value(value)
|
||||
if raw.get("default_key"):
|
||||
bare = rendered
|
||||
continue
|
||||
if _is_optional_flag(raw.get("optional")):
|
||||
continue
|
||||
keyed.append(f"{key}={rendered}")
|
||||
|
||||
if bare is not None and not keyed:
|
||||
return bare
|
||||
if bare is not None:
|
||||
return ",".join([bare, *keyed])
|
||||
if keyed:
|
||||
return ",".join(keyed)
|
||||
# All keys optional — include defaults / default_key style extras for a
|
||||
# usable stub that still matches typetext shape.
|
||||
fallback: list[str] = []
|
||||
for key, raw in sorted(fmt.items()):
|
||||
if not isinstance(raw, Mapping):
|
||||
continue
|
||||
value = _format_key_example(key, raw)
|
||||
rendered = _stringify_property_value(value)
|
||||
if raw.get("default_key"):
|
||||
return rendered
|
||||
fallback.append(f"{key}={rendered}")
|
||||
return ",".join(fallback) if fallback else "example"
|
||||
|
||||
|
||||
def _named_format_example(fmt: str, *, name: str | None) -> object | None:
|
||||
if fmt in _FORMAT_EXAMPLES:
|
||||
return _FORMAT_EXAMPLES[fmt]
|
||||
if fmt.endswith("-list") and fmt[: -len("-list")] in _FORMAT_EXAMPLES:
|
||||
base = _FORMAT_EXAMPLES[fmt[: -len("-list")]]
|
||||
return str(base)
|
||||
if name and name in _FORMAT_EXAMPLES:
|
||||
return _FORMAT_EXAMPLES[name]
|
||||
return None
|
||||
|
||||
|
||||
def _format_key_example(key: str, raw: Mapping[str, Any]) -> object:
|
||||
if "default" in raw and raw["default"] is not None:
|
||||
return raw["default"]
|
||||
enum = raw.get("enum")
|
||||
if isinstance(enum, list | tuple) and enum:
|
||||
return enum[0]
|
||||
nested_format = raw.get("format")
|
||||
if isinstance(nested_format, Mapping):
|
||||
return property_string_example(nested_format)
|
||||
if isinstance(nested_format, str):
|
||||
named = _named_format_example(nested_format, name=key)
|
||||
if named is not None:
|
||||
return named
|
||||
if nested_format == "email":
|
||||
return "user@example.com"
|
||||
if nested_format == "uri":
|
||||
return "https://example.com"
|
||||
hinted = path_param_example(key)
|
||||
if hinted is not None:
|
||||
return hinted
|
||||
typ = raw.get("type")
|
||||
if typ == "boolean":
|
||||
return 1
|
||||
if typ == "integer":
|
||||
minimum = raw.get("minimum")
|
||||
return int(minimum) if isinstance(minimum, int | float) else 1
|
||||
if typ == "number":
|
||||
minimum = raw.get("minimum")
|
||||
return float(minimum) if isinstance(minimum, int | float) else 1.0
|
||||
pattern = raw.get("pattern")
|
||||
if isinstance(pattern, str) and "second|minute|hour|day" in pattern:
|
||||
return "1/second"
|
||||
return "example"
|
||||
|
||||
|
||||
def _stringify_property_value(value: object) -> str:
|
||||
if isinstance(value, bool):
|
||||
return "1" if value else "0"
|
||||
return str(value)
|
||||
|
||||
|
||||
def _is_optional_flag(value: object) -> bool:
|
||||
if value is True or value == 1 or value == "1":
|
||||
return True
|
||||
return False
|
||||
|
||||
@@ -14,6 +14,7 @@ from app.api.registry import (
|
||||
HandlerRegistry,
|
||||
register_contract_routes,
|
||||
register_legacy_handler_routes,
|
||||
register_unbound_handler_routes,
|
||||
)
|
||||
from app.compatibility import (
|
||||
CompatibilityDimension,
|
||||
@@ -106,7 +107,7 @@ def apply_runtime_contract(
|
||||
|
||||
clear_contract_routes(app)
|
||||
registered = register_contract_routes(app, snapshot, handlers, fallback)
|
||||
register_legacy_handler_routes(
|
||||
registered = register_legacy_handler_routes(
|
||||
app,
|
||||
handlers,
|
||||
store_root,
|
||||
@@ -114,6 +115,7 @@ def apply_runtime_contract(
|
||||
primary_version=snapshot.source_version,
|
||||
existing=registered,
|
||||
)
|
||||
register_unbound_handler_routes(app, handlers, fallback, existing=registered)
|
||||
report = build_compatibility_for_snapshot(
|
||||
snapshot,
|
||||
handlers,
|
||||
|
||||
@@ -125,7 +125,7 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
return [
|
||||
{
|
||||
"userid": str(row["name"]),
|
||||
"enable": 1 if row["enabled"] else 0,
|
||||
"enable": bool(row["enabled"]),
|
||||
"realm-type": str(row["realm_kind"]),
|
||||
}
|
||||
for row in rows
|
||||
@@ -168,10 +168,25 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "user does not exist")
|
||||
groups = await database(request).pool.fetch(
|
||||
"""SELECT g.group_id FROM identity_group_members m
|
||||
JOIN identity_groups g ON g.id = m.group_id
|
||||
JOIN principals p ON p.id = m.principal_id
|
||||
WHERE p.name=$1 ORDER BY g.group_id""",
|
||||
userid,
|
||||
)
|
||||
tokens = await database(request).pool.fetch(
|
||||
"""SELECT t.token_id FROM api_tokens t
|
||||
JOIN principals p ON p.id = t.principal_id
|
||||
WHERE p.name=$1 ORDER BY t.token_id""",
|
||||
userid,
|
||||
)
|
||||
return {
|
||||
"userid": str(row["name"]),
|
||||
"enable": 1 if row["enabled"] else 0,
|
||||
"enable": bool(row["enabled"]),
|
||||
"realm-type": str(row["realm_kind"]),
|
||||
"groups": [str(item["group_id"]) for item in groups],
|
||||
"tokens": [{"tokenid": str(item["token_id"]), "privsep": 1} for item in tokens],
|
||||
}
|
||||
|
||||
async def user_update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
@@ -397,10 +412,18 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
path = str(payload["path"])
|
||||
roleid = str(payload["roles"])
|
||||
propagate = bool(int(payload.get("propagate", 1)))
|
||||
delete = bool(int(payload.get("delete", 0)))
|
||||
users = [item.strip() for item in str(payload.get("users", "")).split(",") if item.strip()]
|
||||
groups = [
|
||||
item.strip() for item in str(payload.get("groups", "")).split(",") if item.strip()
|
||||
]
|
||||
tokens = [
|
||||
item.strip() for item in str(payload.get("tokens", "")).split(",") if item.strip()
|
||||
]
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO roles(name) VALUES($1) ON CONFLICT DO NOTHING""",
|
||||
roleid,
|
||||
)
|
||||
for userid in users:
|
||||
principal_id = await database(request).pool.fetchval(
|
||||
"SELECT id FROM principals WHERE name=$1",
|
||||
@@ -408,10 +431,15 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if principal_id is None:
|
||||
raise ApiError(404, f"user {userid} does not exist")
|
||||
if delete:
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO roles(name) VALUES($1) ON CONFLICT DO NOTHING""",
|
||||
"""DELETE FROM acl_entries
|
||||
WHERE principal_id=$1 AND role_name=$2 AND path=$3""",
|
||||
principal_id,
|
||||
roleid,
|
||||
path,
|
||||
)
|
||||
continue
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO acl_entries(principal_id, role_name, path, propagate)
|
||||
VALUES($1, $2, $3, $4)
|
||||
@@ -429,10 +457,15 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if group_id is None:
|
||||
raise ApiError(404, f"group {groupid} does not exist")
|
||||
if delete:
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO roles(name) VALUES($1) ON CONFLICT DO NOTHING""",
|
||||
"""DELETE FROM group_acl_entries
|
||||
WHERE group_id=$1 AND role_name=$2 AND path=$3""",
|
||||
group_id,
|
||||
roleid,
|
||||
path,
|
||||
)
|
||||
continue
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO group_acl_entries(group_id, role_name, path, propagate)
|
||||
VALUES($1, $2, $3, $4)
|
||||
@@ -443,6 +476,46 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
path,
|
||||
propagate,
|
||||
)
|
||||
for token_ref in tokens:
|
||||
# Wire form: userid!tokenid
|
||||
userid, _, tokenid = token_ref.partition("!")
|
||||
if not userid or not tokenid:
|
||||
raise ApiError(400, f"invalid token ACL subject '{token_ref}'")
|
||||
token_row = await database(request).pool.fetchrow(
|
||||
"""SELECT t.id FROM api_tokens t
|
||||
JOIN principals p ON p.id = t.principal_id
|
||||
WHERE p.name=$1 AND t.token_id=$2""",
|
||||
userid,
|
||||
tokenid,
|
||||
)
|
||||
if token_row is None:
|
||||
raise ApiError(404, f"token {token_ref} does not exist")
|
||||
# Persist as principal ACL under the owning user with token scope in path metadata.
|
||||
# Keep durable by tagging path; GET ACL still lists principal entries.
|
||||
scoped_path = f"{path}#token:{tokenid}" if path != "/" else f"/#token:{tokenid}"
|
||||
principal_id = await database(request).pool.fetchval(
|
||||
"SELECT id FROM principals WHERE name=$1",
|
||||
userid,
|
||||
)
|
||||
if delete:
|
||||
await database(request).pool.execute(
|
||||
"""DELETE FROM acl_entries
|
||||
WHERE principal_id=$1 AND role_name=$2 AND path=$3""",
|
||||
principal_id,
|
||||
roleid,
|
||||
scoped_path,
|
||||
)
|
||||
else:
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO acl_entries(principal_id, role_name, path, propagate)
|
||||
VALUES($1, $2, $3, $4)
|
||||
ON CONFLICT (principal_id, role_name, path) DO UPDATE
|
||||
SET propagate=EXCLUDED.propagate""",
|
||||
principal_id,
|
||||
roleid,
|
||||
scoped_path,
|
||||
propagate,
|
||||
)
|
||||
|
||||
async def token_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
userid = str(values(inputs)["userid"])
|
||||
@@ -562,10 +635,8 @@ def register_access_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "role does not exist")
|
||||
return {
|
||||
"roleid": str(row["name"]),
|
||||
"privs": ",".join(str(item) for item in row["privileges"]),
|
||||
}
|
||||
# Contract returns privilege→boolean map (additionalProperties: 0 on named privs).
|
||||
return {str(priv): 1 for priv in row["privileges"]}
|
||||
|
||||
async def role_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from fastapi import Request
|
||||
@@ -10,41 +12,34 @@ from app.api.errors import ApiError
|
||||
from app.api.registry import HandlerRegistry
|
||||
from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs, values
|
||||
|
||||
_DEFAULT_DIRECTORIES = [
|
||||
{
|
||||
"name": "Let's Encrypt V2",
|
||||
"url": "https://acme-v02.api.letsencrypt.org/directory",
|
||||
},
|
||||
{
|
||||
"name": "Let's Encrypt V2 Staging",
|
||||
"url": "https://acme-staging-v02.api.letsencrypt.org/directory",
|
||||
},
|
||||
]
|
||||
|
||||
_CHALLENGE_SCHEMA = [
|
||||
{
|
||||
"id": "dns",
|
||||
"name": "DNS plugin",
|
||||
"type": "dns",
|
||||
"fields": [{"name": "api", "type": "string"}],
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def _acme(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
current = metadata.setdefault(
|
||||
"acme",
|
||||
{"accounts": {}, "plugins": {}, "meta": {}},
|
||||
)
|
||||
current = metadata.get("acme")
|
||||
if not isinstance(current, dict):
|
||||
current = {"accounts": {}, "plugins": {}, "meta": {}}
|
||||
current = {}
|
||||
metadata["acme"] = current
|
||||
current.setdefault("accounts", {})
|
||||
current.setdefault("plugins", {})
|
||||
current.setdefault("meta", {})
|
||||
if not isinstance(current.get("accounts"), dict):
|
||||
current["accounts"] = {}
|
||||
if not isinstance(current.get("plugins"), dict):
|
||||
current["plugins"] = {}
|
||||
if not isinstance(current.get("meta"), dict):
|
||||
current["meta"] = {}
|
||||
if not isinstance(current.get("directories"), list):
|
||||
current["directories"] = []
|
||||
if not isinstance(current.get("challenge_schema"), list):
|
||||
current["challenge_schema"] = []
|
||||
return current
|
||||
|
||||
|
||||
def _default_directory(acme: dict[str, Any]) -> str:
|
||||
directories = acme.get("directories")
|
||||
if isinstance(directories, list):
|
||||
for item in directories:
|
||||
if isinstance(item, dict) and item.get("url"):
|
||||
return str(item["url"])
|
||||
return "https://acme-v02.api.letsencrypt.org/directory"
|
||||
|
||||
|
||||
def register_acme_handlers(registry: HandlerRegistry) -> None:
|
||||
async def index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs(
|
||||
@@ -68,24 +63,26 @@ def register_acme_handlers(registry: HandlerRegistry) -> None:
|
||||
for name, item in sorted(accounts.items())
|
||||
]
|
||||
|
||||
async def account_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def account_create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
name = str(payload.get("name") or "default")
|
||||
metadata = await cluster_metadata(request)
|
||||
accounts = _acme(metadata)["accounts"]
|
||||
acme = _acme(metadata)
|
||||
accounts = acme["accounts"]
|
||||
if name in accounts:
|
||||
raise ApiError(400, f"ACME account '{name}' already exists")
|
||||
directory = str(payload.get("directory") or _default_directory(acme))
|
||||
accounts[name] = {
|
||||
"name": name,
|
||||
"contact": payload.get("contact"),
|
||||
"directory": payload.get("directory") or _DEFAULT_DIRECTORIES[0]["url"],
|
||||
"directory": directory,
|
||||
"tos_url": payload.get("tos_url"),
|
||||
"eab-kid": payload.get("eab-kid"),
|
||||
# eab-hmac-key stored but never returned
|
||||
"eab-hmac-key": payload.get("eab-hmac-key"),
|
||||
"location": f"https://acme.example.local/acct/{name}",
|
||||
}
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return name
|
||||
|
||||
async def account_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
name = str(values(inputs)["name"])
|
||||
@@ -94,8 +91,10 @@ def register_acme_handlers(registry: HandlerRegistry) -> None:
|
||||
if not isinstance(account, dict):
|
||||
raise ApiError(404, "ACME account does not exist")
|
||||
return {
|
||||
"account": {
|
||||
"name": name,
|
||||
"contact": account.get("contact"),
|
||||
},
|
||||
"directory": account.get("directory"),
|
||||
"tos": account.get("tos_url"),
|
||||
"location": account.get("location"),
|
||||
@@ -129,7 +128,13 @@ def register_acme_handlers(registry: HandlerRegistry) -> None:
|
||||
for plugin_id, item in sorted(plugins.items()):
|
||||
if plugin_type and item.get("type") != plugin_type:
|
||||
continue
|
||||
result.append({"plugin": plugin_id, **{k: v for k, v in item.items() if k != "data"}})
|
||||
entry = {"plugin": plugin_id, **{k: v for k, v in item.items() if k != "data"}}
|
||||
if "digest" not in entry:
|
||||
material = {k: v for k, v in entry.items() if k != "digest"}
|
||||
entry["digest"] = hashlib.sha1( # noqa: S324 - Proxmox config digests use SHA-1
|
||||
json.dumps(material, sort_keys=True, separators=(",", ":")).encode()
|
||||
).hexdigest()
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
async def plugins_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
@@ -181,29 +186,33 @@ def register_acme_handlers(registry: HandlerRegistry) -> None:
|
||||
del plugins[plugin_id]
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def directories(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return list(_DEFAULT_DIRECTORIES)
|
||||
async def directories(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
directories = _acme(metadata).get("directories")
|
||||
if isinstance(directories, list):
|
||||
return [dict(item) for item in directories if isinstance(item, dict)]
|
||||
return []
|
||||
|
||||
async def challenge_schema(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return list(_CHALLENGE_SCHEMA)
|
||||
async def challenge_schema(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
schema = _acme(metadata).get("challenge_schema")
|
||||
if isinstance(schema, list):
|
||||
return [dict(item) for item in schema if isinstance(item, dict)]
|
||||
return []
|
||||
|
||||
async def meta(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
directory = str(values(inputs).get("directory") or _DEFAULT_DIRECTORIES[0]["url"])
|
||||
metadata = await cluster_metadata(request)
|
||||
meta_store = _acme(metadata).setdefault("meta", {})
|
||||
payload = meta_store.setdefault(
|
||||
directory,
|
||||
{
|
||||
"termsOfService": f"{directory.rstrip('/')}/tos",
|
||||
"caaIdentities": ["letsencrypt.org"],
|
||||
},
|
||||
)
|
||||
await save_cluster_metadata(request, metadata)
|
||||
acme = _acme(metadata)
|
||||
directory = str(values(inputs).get("directory") or _default_directory(acme))
|
||||
meta_raw = acme.get("meta")
|
||||
meta_store: dict[str, Any] = dict(meta_raw) if isinstance(meta_raw, dict) else {}
|
||||
payload = meta_store.get(directory)
|
||||
if not isinstance(payload, dict):
|
||||
return {}
|
||||
return dict(payload)
|
||||
|
||||
async def tos(request: Request, inputs: dict[str, Any]) -> str:
|
||||
directory = str(values(inputs).get("directory") or _DEFAULT_DIRECTORIES[0]["url"])
|
||||
result = await meta(request, {"values": {"directory": directory}, "provided": frozenset()})
|
||||
result = await meta(request, inputs)
|
||||
return str(result.get("termsOfService") or "")
|
||||
|
||||
registry.register("/cluster/acme", "GET", index)
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
"""Cluster backup and vzdump handlers."""
|
||||
"""Cluster backup jobs and node vzdump handlers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import secrets
|
||||
from typing import Any
|
||||
|
||||
from fastapi import Request
|
||||
@@ -10,170 +10,224 @@ from fastapi import Request
|
||||
from app.api.errors import ApiError
|
||||
from app.api.registry import HandlerRegistry
|
||||
from app.db.primitives import ConflictError
|
||||
from app.handlers.common import database, require_node, state, values
|
||||
from app.handlers.common import (
|
||||
cluster_metadata,
|
||||
database,
|
||||
require_node,
|
||||
save_cluster_metadata,
|
||||
state,
|
||||
subdirs,
|
||||
values,
|
||||
)
|
||||
from app.tasks.repository import TaskRepository
|
||||
from app.tasks.upid import Upid
|
||||
|
||||
|
||||
def _backup_jobs(metadata: dict[str, Any]) -> dict[str, dict[str, Any]]:
|
||||
jobs = metadata.get("backup_jobs")
|
||||
if not isinstance(jobs, dict):
|
||||
return {}
|
||||
return {
|
||||
str(job_id): dict(payload) for job_id, payload in jobs.items() if isinstance(payload, dict)
|
||||
}
|
||||
|
||||
|
||||
def _job_view(job_id: str, payload: dict[str, Any]) -> dict[str, Any]:
|
||||
view = dict(payload)
|
||||
view["id"] = job_id
|
||||
enabled = view.get("enabled", 1)
|
||||
view["enabled"] = bool(int(enabled)) if not isinstance(enabled, bool) else enabled
|
||||
return view
|
||||
|
||||
|
||||
def register_backup_handlers(registry: HandlerRegistry) -> None:
|
||||
async def backup_list(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
rows = await database(_request).pool.fetch(
|
||||
"""SELECT b.id, b.volume_id, b.size_bytes, b.metadata, b.created_at,
|
||||
r.external_id AS vmid, n.name AS node, s.storage_id
|
||||
FROM backups b
|
||||
LEFT JOIN resources r ON r.id = b.resource_id
|
||||
LEFT JOIN nodes n ON n.id = r.node_id
|
||||
JOIN storages s ON s.resource_id = b.storage_resource_id
|
||||
ORDER BY b.created_at DESC LIMIT 2000"""
|
||||
async def backup_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
return [_job_view(job_id, payload) for job_id, payload in sorted(jobs.items())]
|
||||
|
||||
async def backup_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
job_id = str(values(inputs)["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
payload = jobs.get(job_id)
|
||||
if payload is None:
|
||||
raise ApiError(404, "backup job does not exist")
|
||||
return _job_view(job_id, payload)
|
||||
|
||||
async def backup_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
job_id = str(payload.get("id") or f"backup-{secrets.token_hex(3)}")
|
||||
if job_id in jobs:
|
||||
raise ApiError(400, f"backup job '{job_id}' already exists")
|
||||
entry = {
|
||||
key: value
|
||||
for key, value in payload.items()
|
||||
if key not in {"delete", "digest", "node", "target"}
|
||||
}
|
||||
entry["id"] = job_id
|
||||
entry.setdefault("enabled", 1)
|
||||
entry.setdefault("storage", payload.get("storage") or "local")
|
||||
jobs[job_id] = entry
|
||||
metadata["backup_jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def backup_update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
job_id = str(values(inputs)["id"])
|
||||
payload = values(inputs)
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
current = jobs.get(job_id)
|
||||
if current is None:
|
||||
raise ApiError(404, "backup job does not exist")
|
||||
updated = dict(current)
|
||||
for key, value in payload.items():
|
||||
if key in {"id", "delete", "digest"}:
|
||||
continue
|
||||
updated[key] = value
|
||||
updated["id"] = job_id
|
||||
jobs[job_id] = updated
|
||||
metadata["backup_jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def backup_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
job_id = str(values(inputs)["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
if job_id not in jobs:
|
||||
raise ApiError(404, "backup job does not exist")
|
||||
del jobs[job_id]
|
||||
metadata["backup_jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def backup_info(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("not-backed-up")
|
||||
|
||||
async def backupinfo_stub(_request: Request, _inputs: dict[str, Any]) -> str:
|
||||
# PVE 6.x `/cluster/backupinfo` returns a stub string; 7+ uses `/backup-info` index.
|
||||
return "Please use the 'not-backed-up' API"
|
||||
|
||||
async def backup_not_backed_up(
|
||||
request: Request, _inputs: dict[str, Any]
|
||||
) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
covered: set[str] = set()
|
||||
for payload in jobs.values():
|
||||
if int(payload.get("all") or 0):
|
||||
covered.add("*")
|
||||
for item in str(payload.get("vmid") or "").split(","):
|
||||
item = item.strip()
|
||||
if item:
|
||||
covered.add(item)
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT r.external_id, r.kind, r.state FROM resources r
|
||||
WHERE r.kind IN ('qemu', 'lxc')
|
||||
ORDER BY r.external_id::integer"""
|
||||
)
|
||||
result: list[dict[str, Any]] = []
|
||||
if "*" in covered:
|
||||
return result
|
||||
for row in rows:
|
||||
metadata = state(row["metadata"])
|
||||
vmid = str(row["external_id"])
|
||||
if vmid in covered:
|
||||
continue
|
||||
guest_state = state(row["state"])
|
||||
result.append(
|
||||
{
|
||||
"id": str(row["id"]),
|
||||
"volid": str(row["volume_id"]),
|
||||
"size": int(row["size_bytes"]),
|
||||
"vmid": int(row["vmid"]) if row["vmid"] is not None else None,
|
||||
"node": str(row["node"]) if row["node"] is not None else None,
|
||||
"storage": str(row["storage_id"]),
|
||||
"starttime": int(row["created_at"].timestamp()),
|
||||
"mode": metadata.get("mode", "snapshot"),
|
||||
"type": metadata.get("type", "vzdump"),
|
||||
"vmid": int(vmid),
|
||||
"type": "qemu" if row["kind"] == "qemu" else "lxc",
|
||||
"name": str(guest_state.get("name") or f"{row['kind']}-{vmid}"),
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
async def backup_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
backup_id = str(values(inputs)["id"])
|
||||
row = await database(request).pool.fetchrow(
|
||||
"""SELECT b.id, b.volume_id, b.size_bytes, b.metadata, b.created_at,
|
||||
r.external_id AS vmid, n.name AS node, s.storage_id
|
||||
FROM backups b
|
||||
LEFT JOIN resources r ON r.id = b.resource_id
|
||||
LEFT JOIN nodes n ON n.id = r.node_id
|
||||
JOIN storages s ON s.resource_id = b.storage_resource_id
|
||||
WHERE b.id::text = $1 OR b.volume_id = $1""",
|
||||
backup_id,
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "backup does not exist")
|
||||
metadata = state(row["metadata"])
|
||||
return {
|
||||
"id": str(row["id"]),
|
||||
"volid": str(row["volume_id"]),
|
||||
"size": int(row["size_bytes"]),
|
||||
"vmid": int(row["vmid"]) if row["vmid"] is not None else None,
|
||||
"node": str(row["node"]) if row["node"] is not None else None,
|
||||
"storage": str(row["storage_id"]),
|
||||
"starttime": int(row["created_at"].timestamp()),
|
||||
"notes": metadata.get("notes-template"),
|
||||
**metadata,
|
||||
}
|
||||
|
||||
async def backup_update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
backup_id = str(values(inputs)["id"])
|
||||
row = await database(request).pool.fetchrow(
|
||||
"SELECT id, metadata FROM backups WHERE id::text = $1",
|
||||
backup_id,
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "backup does not exist")
|
||||
metadata = state(row["metadata"])
|
||||
payload = values(inputs)
|
||||
if "notes" in payload:
|
||||
metadata["notes-template"] = payload["notes"]
|
||||
await database(request).pool.execute(
|
||||
"UPDATE backups SET metadata=$2::jsonb WHERE id=$1",
|
||||
row["id"],
|
||||
json.dumps(metadata, sort_keys=True),
|
||||
)
|
||||
|
||||
async def backup_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
backup_id = str(values(inputs)["id"])
|
||||
status = await database(request).pool.execute(
|
||||
"DELETE FROM backups WHERE id::text = $1",
|
||||
backup_id,
|
||||
)
|
||||
if status != "DELETE 1":
|
||||
raise ApiError(404, "backup does not exist")
|
||||
|
||||
async def backup_create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
node = str(payload.get("node") or payload.get("target") or "pve01")
|
||||
await require_node(request, node)
|
||||
vmid = payload.get("vmid")
|
||||
return await _schedule_vzdump(
|
||||
request,
|
||||
node=node,
|
||||
vmids=[str(vmid)] if vmid is not None else None,
|
||||
payload=payload,
|
||||
)
|
||||
|
||||
async def backup_info(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
rows = await database(_request).pool.fetch(
|
||||
"""SELECT r.external_id AS vmid, n.name AS node, max(b.created_at) AS last_backup
|
||||
async def backup_included_volumes(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
job_id = str(values(inputs)["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _backup_jobs(metadata)
|
||||
job = jobs.get(job_id)
|
||||
if job is None:
|
||||
raise ApiError(404, "backup job does not exist")
|
||||
vmids = [item.strip() for item in str(job.get("vmid") or "").split(",") if item.strip()]
|
||||
guest_select = """SELECT r.external_id, r.kind, r.state,
|
||||
v.config AS qemu_config, c.config AS lxc_config
|
||||
FROM resources r
|
||||
JOIN nodes n ON n.id = r.node_id
|
||||
LEFT JOIN backups b ON b.resource_id = r.id
|
||||
WHERE r.kind = 'qemu'
|
||||
GROUP BY r.external_id, n.name
|
||||
ORDER BY r.external_id::integer
|
||||
LIMIT 5000"""
|
||||
)
|
||||
return [
|
||||
{
|
||||
"vmid": int(row["vmid"]),
|
||||
"node": str(row["node"]),
|
||||
"lastbackup": int(row["last_backup"].timestamp()) if row["last_backup"] else 0,
|
||||
"protected": 0,
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
|
||||
async def backup_not_backed_up(_request: Request, _inputs: dict[str, Any]) -> list[int]:
|
||||
rows = await database(_request).pool.fetch(
|
||||
"""SELECT r.external_id::integer AS vmid
|
||||
FROM resources r
|
||||
LEFT JOIN backups b ON b.resource_id = r.id
|
||||
WHERE r.kind = 'qemu' AND b.id IS NULL
|
||||
LEFT JOIN virtual_machines v ON v.resource_id = r.id
|
||||
LEFT JOIN containers c ON c.resource_id = r.id"""
|
||||
if int(job.get("all") or 0):
|
||||
rows = await database(request).pool.fetch(
|
||||
f"""{guest_select}
|
||||
WHERE r.kind IN ('qemu', 'lxc')
|
||||
ORDER BY r.external_id::integer"""
|
||||
)
|
||||
return [int(row["vmid"]) for row in rows]
|
||||
|
||||
async def backup_included_volumes(request: Request, inputs: dict[str, Any]) -> list[str]:
|
||||
backup_id = str(values(inputs)["id"])
|
||||
row = await database(request).pool.fetchrow(
|
||||
"""SELECT b.volume_id, r.external_id AS vmid
|
||||
FROM backups b LEFT JOIN resources r ON r.id = b.resource_id
|
||||
WHERE b.id::text = $1""",
|
||||
backup_id,
|
||||
else:
|
||||
rows = await database(request).pool.fetch(
|
||||
f"""{guest_select}
|
||||
WHERE r.kind IN ('qemu', 'lxc') AND r.external_id = ANY($1::text[])
|
||||
ORDER BY r.external_id::integer""",
|
||||
vmids,
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "backup does not exist")
|
||||
vmid = row["vmid"]
|
||||
return [f"qemu/{vmid}"] if vmid is not None else [str(row["volume_id"])]
|
||||
|
||||
async def vzdump_defaults(_request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
await require_node(_request, str(values(inputs)["node"]))
|
||||
return {
|
||||
"all": 0,
|
||||
"bwlimit": 0,
|
||||
"compress": "zstd",
|
||||
"dumpdir": "backup",
|
||||
"mode": "snapshot",
|
||||
"remove": 0,
|
||||
"storage": "nfs-backup",
|
||||
"mailto": "",
|
||||
"notes-template": "{{guestname}}",
|
||||
children: list[dict[str, Any]] = []
|
||||
for row in rows:
|
||||
guest_state = state(row["state"])
|
||||
config_raw = row["qemu_config"] if row["kind"] == "qemu" else row["lxc_config"]
|
||||
config = state(config_raw) if config_raw is not None else {}
|
||||
volumes: list[dict[str, Any]] = []
|
||||
for key, value in sorted(config.items()):
|
||||
if not (
|
||||
key.startswith("scsi")
|
||||
or key.startswith("virtio")
|
||||
or key.startswith("sata")
|
||||
or key.startswith("ide")
|
||||
or key in {"rootfs", "mp0", "mp1", "mp2", "mp3"}
|
||||
):
|
||||
continue
|
||||
volumes.append(
|
||||
{
|
||||
"id": key,
|
||||
"name": str(value).split(",", 1)[0],
|
||||
"included": True,
|
||||
"reason": "included by default",
|
||||
}
|
||||
)
|
||||
children.append(
|
||||
{
|
||||
"id": int(row["external_id"]),
|
||||
"name": str(guest_state.get("name") or f"{row['kind']}-{row['external_id']}"),
|
||||
"type": "qemu" if row["kind"] == "qemu" else "lxc",
|
||||
"children": volumes,
|
||||
}
|
||||
)
|
||||
return {"children": children}
|
||||
|
||||
async def vzdump_defaults(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
from app.handlers.nodes import load_node_ops
|
||||
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
vzdump = ops.get("vzdump")
|
||||
defaults = vzdump.get("defaults") if isinstance(vzdump, dict) else None
|
||||
return dict(defaults) if isinstance(defaults, dict) else {}
|
||||
|
||||
async def vzdump_extractconfig(request: Request, inputs: dict[str, Any]) -> str:
|
||||
from app.handlers.nodes import load_node_ops
|
||||
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
volid = str(payload.get("volume") or payload.get("volid") or "")
|
||||
if not volid:
|
||||
raise ApiError(400, "volume parameter required")
|
||||
return f"# simulated vzdump config extracted from {volid}\name: demo\nmemory: 2048\n"
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
vzdump = ops.get("vzdump")
|
||||
configs = vzdump.get("extractconfig") if isinstance(vzdump, dict) else None
|
||||
if not isinstance(configs, dict):
|
||||
return ""
|
||||
return str(configs.get(volid) or "")
|
||||
|
||||
async def vzdump_create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
@@ -188,6 +242,7 @@ def register_backup_handlers(registry: HandlerRegistry) -> None:
|
||||
registry.register("/cluster/backup", "GET", backup_list)
|
||||
registry.register("/cluster/backup", "POST", backup_create)
|
||||
registry.register("/cluster/backup-info", "GET", backup_info)
|
||||
registry.register("/cluster/backupinfo", "GET", backupinfo_stub)
|
||||
registry.register("/cluster/backup-info/not-backed-up", "GET", backup_not_backed_up)
|
||||
registry.register("/cluster/backup/{id}", "GET", backup_get)
|
||||
registry.register("/cluster/backup/{id}", "PUT", backup_update)
|
||||
@@ -224,11 +279,11 @@ async def _schedule_vzdump(
|
||||
payload={
|
||||
"node": node,
|
||||
"vmids": vmids,
|
||||
"storage": str(payload.get("storage") or "nfs-backup"),
|
||||
"storage": str(payload.get("storage") or "local"),
|
||||
"mode": str(payload.get("mode") or "snapshot"),
|
||||
"compress": str(payload.get("compress") or "zstd"),
|
||||
},
|
||||
resource_key=f"backup:{node}",
|
||||
resource_key=f"backup:{node}:{secrets.token_hex(4)}",
|
||||
idempotency_key=request.headers.get("Idempotency-Key"),
|
||||
)
|
||||
except ConflictError as error:
|
||||
|
||||
@@ -4,7 +4,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
import secrets
|
||||
from typing import Any, cast
|
||||
from typing import Any
|
||||
|
||||
from fastapi import Request
|
||||
|
||||
@@ -21,37 +21,6 @@ from app.handlers.common import (
|
||||
)
|
||||
from app.simulation.seed import CLUSTER_ID
|
||||
|
||||
DEFAULT_CLUSTER_CEPH = {
|
||||
"initialized": True,
|
||||
"config": {
|
||||
"network": "10.10.10.0/24",
|
||||
"cluster-network": "10.10.10.0/24",
|
||||
"size": 3,
|
||||
"min_size": 2,
|
||||
"pg_bits": 7,
|
||||
},
|
||||
"cfg_db": [
|
||||
{"section": "global", "name": "auth_client_required", "value": "cephx"},
|
||||
{"section": "global", "name": "fsid", "value": "pve-simulator-fsid"},
|
||||
],
|
||||
"cfg_raw": "[global]\nfsid = pve-simulator-fsid\nauth_client_required = cephx\n",
|
||||
"cfg_values": {},
|
||||
"pools": {
|
||||
"rbd": {
|
||||
"pool": "rbd",
|
||||
"size": 3,
|
||||
"min_size": 2,
|
||||
"pg_num": 128,
|
||||
"application": "rbd",
|
||||
"crush_rule": "replicated_rule",
|
||||
}
|
||||
},
|
||||
"fs": {},
|
||||
"rules": [{"name": "replicated_rule", "id": 0}],
|
||||
"crush": "device 0 osd.0 class hdd\n",
|
||||
"running": True,
|
||||
}
|
||||
|
||||
|
||||
async def _load_cluster_ceph(request: Request) -> dict[str, Any]:
|
||||
row = await database(request).pool.fetchrow(
|
||||
@@ -60,14 +29,9 @@ async def _load_cluster_ceph(request: Request) -> dict[str, Any]:
|
||||
)
|
||||
metadata = state(row["metadata"]) if row is not None else {}
|
||||
ceph = metadata.get("ceph")
|
||||
if not isinstance(ceph, dict) or not ceph:
|
||||
return dict(DEFAULT_CLUSTER_CEPH)
|
||||
merged = dict(DEFAULT_CLUSTER_CEPH)
|
||||
merged.update(ceph)
|
||||
for key in ("config", "pools", "fs", "cfg_values"):
|
||||
if not isinstance(merged.get(key), dict):
|
||||
merged[key] = dict(cast(dict[str, Any], DEFAULT_CLUSTER_CEPH[key]))
|
||||
return merged
|
||||
if not isinstance(ceph, dict):
|
||||
return {}
|
||||
return dict(ceph)
|
||||
|
||||
|
||||
async def _save_cluster_ceph(request: Request, ceph: dict[str, Any]) -> None:
|
||||
@@ -82,34 +46,27 @@ async def _save_cluster_ceph(request: Request, ceph: dict[str, Any]) -> None:
|
||||
|
||||
async def _load_node_ceph(request: Request, node: str) -> dict[str, Any]:
|
||||
metadata = await node_metadata(request, node)
|
||||
ops = metadata.setdefault("ops", {})
|
||||
ceph = ops.setdefault(
|
||||
"ceph",
|
||||
{
|
||||
"mds": {},
|
||||
"mgr": {},
|
||||
"mon": {f"{node}": {"name": node, "addr": f"{node}.local:6789", "rank": 0}},
|
||||
"log": [{"t": 1_700_000_000, "n": 0, "line": "ceph simulator ready"}],
|
||||
},
|
||||
)
|
||||
ops = metadata.get("ops")
|
||||
if not isinstance(ops, dict):
|
||||
return {"mds": {}, "mgr": {}, "mon": {}, "log": []}
|
||||
ceph = ops.get("ceph")
|
||||
if not isinstance(ceph, dict):
|
||||
ceph = {
|
||||
"mds": {},
|
||||
"mgr": {},
|
||||
"mon": {},
|
||||
"log": [],
|
||||
return {"mds": {}, "mgr": {}, "mon": {}, "log": []}
|
||||
return {
|
||||
"mds": ceph.get("mds") if isinstance(ceph.get("mds"), dict) else {},
|
||||
"mgr": ceph.get("mgr") if isinstance(ceph.get("mgr"), dict) else {},
|
||||
"mon": ceph.get("mon") if isinstance(ceph.get("mon"), dict) else {},
|
||||
"log": list(ceph.get("log") or []) if isinstance(ceph.get("log"), list) else [],
|
||||
**{key: value for key, value in ceph.items() if key not in {"mds", "mgr", "mon", "log"}},
|
||||
}
|
||||
ops["ceph"] = ceph
|
||||
ceph.setdefault("mds", {})
|
||||
ceph.setdefault("mgr", {})
|
||||
ceph.setdefault("mon", {})
|
||||
ceph.setdefault("log", [])
|
||||
return ceph
|
||||
|
||||
|
||||
async def _save_node_ceph(request: Request, node: str, ceph: dict[str, Any]) -> None:
|
||||
metadata = await node_metadata(request, node)
|
||||
ops = metadata.setdefault("ops", {})
|
||||
ops = metadata.get("ops")
|
||||
if not isinstance(ops, dict):
|
||||
ops = {}
|
||||
metadata["ops"] = ops
|
||||
ops["ceph"] = ceph
|
||||
await save_node_metadata(request, node, metadata)
|
||||
|
||||
@@ -175,10 +132,12 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, str(payload["node"]))
|
||||
ceph = await _load_cluster_ceph(request)
|
||||
keys = [item.strip() for item in str(payload.get("config-keys") or "").split(",") if item]
|
||||
stored = ceph.setdefault("cfg_values", {})
|
||||
result = {key: stored.get(key, "") for key in keys} if keys else dict(stored)
|
||||
await _save_cluster_ceph(request, ceph)
|
||||
return result
|
||||
stored = ceph.get("cfg_values")
|
||||
if not isinstance(stored, dict):
|
||||
stored = {}
|
||||
if keys:
|
||||
return {key: stored.get(key, "") for key in keys}
|
||||
return dict(stored)
|
||||
|
||||
async def crush(request: Request, inputs: dict[str, Any]) -> str:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -202,12 +161,13 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
async def cmd_safety(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
return {
|
||||
"safe": 1,
|
||||
"action": payload.get("action"),
|
||||
"service": payload.get("service"),
|
||||
"id": payload.get("id"),
|
||||
}
|
||||
ceph = await _load_cluster_ceph(request)
|
||||
safety = ceph.get("cmd_safety")
|
||||
result = dict(safety) if isinstance(safety, dict) else {}
|
||||
for key in ("action", "service", "id"):
|
||||
if key in payload:
|
||||
result[key] = payload[key]
|
||||
return result
|
||||
|
||||
async def init(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
@@ -282,11 +242,45 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
await _save_cluster_ceph(request, ceph)
|
||||
return _upid(node, "cephrestart")
|
||||
|
||||
def _wire_ceph_pool(name: str, item: dict[str, Any], *, index: int) -> dict[str, Any]:
|
||||
pool_id = item.get("pool_id", item.get("pool"))
|
||||
try:
|
||||
pool_num = int(pool_id) if pool_id is not None else index
|
||||
except (TypeError, ValueError):
|
||||
pool_num = index
|
||||
crush_raw = item.get("crush_rule", 0)
|
||||
try:
|
||||
crush_id = int(crush_raw)
|
||||
crush_name = str(item.get("crush_rule_name") or "replicated_rule")
|
||||
except (TypeError, ValueError):
|
||||
crush_name = str(crush_raw or "replicated_rule")
|
||||
crush_id = 0
|
||||
application = str(item.get("application") or "rbd")
|
||||
return {
|
||||
"pool": pool_num,
|
||||
"pool_name": str(item.get("pool_name") or name),
|
||||
"size": int(item.get("size") or 3),
|
||||
"min_size": int(item.get("min_size") or 2),
|
||||
"pg_num": int(item.get("pg_num") or 128),
|
||||
"crush_rule": crush_id,
|
||||
"crush_rule_name": crush_name,
|
||||
"application_metadata": item.get("application_metadata")
|
||||
if isinstance(item.get("application_metadata"), dict)
|
||||
else {application: {}},
|
||||
"bytes_used": int(item.get("bytes_used") or 0),
|
||||
"percent_used": float(item.get("percent_used") or 0.0),
|
||||
"healthy": bool(item.get("healthy", True)),
|
||||
}
|
||||
|
||||
async def pool_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
ceph = await _load_cluster_ceph(request)
|
||||
pools = ceph.get("pools") or {}
|
||||
return [dict(item) for _, item in sorted(pools.items()) if isinstance(item, dict)]
|
||||
return [
|
||||
_wire_ceph_pool(name, item, index=index)
|
||||
for index, (name, item) in enumerate(sorted(pools.items()))
|
||||
if isinstance(item, dict)
|
||||
]
|
||||
|
||||
async def pool_create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
@@ -318,7 +312,10 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
pool = (ceph.get("pools") or {}).get(name)
|
||||
if not isinstance(pool, dict):
|
||||
raise ApiError(404, "pool does not exist")
|
||||
return [dict(pool)]
|
||||
# Reuse list wire shape for detail GETs.
|
||||
names = sorted((ceph.get("pools") or {}).keys())
|
||||
index = names.index(name) if name in names else 0
|
||||
return [_wire_ceph_pool(name, pool, index=index)]
|
||||
|
||||
async def pool_update(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
@@ -368,13 +365,7 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
pool = (ceph.get("pools") or {}).get(name)
|
||||
if not isinstance(pool, dict):
|
||||
raise ApiError(404, "pool does not exist")
|
||||
return {
|
||||
**pool,
|
||||
"pg_num": pool.get("pg_num", 128),
|
||||
"bytes_used": 0,
|
||||
"percent_used": 0.0,
|
||||
"healthy": True,
|
||||
}
|
||||
return dict(pool)
|
||||
|
||||
async def fs_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -528,30 +519,63 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
await _save_node_ceph(request, node, ceph)
|
||||
return _upid(node, "cephdestroymon")
|
||||
|
||||
async def osd_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
async def osd_list(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT r.external_id, r.state
|
||||
"""SELECT r.external_id, r.state, n.name AS node
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
WHERE n.name=$1 AND r.kind='ceph-osd'
|
||||
ORDER BY r.external_id""",
|
||||
node,
|
||||
WHERE r.kind='ceph-osd'
|
||||
ORDER BY n.name, r.external_id"""
|
||||
)
|
||||
result: list[dict[str, Any]] = []
|
||||
hosts: dict[str, dict[str, Any]] = {}
|
||||
host_id = -2
|
||||
for row in rows:
|
||||
host_name = str(row["node"])
|
||||
host = hosts.get(host_name)
|
||||
if host is None:
|
||||
host = {
|
||||
"id": host_id,
|
||||
"name": host_name,
|
||||
"type": "host",
|
||||
"type_id": 1,
|
||||
"children": [],
|
||||
}
|
||||
hosts[host_name] = host
|
||||
host_id -= 1
|
||||
payload = state(row["state"])
|
||||
osd_id = payload.get("osd_id", row["external_id"])
|
||||
result.append(
|
||||
osd_raw = payload.get("osd_id", row["external_id"])
|
||||
try:
|
||||
osd_id = int(osd_raw)
|
||||
except (TypeError, ValueError):
|
||||
digits = "".join(ch for ch in str(osd_raw) if ch.isdigit())
|
||||
osd_id = int(digits) if digits else 0
|
||||
host["children"].append(
|
||||
{
|
||||
"osd": int(osd_id) if str(osd_id).isdigit() else osd_id,
|
||||
"id": osd_id,
|
||||
"name": f"osd.{osd_id}",
|
||||
"type": "osd",
|
||||
"type_id": 0,
|
||||
"status": payload.get("status", "up"),
|
||||
"in": 1 if payload.get("in", True) else 0,
|
||||
"weight": payload.get("weight", 1.0),
|
||||
"weight": float(payload.get("weight", 1.0)),
|
||||
"device_class": payload.get("device_class", "hdd"),
|
||||
"host": host_name,
|
||||
}
|
||||
)
|
||||
return result
|
||||
root_children = list(hosts.values())
|
||||
# Prefer the requested node first in the tree for local dumps.
|
||||
root_children.sort(key=lambda item: (0 if item["name"] == node else 1, item["name"]))
|
||||
return {
|
||||
"flags": "",
|
||||
"root": {
|
||||
"id": -1,
|
||||
"name": "default",
|
||||
"type": "root",
|
||||
"type_id": 10,
|
||||
"children": root_children,
|
||||
},
|
||||
}
|
||||
|
||||
async def osd_create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
@@ -562,9 +586,8 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
MAX(NULLIF(regexp_replace(external_id, '\\D', '', 'g'), '')::int),
|
||||
-1
|
||||
) + 1
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
WHERE n.name=$1 AND r.kind='ceph-osd'""",
|
||||
node,
|
||||
FROM resources
|
||||
WHERE kind='ceph-osd'""",
|
||||
)
|
||||
osd_id = int(next_id or 0)
|
||||
external_id = f"osd.{osd_id}"
|
||||
@@ -583,8 +606,15 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
"used_bytes": 0,
|
||||
}
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO resources(id, node_id, kind, external_id, state)
|
||||
VALUES(gen_random_uuid(), $1, 'ceph-osd', $2, $3::jsonb)""",
|
||||
"""INSERT INTO resources(id, node_id, cluster_id, kind, external_id, state)
|
||||
VALUES(
|
||||
gen_random_uuid(),
|
||||
$1,
|
||||
(SELECT cluster_id FROM nodes WHERE id=$1),
|
||||
'ceph-osd',
|
||||
$2,
|
||||
$3::jsonb
|
||||
)""",
|
||||
node_id,
|
||||
external_id,
|
||||
json.dumps(osd_state, sort_keys=True),
|
||||
@@ -682,13 +712,16 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
row = await _osd_row(request, node, osdid)
|
||||
current = state(row["state"])
|
||||
metadata = current.get("metadata")
|
||||
if isinstance(metadata, dict):
|
||||
return dict(metadata)
|
||||
return {
|
||||
"osd": {
|
||||
"id": int(osdid) if osdid.isdigit() else osdid,
|
||||
"uuid": current.get("uuid") or f"osd-uuid-{osdid}",
|
||||
"device_class": current.get("device_class", "hdd"),
|
||||
"uuid": current.get("uuid", ""),
|
||||
"device_class": current.get("device_class", ""),
|
||||
},
|
||||
"devices": [{"dev": current.get("dev") or f"/dev/sd{osdid}"}],
|
||||
"devices": [{"dev": current.get("dev", "")}] if current.get("dev") else [],
|
||||
}
|
||||
|
||||
async def cluster_ceph_status(_request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -698,20 +731,58 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
total = int(row["capacity_bytes"] or 0) if row is not None else 0
|
||||
used = int(row["used_bytes"] or 0) if row is not None else 0
|
||||
osd_count = await database(_request).pool.fetchval(
|
||||
"SELECT count(*)::int FROM resources WHERE kind='ceph-osd'"
|
||||
osd_rows = await database(_request).pool.fetch(
|
||||
"SELECT state FROM resources WHERE kind='ceph-osd'"
|
||||
)
|
||||
num_osds = len(osd_rows)
|
||||
num_up = 0
|
||||
num_in = 0
|
||||
for osd_row in osd_rows:
|
||||
osd_state = state(osd_row["state"])
|
||||
if str(osd_state.get("status") or "") == "up":
|
||||
num_up += 1
|
||||
if osd_state.get("in") in (True, 1, "1"):
|
||||
num_in += 1
|
||||
ceph = await _load_cluster_ceph(_request)
|
||||
version = ceph.get("version")
|
||||
version_str = (
|
||||
str(version.get("str"))
|
||||
if isinstance(version, dict) and version.get("str") is not None
|
||||
else str(version or "")
|
||||
)
|
||||
health = ceph.get("health")
|
||||
if not isinstance(health, dict):
|
||||
health = {"status": "HEALTH_OK" if ceph.get("running") else "HEALTH_WARN"}
|
||||
node_count = int(await database(_request).pool.fetchval("SELECT COUNT(*) FROM nodes") or 0)
|
||||
return {
|
||||
"version": "17.2.7",
|
||||
"health": {"status": "HEALTH_OK" if ceph.get("running", True) else "HEALTH_WARN"},
|
||||
"osdmap": {
|
||||
"num_osds": osd_count,
|
||||
"num_up_osds": osd_count - 1,
|
||||
"num_in_osds": osd_count - 1,
|
||||
"version": {
|
||||
"version": version_str,
|
||||
"release": "reef",
|
||||
"epoch": 0,
|
||||
},
|
||||
"pgmap": {"bytes_used": used, "bytes_total": total},
|
||||
"fsmap": {"filesystems": list((ceph.get("fs") or {}).keys())},
|
||||
"health": health,
|
||||
"osdmap": {
|
||||
"epoch": 1,
|
||||
"num_osds": num_osds,
|
||||
"num_up_osds": num_up,
|
||||
"num_in_osds": num_in,
|
||||
"num_remapped_pgs": 0,
|
||||
},
|
||||
"pgmap": {
|
||||
"bytes_used": used,
|
||||
"bytes_total": total,
|
||||
"bytes_avail": max(total - used, 0),
|
||||
"data_bytes": used,
|
||||
"num_pgs": max(num_osds * 32, 32),
|
||||
},
|
||||
"fsmap": {
|
||||
"epoch": 1,
|
||||
"by_rank": [],
|
||||
"filesystems": list((ceph.get("fs") or {}).keys()),
|
||||
},
|
||||
"monmap": {"num_mons": max(node_count, 1)},
|
||||
"mgrmap": {"available": True, "num_standbys": 0},
|
||||
"servicemap": {"epoch": 1, "services": {}},
|
||||
}
|
||||
|
||||
base = "/nodes/{node}/ceph"
|
||||
|
||||
@@ -24,7 +24,43 @@ def _replication_jobs(metadata: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
jobs = metadata.get("replication", [])
|
||||
if not isinstance(jobs, list):
|
||||
return []
|
||||
return [dict(item) for item in jobs if isinstance(item, dict)]
|
||||
result: list[dict[str, Any]] = []
|
||||
for item in jobs:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
guest_raw = item.get("guest", 0)
|
||||
try:
|
||||
guest = int(guest_raw)
|
||||
except (TypeError, ValueError):
|
||||
guest = int(str(guest_raw).split(":")[-1] or 0)
|
||||
jobnum_raw = item.get("jobnum")
|
||||
if jobnum_raw is None:
|
||||
id_text = str(item.get("id") or "0")
|
||||
tail = id_text.rsplit("-", 1)[-1]
|
||||
jobnum = int(tail) if tail.isdigit() else 0
|
||||
else:
|
||||
jobnum = int(jobnum_raw)
|
||||
disable = item.get("disable")
|
||||
if disable is None:
|
||||
disable = 0 if int(item.get("enabled", 1) or 1) else 1
|
||||
entry = {
|
||||
"id": str(item.get("id") or f"{guest}-{jobnum}"),
|
||||
"guest": guest,
|
||||
"jobnum": jobnum,
|
||||
"target": str(item.get("target") or ""),
|
||||
"type": str(item.get("type") or "local"),
|
||||
"disable": bool(int(disable)),
|
||||
}
|
||||
if item.get("schedule") is not None:
|
||||
entry["schedule"] = str(item["schedule"])
|
||||
if item.get("rate") is not None:
|
||||
entry["rate"] = int(item["rate"])
|
||||
if item.get("comment") is not None:
|
||||
entry["comment"] = str(item["comment"])
|
||||
if item.get("source") is not None:
|
||||
entry["source"] = str(item["source"])
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
|
||||
def register_cluster_handlers(registry: HandlerRegistry) -> None:
|
||||
@@ -46,24 +82,48 @@ def register_cluster_handlers(registry: HandlerRegistry) -> None:
|
||||
"tasks",
|
||||
)
|
||||
|
||||
async def cluster_status(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
rows = await database(_request).pool.fetch(
|
||||
async def cluster_status(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
from app.handlers.nodes import load_node_ops
|
||||
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT id, name, status FROM nodes ORDER BY name"""
|
||||
)
|
||||
result: list[dict[str, Any]] = []
|
||||
metadata = await cluster_metadata(request)
|
||||
quorate = bool(int(metadata.get("quorate", 1) or 1))
|
||||
cluster_name = metadata.get("cluster_config")
|
||||
if isinstance(cluster_name, dict):
|
||||
name = str(cluster_name.get("clustername") or "pve-simulator")
|
||||
version = int(cluster_name.get("version") or 2)
|
||||
else:
|
||||
name = "pve-simulator"
|
||||
version = 2
|
||||
result: list[dict[str, Any]] = [
|
||||
{
|
||||
"type": "cluster",
|
||||
"id": "cluster",
|
||||
"name": name,
|
||||
"nodes": len(rows),
|
||||
"quorate": quorate,
|
||||
"version": version,
|
||||
}
|
||||
]
|
||||
for index, row in enumerate(rows):
|
||||
online = str(row["status"]) == "online"
|
||||
ops = await load_node_ops(request, str(row["name"]))
|
||||
cluster_node = ops.get("cluster_status")
|
||||
entry = dict(cluster_node) if isinstance(cluster_node, dict) else {}
|
||||
local_raw = entry.get("local", index == 0)
|
||||
node_name = str(row["name"])
|
||||
result.append(
|
||||
{
|
||||
"id": str(row["id"]),
|
||||
"name": str(row["name"]),
|
||||
"nodeid": index,
|
||||
"online": 1 if online else 0,
|
||||
"local": 1 if index == 0 else 0,
|
||||
"ip": f"10.32.{index // 254 + 1}.{index % 254 + 10}",
|
||||
"level": "c",
|
||||
"type": "node",
|
||||
"quorate": 1,
|
||||
"id": f"node/{node_name}",
|
||||
"name": node_name,
|
||||
"nodeid": int(entry.get("nodeid", index + 1)),
|
||||
"online": online,
|
||||
"local": bool(int(local_raw)) if not isinstance(local_raw, bool) else local_raw,
|
||||
"ip": str(entry.get("ip") or ops.get("ip") or ""),
|
||||
"level": str(entry.get("level", "")),
|
||||
}
|
||||
)
|
||||
return result
|
||||
@@ -95,14 +155,8 @@ def register_cluster_handlers(registry: HandlerRegistry) -> None:
|
||||
metadata = state(row["metadata"]) if row is not None else {}
|
||||
options = metadata.get("options", {})
|
||||
if not isinstance(options, dict):
|
||||
options = {}
|
||||
return {
|
||||
"keyboard": options.get("keyboard", "en-us"),
|
||||
"email_from": options.get("email_from", "root@localhost"),
|
||||
"http_proxy": options.get("http_proxy", ""),
|
||||
"description": options.get("description", "Proxmox API emulator cluster"),
|
||||
**options,
|
||||
}
|
||||
return {}
|
||||
return dict(options)
|
||||
|
||||
async def cluster_options_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
current = await cluster_options_get(request, inputs)
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import secrets
|
||||
from typing import Any
|
||||
|
||||
@@ -12,50 +13,145 @@ from app.api.registry import HandlerRegistry
|
||||
from app.handlers.common import (
|
||||
cluster_metadata,
|
||||
database,
|
||||
require_value,
|
||||
save_cluster_metadata,
|
||||
subdirs,
|
||||
values,
|
||||
)
|
||||
from app.tasks.repository import TaskRepository
|
||||
from app.tasks.upid import Upid
|
||||
|
||||
|
||||
def _config(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
current = metadata.setdefault(
|
||||
"cluster_config",
|
||||
{
|
||||
"clustername": "pve-simulator",
|
||||
"votes": 1,
|
||||
"links": {},
|
||||
"join_info": {},
|
||||
"totem": {"version": 2, "secauth": "on", "cluster_name": "pve-simulator"},
|
||||
"qdevice": {"status": "disabled"},
|
||||
"apiversion": 1,
|
||||
},
|
||||
current = metadata.get("cluster_config")
|
||||
return current if isinstance(current, dict) else {}
|
||||
|
||||
|
||||
def _principal(request: Request) -> str:
|
||||
return str(getattr(request.state, "principal", None) or "root@pam")
|
||||
|
||||
|
||||
def _fingerprint_sha256() -> str:
|
||||
return ":".join(f"{byte:02X}" for byte in secrets.token_bytes(32))
|
||||
|
||||
|
||||
def _stable_digest(material: str) -> str:
|
||||
return hashlib.sha256(material.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _node_addr(node: str, index: int, entry: dict[str, Any]) -> str:
|
||||
for key in ("pve_addr", "new_node_ip", "ring0_addr"):
|
||||
value = entry.get(key)
|
||||
if isinstance(value, str) and value and not value.endswith(".local"):
|
||||
# Prefer literal IPs; hostnames are still accepted as ring addresses.
|
||||
if key != "ring0_addr" or all(part.isdigit() for part in value.split(".")):
|
||||
return value
|
||||
return f"10.0.0.{index}"
|
||||
|
||||
|
||||
def _ring0_addr(node: str, index: int, entry: dict[str, Any], links: dict[str, Any]) -> str:
|
||||
if entry.get("ring0_addr"):
|
||||
return str(entry["ring0_addr"])
|
||||
link0 = links.get("link0")
|
||||
if isinstance(link0, str) and link0:
|
||||
# Property string may be bare IP or address=IP[,priority=N].
|
||||
if link0.startswith("address="):
|
||||
return link0.removeprefix("address=").split(",", 1)[0]
|
||||
return link0.split(",", 1)[0]
|
||||
if entry.get("new_node_ip"):
|
||||
return str(entry["new_node_ip"])
|
||||
return f"10.0.0.{index}"
|
||||
|
||||
|
||||
def _render_corosync_conf(config: dict[str, Any], node_names: list[str]) -> str:
|
||||
clustername = str(config.get("clustername") or "proxmox")
|
||||
totem = dict(config.get("totem") or {})
|
||||
added_raw = config.get("added_nodes")
|
||||
added: dict[str, Any] = dict(added_raw) if isinstance(added_raw, dict) else {}
|
||||
links = dict(config.get("links") or {})
|
||||
lines = [
|
||||
"totem {",
|
||||
" version: 2",
|
||||
f" cluster_name: {totem.get('cluster_name', clustername)}",
|
||||
f" secauth: {totem.get('secauth', 'on')}",
|
||||
"}",
|
||||
"nodelist {",
|
||||
]
|
||||
for index, name in enumerate(node_names, start=1):
|
||||
entry_raw = added.get(name)
|
||||
entry: dict[str, Any] = dict(entry_raw) if isinstance(entry_raw, dict) else {}
|
||||
nodeid = int(entry.get("nodeid") or index)
|
||||
votes = int(entry.get("quorum_votes") or entry.get("votes") or config.get("votes") or 1)
|
||||
ring0 = _ring0_addr(name, index, entry, links)
|
||||
lines.extend(
|
||||
[
|
||||
" node {",
|
||||
f" name: {name}",
|
||||
f" nodeid: {nodeid}",
|
||||
f" quorum_votes: {votes}",
|
||||
f" ring0_addr: {ring0}",
|
||||
" }",
|
||||
]
|
||||
)
|
||||
if not isinstance(current, dict):
|
||||
current = {
|
||||
"clustername": "pve-simulator",
|
||||
"votes": 1,
|
||||
"links": {},
|
||||
"join_info": {},
|
||||
"totem": {"version": 2, "secauth": "on", "cluster_name": "pve-simulator"},
|
||||
"qdevice": {"status": "disabled"},
|
||||
"apiversion": 1,
|
||||
}
|
||||
metadata["cluster_config"] = current
|
||||
return current
|
||||
lines.append("}")
|
||||
lines.extend(["quorum {", " provider: corosync_votequorum", "}"])
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def _ensure_corosync_materials(
|
||||
config: dict[str, Any], node_names: list[str]
|
||||
) -> tuple[str, str, str]:
|
||||
authkey = config.get("corosync_authkey")
|
||||
if not isinstance(authkey, str) or not authkey:
|
||||
authkey = secrets.token_hex(32)
|
||||
config["corosync_authkey"] = authkey
|
||||
conf = _render_corosync_conf(config, node_names)
|
||||
config["corosync_conf"] = conf
|
||||
digest = _stable_digest(conf)
|
||||
config["config_digest"] = digest
|
||||
return authkey, conf, digest
|
||||
|
||||
|
||||
async def _cluster_task(
|
||||
request: Request, *, task_type: str, worker: str, task_id: str = "0"
|
||||
) -> str:
|
||||
from app.db.primitives import ConflictError
|
||||
|
||||
pool = database(request).pool
|
||||
node = await pool.fetchval("SELECT name FROM nodes ORDER BY name LIMIT 1") or "localhost"
|
||||
upid = str(Upid.allocate(str(node), worker, task_id, _principal(request)))
|
||||
try:
|
||||
task = await TaskRepository(pool).create(
|
||||
upid=upid,
|
||||
task_type=task_type,
|
||||
payload={"cluster": True, "worker": worker},
|
||||
resource_key=f"cluster:{task_type}:{secrets.token_hex(4)}",
|
||||
)
|
||||
except ConflictError as error:
|
||||
raise ApiError(409, str(error)) from error
|
||||
return task.upid
|
||||
|
||||
|
||||
def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
|
||||
async def index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("apiversion", "join", "nodes", "qdevice", "totem")
|
||||
# PVE uses {name}; contract child link is href="{name}".
|
||||
return [{"name": name} for name in ("nodes", "totem", "join", "qdevice", "apiversion")]
|
||||
|
||||
async def create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
clustername = str(require_value(payload, "clustername"))
|
||||
metadata = await cluster_metadata(request)
|
||||
config = _config(metadata)
|
||||
if payload.get("clustername"):
|
||||
config["clustername"] = str(payload["clustername"])
|
||||
config.setdefault("totem", {})["cluster_name"] = str(payload["clustername"])
|
||||
config = dict(_config(metadata))
|
||||
# Match PVE: refuse when corosync materials already exist (seeded or prior create).
|
||||
if config.get("corosync_conf") or config.get("config_digest"):
|
||||
raise ApiError(400, "cluster config already exists")
|
||||
config["clustername"] = clustername
|
||||
totem = dict(config.get("totem") or {})
|
||||
totem["version"] = int(totem.get("version") or 2)
|
||||
totem["secauth"] = totem.get("secauth") or "on"
|
||||
totem["cluster_name"] = clustername
|
||||
# PVE defaults token-coefficient to 125 when omitted (majors that expose it).
|
||||
totem["token_coefficient"] = payload.get("token-coefficient", 125)
|
||||
config["totem"] = totem
|
||||
if "votes" in payload:
|
||||
config["votes"] = payload["votes"]
|
||||
if "nodeid" in payload:
|
||||
@@ -64,12 +160,43 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
|
||||
if links:
|
||||
config["links"] = links
|
||||
config["token"] = secrets.token_hex(16)
|
||||
|
||||
rows = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
node_names = [str(row["name"]) for row in rows]
|
||||
if not node_names:
|
||||
node_names = ["localhost"]
|
||||
creator = node_names[0]
|
||||
added = dict(config.get("added_nodes") or {})
|
||||
creator_entry = dict(added.get(creator) or {})
|
||||
creator_entry.update(
|
||||
{
|
||||
"node": creator,
|
||||
"nodeid": payload.get("nodeid") or creator_entry.get("nodeid") or 1,
|
||||
"quorum_votes": payload.get("votes") or creator_entry.get("quorum_votes") or 1,
|
||||
"ring0_addr": _ring0_addr(
|
||||
creator, 1, creator_entry, dict(config.get("links") or {})
|
||||
),
|
||||
"pve_addr": creator_entry.get("pve_addr") or _node_addr(creator, 1, creator_entry),
|
||||
"pve_fp": creator_entry.get("pve_fp") or _fingerprint_sha256(),
|
||||
}
|
||||
)
|
||||
added[creator] = creator_entry
|
||||
config["added_nodes"] = added
|
||||
_ensure_corosync_materials(config, node_names)
|
||||
|
||||
metadata["cluster_config"] = config
|
||||
await save_cluster_metadata(request, metadata)
|
||||
await database(request).pool.execute(
|
||||
"""UPDATE clusters
|
||||
SET name=$1, updated_at=now()
|
||||
WHERE id=(SELECT id FROM clusters LIMIT 1)""",
|
||||
str(config["clustername"]),
|
||||
clustername,
|
||||
)
|
||||
return await _cluster_task(
|
||||
request,
|
||||
task_type="cluster-create",
|
||||
worker="clustercreate",
|
||||
task_id=clustername,
|
||||
)
|
||||
|
||||
async def apiversion(_request: Request, _inputs: dict[str, Any]) -> int:
|
||||
@@ -79,84 +206,143 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
|
||||
async def join_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = await cluster_metadata(request)
|
||||
config = _config(metadata)
|
||||
node = values(inputs).get("node")
|
||||
preferred_raw = values(inputs).get("node")
|
||||
# Contract documents the omitted-node default as this phrase; PVE uses local nodename.
|
||||
if preferred_raw in {None, "", "current connected node"}:
|
||||
preferred = None
|
||||
else:
|
||||
preferred = str(preferred_raw)
|
||||
rows = await database(request).pool.fetch("SELECT name, status FROM nodes ORDER BY name")
|
||||
nodelist = [
|
||||
{"name": str(row["name"]), "online": 1 if row["status"] == "online" else 0}
|
||||
for row in rows
|
||||
]
|
||||
added_raw = config.get("added_nodes")
|
||||
added: dict[str, Any] = dict(added_raw) if isinstance(added_raw, dict) else {}
|
||||
links = dict(config.get("links") or {})
|
||||
nodelist: list[dict[str, Any]] = []
|
||||
for index, row in enumerate(rows, start=1):
|
||||
name = str(row["name"])
|
||||
entry_raw = added.get(name)
|
||||
entry: dict[str, Any] = dict(entry_raw) if isinstance(entry_raw, dict) else {}
|
||||
if "pve_fp" not in entry:
|
||||
entry["pve_fp"] = _fingerprint_sha256()
|
||||
added[name] = entry
|
||||
if "pve_addr" not in entry:
|
||||
entry["pve_addr"] = _node_addr(name, index, entry)
|
||||
added[name] = entry
|
||||
nodelist.append(
|
||||
{
|
||||
"name": name,
|
||||
"nodeid": int(entry.get("nodeid") or index),
|
||||
"quorum_votes": int(
|
||||
entry.get("quorum_votes") or entry.get("votes") or config.get("votes") or 1
|
||||
),
|
||||
"ring0_addr": _ring0_addr(name, index, entry, links),
|
||||
"pve_addr": str(entry["pve_addr"]),
|
||||
"pve_fp": str(entry["pve_fp"]),
|
||||
}
|
||||
)
|
||||
if added != dict(added_raw or {}):
|
||||
updated = dict(config)
|
||||
updated["added_nodes"] = added
|
||||
if not updated.get("config_digest"):
|
||||
_ensure_corosync_materials(updated, [item["name"] for item in nodelist])
|
||||
metadata["cluster_config"] = updated
|
||||
await save_cluster_metadata(request, metadata)
|
||||
config = updated
|
||||
preferred_node = preferred or (nodelist[0]["name"] if nodelist else None)
|
||||
digest = config.get("config_digest")
|
||||
if not isinstance(digest, str) or not digest:
|
||||
updated = dict(config)
|
||||
_, _, digest = _ensure_corosync_materials(updated, [item["name"] for item in nodelist])
|
||||
metadata["cluster_config"] = updated
|
||||
await save_cluster_metadata(request, metadata)
|
||||
config = updated
|
||||
return {
|
||||
"clustername": config.get("clustername"),
|
||||
"config_digest": secrets.token_hex(8),
|
||||
"nodelist": nodelist,
|
||||
"preferred_node": node or (nodelist[0]["name"] if nodelist else None),
|
||||
"totem": config.get("totem", {}),
|
||||
"links": config.get("links", {}),
|
||||
"preferred_node": preferred_node,
|
||||
"totem": dict(config.get("totem") or {}),
|
||||
"config_digest": digest,
|
||||
}
|
||||
|
||||
async def join_post(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def join_post(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
hostname = str(require_value(payload, "hostname"))
|
||||
require_value(payload, "fingerprint")
|
||||
require_value(payload, "password")
|
||||
metadata = await cluster_metadata(request)
|
||||
config = _config(metadata)
|
||||
hostname = str(payload.get("hostname") or payload.get("node") or "")
|
||||
if not hostname:
|
||||
raise ApiError(400, "parameter verification failed - 'hostname' missing")
|
||||
joins = config.setdefault("join_info", {})
|
||||
config = dict(_config(metadata))
|
||||
joins = dict(config.get("join_info") or {})
|
||||
joins[hostname] = {
|
||||
"hostname": hostname,
|
||||
"fingerprint": payload.get("fingerprint"),
|
||||
"nodeid": payload.get("nodeid"),
|
||||
"votes": payload.get("votes", 1),
|
||||
"force": payload.get("force"),
|
||||
"password_set": True,
|
||||
}
|
||||
# password accepted but not stored in clear form
|
||||
if payload.get("password"):
|
||||
joins[hostname]["password_set"] = True
|
||||
exists = await database(request).pool.fetchval(
|
||||
"SELECT EXISTS(SELECT 1 FROM nodes WHERE name=$1)",
|
||||
hostname,
|
||||
)
|
||||
if not exists:
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO nodes(id, name, status, metadata)
|
||||
VALUES(gen_random_uuid(), $1, 'online', '{}'::jsonb)""",
|
||||
hostname,
|
||||
)
|
||||
links = {key: value for key, value in payload.items() if key.startswith("link")}
|
||||
if links:
|
||||
joins[hostname]["links"] = links
|
||||
config["join_info"] = joins
|
||||
rows = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
node_names = [str(row["name"]) for row in rows]
|
||||
_ensure_corosync_materials(config, node_names or ["localhost"])
|
||||
metadata["cluster_config"] = config
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return await _cluster_task(
|
||||
request,
|
||||
task_type="cluster-join",
|
||||
worker="clusterjoin",
|
||||
)
|
||||
|
||||
async def nodes_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
rows = await database(request).pool.fetch("SELECT name, status FROM nodes ORDER BY name")
|
||||
metadata = await cluster_metadata(request)
|
||||
config = _config(metadata)
|
||||
added_raw = config.get("added_nodes")
|
||||
added: dict[str, Any] = dict(added_raw) if isinstance(added_raw, dict) else {}
|
||||
links = dict(config.get("links") or {})
|
||||
result = []
|
||||
for index, row in enumerate(rows, start=1):
|
||||
name = str(row["name"])
|
||||
entry_raw = added.get(name)
|
||||
entry: dict[str, Any] = dict(entry_raw) if isinstance(entry_raw, dict) else {}
|
||||
result.append(
|
||||
{
|
||||
"node": str(row["name"]),
|
||||
"nodeid": index,
|
||||
"ring0_addr": f"{row['name']}.local",
|
||||
"quorum_votes": config.get("votes", 1),
|
||||
"node": name,
|
||||
"nodeid": entry.get("nodeid", index),
|
||||
"ring0_addr": _ring0_addr(name, index, entry, links),
|
||||
"quorum_votes": entry.get("quorum_votes", config.get("votes", 0)),
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
async def nodes_add(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def nodes_add(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
node = str(require_value(payload, "node"))
|
||||
metadata = await cluster_metadata(request)
|
||||
config = _config(metadata)
|
||||
added = config.setdefault("added_nodes", {})
|
||||
added[node] = {
|
||||
config = dict(_config(metadata))
|
||||
added = dict(config.get("added_nodes") or {})
|
||||
if node in added and not payload.get("force"):
|
||||
raise ApiError(400, f"can't add existing node '{node}'")
|
||||
index = len(added) + 1
|
||||
links = {key: value for key, value in payload.items() if key.startswith("link")}
|
||||
entry = {
|
||||
"node": node,
|
||||
"nodeid": payload.get("nodeid"),
|
||||
"nodeid": payload.get("nodeid") or index,
|
||||
"new_node_ip": payload.get("new_node_ip"),
|
||||
"votes": payload.get("votes", 1),
|
||||
"apiversion": payload.get("apiversion"),
|
||||
"force": payload.get("force"),
|
||||
"ring0_addr": payload.get("ring0_addr")
|
||||
or payload.get("new_node_ip")
|
||||
or _ring0_addr(node, index, {}, links or dict(config.get("links") or {})),
|
||||
"quorum_votes": payload.get("quorum_votes", payload.get("votes", 1)),
|
||||
"pve_addr": payload.get("new_node_ip") or _node_addr(node, index, {}),
|
||||
"pve_fp": _fingerprint_sha256(),
|
||||
}
|
||||
links = {key: value for key, value in payload.items() if key.startswith("link")}
|
||||
if links:
|
||||
added[node]["links"] = links
|
||||
entry["links"] = links
|
||||
added[node] = entry
|
||||
config["added_nodes"] = added
|
||||
exists = await database(request).pool.fetchval(
|
||||
"SELECT EXISTS(SELECT 1 FROM nodes WHERE name=$1)",
|
||||
node,
|
||||
@@ -167,16 +353,33 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
|
||||
VALUES(gen_random_uuid(), $1, 'online', '{}'::jsonb)""",
|
||||
node,
|
||||
)
|
||||
rows = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
node_names = [str(row["name"]) for row in rows]
|
||||
if node not in node_names:
|
||||
node_names.append(node)
|
||||
authkey, conf, _digest = _ensure_corosync_materials(config, node_names)
|
||||
metadata["cluster_config"] = config
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {
|
||||
"corosync_authkey": authkey,
|
||||
"corosync_conf": conf,
|
||||
"warnings": [],
|
||||
}
|
||||
|
||||
async def nodes_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
node = str(values(inputs)["node"])
|
||||
metadata = await cluster_metadata(request)
|
||||
config = _config(metadata)
|
||||
added = config.setdefault("added_nodes", {})
|
||||
config = dict(_config(metadata))
|
||||
added = dict(config.get("added_nodes") or {})
|
||||
added.pop(node, None)
|
||||
joins = config.setdefault("join_info", {})
|
||||
joins = dict(config.get("join_info") or {})
|
||||
joins.pop(node, None)
|
||||
config["added_nodes"] = added
|
||||
config["join_info"] = joins
|
||||
rows = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
node_names = [str(row["name"]) for row in rows if str(row["name"]) != node]
|
||||
_ensure_corosync_materials(config, node_names or ["localhost"])
|
||||
metadata["cluster_config"] = config
|
||||
await save_cluster_metadata(request, metadata)
|
||||
# Keep node row; mark offline to avoid cascading guest deletes.
|
||||
await database(request).pool.execute(
|
||||
@@ -186,11 +389,13 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def qdevice(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = await cluster_metadata(request)
|
||||
return dict(_config(metadata).get("qdevice") or {"status": "disabled"})
|
||||
qdevice = _config(metadata).get("qdevice")
|
||||
return dict(qdevice) if isinstance(qdevice, dict) else {}
|
||||
|
||||
async def totem(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = await cluster_metadata(request)
|
||||
return dict(_config(metadata).get("totem") or {})
|
||||
totem = _config(metadata).get("totem")
|
||||
return dict(totem) if isinstance(totem, dict) else {}
|
||||
|
||||
registry.register("/cluster/config", "GET", index)
|
||||
registry.register("/cluster/config", "POST", create)
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import secrets
|
||||
import time
|
||||
from typing import Any
|
||||
@@ -16,62 +15,28 @@ from app.handlers.common import (
|
||||
database,
|
||||
require_node,
|
||||
save_cluster_metadata,
|
||||
state,
|
||||
subdirs,
|
||||
values,
|
||||
)
|
||||
from app.simulation.ceph_defaults import CEPH_FLAG_DESCRIPTIONS, default_ceph_flags
|
||||
from app.tasks.repository import TaskRepository
|
||||
from app.tasks.upid import Upid
|
||||
|
||||
DEFAULT_CEPH_FLAGS: dict[str, int] = {
|
||||
"nobackfill": 0,
|
||||
"nodeep-scrub": 0,
|
||||
"nodown": 0,
|
||||
"noin": 0,
|
||||
"noout": 0,
|
||||
"norebalance": 0,
|
||||
"norecover": 0,
|
||||
"noscrub": 0,
|
||||
"notieragent": 0,
|
||||
"pause": 0,
|
||||
}
|
||||
|
||||
DEFAULT_CPU_FLAGS: list[dict[str, Any]] = [
|
||||
{"name": "aes", "introduces": "Westmere"},
|
||||
{"name": "avx", "introduces": "SandyBridge"},
|
||||
{"name": "avx2", "introduces": "Haswell"},
|
||||
]
|
||||
|
||||
|
||||
def _jobs(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
jobs = metadata.setdefault("jobs", {})
|
||||
if not isinstance(jobs, dict):
|
||||
jobs = {}
|
||||
metadata["jobs"] = jobs
|
||||
sync = jobs.setdefault("realm_sync", {})
|
||||
if not isinstance(sync, dict):
|
||||
sync = {}
|
||||
jobs["realm_sync"] = sync
|
||||
return jobs
|
||||
jobs = metadata.get("jobs")
|
||||
return jobs if isinstance(jobs, dict) else {}
|
||||
|
||||
|
||||
def _metrics(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
metrics = metadata.setdefault("metrics", {})
|
||||
if not isinstance(metrics, dict):
|
||||
metrics = {}
|
||||
metadata["metrics"] = metrics
|
||||
servers = metrics.setdefault("servers", {})
|
||||
if not isinstance(servers, dict):
|
||||
servers = {}
|
||||
metrics["servers"] = servers
|
||||
return metrics
|
||||
metrics = metadata.get("metrics")
|
||||
return metrics if isinstance(metrics, dict) else {}
|
||||
|
||||
|
||||
def _cpu_models(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
models = metadata.get("qemu_cpu_models")
|
||||
if not isinstance(models, dict):
|
||||
models = {}
|
||||
metadata["qemu_cpu_models"] = models
|
||||
return models
|
||||
return models if isinstance(models, dict) else {}
|
||||
|
||||
|
||||
def _ha_rules_store(metadata: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
@@ -84,12 +49,7 @@ def _ha_rules_store(metadata: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
]
|
||||
if isinstance(rules, list):
|
||||
return [dict(item) for item in rules if isinstance(item, dict)]
|
||||
defaults = [
|
||||
{"rule": "node-fencing", "type": "node", "action": "restart"},
|
||||
{"rule": "service-ha", "type": "resource", "action": "failover"},
|
||||
]
|
||||
metadata["ha_rules"] = defaults
|
||||
return list(defaults)
|
||||
return []
|
||||
|
||||
|
||||
def _save_ha_rules(metadata: dict[str, Any], rules: list[dict[str, Any]]) -> None:
|
||||
@@ -97,26 +57,14 @@ def _save_ha_rules(metadata: dict[str, Any], rules: list[dict[str, Any]]) -> Non
|
||||
|
||||
|
||||
def _replication_jobs(metadata: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
jobs = metadata.get("replication", [])
|
||||
if not isinstance(jobs, list):
|
||||
return []
|
||||
return [dict(item) for item in jobs if isinstance(item, dict)]
|
||||
from app.handlers.cluster import _replication_jobs as normalize
|
||||
|
||||
return normalize(metadata)
|
||||
|
||||
|
||||
def _ceph(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
ceph = metadata.get("ceph")
|
||||
if not isinstance(ceph, dict):
|
||||
ceph = {}
|
||||
flags = ceph.get("flags")
|
||||
if not isinstance(flags, dict):
|
||||
flags = copy.deepcopy(DEFAULT_CEPH_FLAGS)
|
||||
else:
|
||||
merged = copy.deepcopy(DEFAULT_CEPH_FLAGS)
|
||||
merged.update({str(key): int(value) for key, value in flags.items()})
|
||||
flags = merged
|
||||
ceph["flags"] = flags
|
||||
metadata["ceph"] = ceph
|
||||
return ceph
|
||||
return ceph if isinstance(ceph, dict) else {}
|
||||
|
||||
|
||||
async def _cluster_task(request: Request, *, task_type: str, worker: str) -> str:
|
||||
@@ -174,8 +122,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
job_id = str(payload["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _jobs(metadata)
|
||||
sync = jobs.setdefault("realm_sync", {})
|
||||
jobs = dict(_jobs(metadata))
|
||||
sync = dict(jobs.get("realm_sync") or {})
|
||||
if job_id in sync:
|
||||
raise ApiError(409, "realm-sync job already exists")
|
||||
entry = {
|
||||
@@ -185,6 +133,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
entry.setdefault("enabled", 1)
|
||||
entry.setdefault("realm", str(payload.get("realm") or "pam"))
|
||||
sync[job_id] = entry
|
||||
jobs["realm_sync"] = sync
|
||||
metadata["jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {"id": job_id, **entry}
|
||||
@@ -193,8 +142,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
job_id = str(payload["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _jobs(metadata)
|
||||
sync = jobs.setdefault("realm_sync", {})
|
||||
jobs = dict(_jobs(metadata))
|
||||
sync = dict(jobs.get("realm_sync") or {})
|
||||
if job_id not in sync:
|
||||
raise ApiError(404, "realm-sync job does not exist")
|
||||
updated = {
|
||||
@@ -206,6 +155,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
},
|
||||
}
|
||||
sync[job_id] = updated
|
||||
jobs["realm_sync"] = sync
|
||||
metadata["jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {"id": job_id, **updated}
|
||||
@@ -213,23 +163,26 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def realm_sync_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
job_id = str(values(inputs)["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _jobs(metadata)
|
||||
sync = jobs.setdefault("realm_sync", {})
|
||||
jobs = dict(_jobs(metadata))
|
||||
sync = dict(jobs.get("realm_sync") or {})
|
||||
if job_id not in sync:
|
||||
raise ApiError(404, "realm-sync job does not exist")
|
||||
del sync[job_id]
|
||||
jobs["realm_sync"] = sync
|
||||
metadata["jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def schedule_analyze(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
schedule = str(values(inputs).get("schedule") or "*/15")
|
||||
metadata = await cluster_metadata(request)
|
||||
jobs = _jobs(metadata)
|
||||
jobs = dict(_jobs(metadata))
|
||||
results = jobs.get("schedule_analyze_results")
|
||||
if not isinstance(results, list):
|
||||
results = []
|
||||
jobs["last_schedule_analyze"] = {"schedule": schedule, "at": int(time.time())}
|
||||
metadata["jobs"] = jobs
|
||||
await save_cluster_metadata(request, metadata)
|
||||
now = int(time.time())
|
||||
return [{"timestamp": now + offset * 900, "utc": True} for offset in range(4)]
|
||||
return [dict(item) for item in results if isinstance(item, dict)]
|
||||
|
||||
async def metrics_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("export", "server")
|
||||
@@ -237,20 +190,54 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def metrics_export(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = await cluster_metadata(request)
|
||||
metrics = _metrics(metadata)
|
||||
return {
|
||||
"data": metrics.get("export_data")
|
||||
or '# HELP pve_up Node is up\npve_up{node="pve01"} 1\n',
|
||||
"timestamp": int(time.time()),
|
||||
now = int(time.time())
|
||||
seeded = metrics.get("export_series")
|
||||
if isinstance(seeded, list) and seeded:
|
||||
seeded_series = [dict(item) for item in seeded if isinstance(item, dict)]
|
||||
return {"data": seeded_series, "timestamp": now}
|
||||
# Build a PVE-shaped metric array from live inventory (scales with seed).
|
||||
rows = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
series: list[dict[str, Any]] = []
|
||||
for row in rows:
|
||||
node = str(row["name"])
|
||||
series.append(
|
||||
{
|
||||
"id": f"node/{node}",
|
||||
"metric": "up",
|
||||
"timestamp": now,
|
||||
"type": "gauge",
|
||||
"value": 1,
|
||||
}
|
||||
)
|
||||
guests = await database(request).pool.fetch(
|
||||
"""SELECT r.kind, r.external_id, n.name AS node
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
WHERE r.kind IN ('qemu', 'lxc')
|
||||
ORDER BY r.external_id::integer"""
|
||||
)
|
||||
for guest in guests:
|
||||
series.append(
|
||||
{
|
||||
"id": f"{guest['kind']}/{guest['external_id']}",
|
||||
"metric": "up",
|
||||
"timestamp": now,
|
||||
"type": "gauge",
|
||||
"value": 1,
|
||||
}
|
||||
)
|
||||
return {"data": series, "timestamp": now}
|
||||
|
||||
async def metrics_server_list(
|
||||
request: Request, _inputs: dict[str, Any]
|
||||
) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
metrics = _metrics(metadata)
|
||||
servers = metrics.get("servers")
|
||||
if not isinstance(servers, dict):
|
||||
return []
|
||||
return [
|
||||
{"id": server_id, **dict(payload)}
|
||||
for server_id, payload in sorted(metrics.get("servers", {}).items())
|
||||
for server_id, payload in sorted(servers.items())
|
||||
if isinstance(payload, dict)
|
||||
]
|
||||
|
||||
@@ -267,8 +254,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
server_id = str(payload["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
metrics = _metrics(metadata)
|
||||
servers = metrics.setdefault("servers", {})
|
||||
metrics = dict(_metrics(metadata))
|
||||
servers = dict(metrics.get("servers") or {})
|
||||
if server_id in servers:
|
||||
raise ApiError(409, "metrics server already exists")
|
||||
entry = {
|
||||
@@ -279,6 +266,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
entry.setdefault("port", 8086)
|
||||
entry.setdefault("enable", 1)
|
||||
servers[server_id] = entry
|
||||
metrics["servers"] = servers
|
||||
metadata["metrics"] = metrics
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {"id": server_id, **entry}
|
||||
@@ -287,8 +275,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
server_id = str(payload["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
metrics = _metrics(metadata)
|
||||
servers = metrics.setdefault("servers", {})
|
||||
metrics = dict(_metrics(metadata))
|
||||
servers = dict(metrics.get("servers") or {})
|
||||
if server_id not in servers:
|
||||
raise ApiError(404, "metrics server does not exist")
|
||||
updated = {
|
||||
@@ -300,6 +288,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
},
|
||||
}
|
||||
servers[server_id] = updated
|
||||
metrics["servers"] = servers
|
||||
metadata["metrics"] = metrics
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {"id": server_id, **updated}
|
||||
@@ -307,19 +296,24 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def metrics_server_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
server_id = str(values(inputs)["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
metrics = _metrics(metadata)
|
||||
servers = metrics.setdefault("servers", {})
|
||||
metrics = dict(_metrics(metadata))
|
||||
servers = dict(metrics.get("servers") or {})
|
||||
if server_id not in servers:
|
||||
raise ApiError(404, "metrics server does not exist")
|
||||
del servers[server_id]
|
||||
metrics["servers"] = servers
|
||||
metadata["metrics"] = metrics
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def qemu_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("cpu-flags", "custom-cpu-models")
|
||||
|
||||
async def qemu_cpu_flags(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return list(DEFAULT_CPU_FLAGS)
|
||||
async def qemu_cpu_flags(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
flags = metadata.get("qemu_cpu_flags")
|
||||
if isinstance(flags, list):
|
||||
return [dict(item) for item in flags if isinstance(item, dict)]
|
||||
return []
|
||||
|
||||
async def cpu_models_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
@@ -437,41 +431,69 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
) -> list[dict[str, str]]:
|
||||
return subdirs("flags", "metadata", "status")
|
||||
|
||||
async def ceph_flags_get(request: Request, _inputs: dict[str, Any]) -> dict[str, int]:
|
||||
async def ceph_flags_get(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
ceph = _ceph(metadata)
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {str(key): int(value) for key, value in ceph["flags"].items()}
|
||||
stored = ceph.get("flags")
|
||||
flags = default_ceph_flags()
|
||||
if isinstance(stored, dict):
|
||||
for name, value in stored.items():
|
||||
if name in flags:
|
||||
flags[name] = int(value)
|
||||
return [
|
||||
{
|
||||
"name": name,
|
||||
"value": bool(flags[name]),
|
||||
"description": description,
|
||||
}
|
||||
for name, description in sorted(CEPH_FLAG_DESCRIPTIONS.items())
|
||||
]
|
||||
|
||||
async def ceph_flags_put(request: Request, inputs: dict[str, Any]) -> dict[str, int]:
|
||||
async def ceph_flags_put(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
metadata = await cluster_metadata(request)
|
||||
ceph = _ceph(metadata)
|
||||
flags = dict(ceph["flags"])
|
||||
ceph = dict(_ceph(metadata))
|
||||
flags = default_ceph_flags()
|
||||
stored = ceph.get("flags")
|
||||
if isinstance(stored, dict):
|
||||
for name, value in stored.items():
|
||||
if name in flags:
|
||||
flags[name] = int(value)
|
||||
for key, value in payload.items():
|
||||
if key in {"delete", "digest"}:
|
||||
continue
|
||||
if key not in CEPH_FLAG_DESCRIPTIONS:
|
||||
raise ApiError(400, f"unknown ceph flag '{key}'")
|
||||
flags[str(key)] = int(value)
|
||||
ceph["flags"] = flags
|
||||
metadata["ceph"] = ceph
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {str(key): int(value) for key, value in flags.items()}
|
||||
return await _cluster_task(request, task_type="ceph-flags", worker="cephflags")
|
||||
|
||||
async def ceph_flag_get(request: Request, inputs: dict[str, Any]) -> dict[str, int]:
|
||||
async def ceph_flag_get(request: Request, inputs: dict[str, Any]) -> bool:
|
||||
flag = str(values(inputs)["flag"])
|
||||
if flag not in CEPH_FLAG_DESCRIPTIONS:
|
||||
raise ApiError(404, "ceph flag does not exist")
|
||||
metadata = await cluster_metadata(request)
|
||||
ceph = _ceph(metadata)
|
||||
flags = ceph["flags"]
|
||||
if flag not in flags:
|
||||
raise ApiError(404, "ceph flag does not exist")
|
||||
return {flag: int(flags[flag])}
|
||||
flags = ceph.get("flags")
|
||||
if isinstance(flags, dict) and flag in flags:
|
||||
return bool(int(flags[flag]))
|
||||
return False
|
||||
|
||||
async def ceph_flag_put(request: Request, inputs: dict[str, Any]) -> dict[str, int]:
|
||||
async def ceph_flag_put(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
flag = str(payload["flag"])
|
||||
if flag not in CEPH_FLAG_DESCRIPTIONS:
|
||||
raise ApiError(404, "ceph flag does not exist")
|
||||
metadata = await cluster_metadata(request)
|
||||
ceph = _ceph(metadata)
|
||||
flags = dict(ceph["flags"])
|
||||
ceph = dict(_ceph(metadata))
|
||||
flags = default_ceph_flags()
|
||||
stored = ceph.get("flags")
|
||||
if isinstance(stored, dict):
|
||||
for name, value in stored.items():
|
||||
if name in flags:
|
||||
flags[name] = int(value)
|
||||
if "value" in payload:
|
||||
flags[flag] = int(payload["value"])
|
||||
elif flag in payload:
|
||||
@@ -481,19 +503,84 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
ceph["flags"] = flags
|
||||
metadata["ceph"] = ceph
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {flag: int(flags[flag])}
|
||||
|
||||
async def ceph_metadata(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = await cluster_metadata(request)
|
||||
ceph = _ceph(metadata)
|
||||
await save_cluster_metadata(request, metadata)
|
||||
config_raw = ceph.get("config")
|
||||
config: dict[str, Any] = dict(config_raw) if isinstance(config_raw, dict) else {}
|
||||
version = ceph.get("version")
|
||||
nodes = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
node_names = [str(row["name"]) for row in nodes]
|
||||
version_str = (
|
||||
str(version.get("str"))
|
||||
if isinstance(version, dict) and version.get("str")
|
||||
else "18.2.2"
|
||||
)
|
||||
version_short = ".".join(version_str.split(".")[:2]) if version_str else "18.2"
|
||||
osd_rows = await database(request).pool.fetch(
|
||||
"""SELECT r.external_id, r.state, n.name AS node
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
WHERE r.kind='ceph-osd' ORDER BY r.external_id"""
|
||||
)
|
||||
osd_map: dict[str, Any] = {}
|
||||
for row in osd_rows:
|
||||
payload = state(row["state"])
|
||||
external = str(row["external_id"])
|
||||
osd_id = str(payload.get("osd_id", external.removeprefix("osd.")))
|
||||
hostname = str(row["node"])
|
||||
osd_map[osd_id] = {
|
||||
"id": int(osd_id) if osd_id.isdigit() else osd_id,
|
||||
"hostname": hostname,
|
||||
"name": f"osd.{osd_id}",
|
||||
"ceph_version": version_str,
|
||||
"ceph_version_short": version_short,
|
||||
"ceph_release": "reef",
|
||||
"device_class": payload.get("device_class", "hdd"),
|
||||
"dev": payload.get("dev", ""),
|
||||
}
|
||||
if not osd_map:
|
||||
fallback_host = node_names[0] if node_names else "pve01"
|
||||
osd_map["0"] = {
|
||||
"id": 0,
|
||||
"hostname": fallback_host,
|
||||
"name": "osd.0",
|
||||
"ceph_version": version_str,
|
||||
"ceph_version_short": version_short,
|
||||
"ceph_release": "reef",
|
||||
}
|
||||
|
||||
def _service_entry(name: str, index: int) -> dict[str, Any]:
|
||||
return {
|
||||
"version": ceph.get("version") or {"str": "18.2.2", "parts": [18, 2, 2]},
|
||||
"fsid": ceph.get("config", {}).get("fsid")
|
||||
if isinstance(ceph.get("config"), dict)
|
||||
else "pve-simulator-fsid",
|
||||
"initialized": int(bool(ceph.get("initialized", True))),
|
||||
"flags": ceph.get("flags", {}),
|
||||
"name": name,
|
||||
"hostname": name,
|
||||
"addr": f"10.0.0.{index}",
|
||||
"ceph_version": version_str,
|
||||
"ceph_version_short": version_short,
|
||||
"ceph_release": "reef",
|
||||
}
|
||||
|
||||
return {
|
||||
"mds": {},
|
||||
"mgr": {
|
||||
f"mgr@{node}": _service_entry(node, index)
|
||||
for index, node in enumerate(node_names[:1] or ["pve01"], start=1)
|
||||
},
|
||||
"mon": {
|
||||
f"mon@{node}": _service_entry(node, index)
|
||||
for index, node in enumerate(node_names[:3] or ["pve01"], start=1)
|
||||
},
|
||||
"node": {
|
||||
node: {
|
||||
"name": node,
|
||||
"hostname": node,
|
||||
"ceph_version": version_str,
|
||||
"ceph_version_short": version_short,
|
||||
}
|
||||
for node in (node_names or ["pve01"])
|
||||
},
|
||||
"osd": list(osd_map.values()),
|
||||
"fsid": str(config.get("fsid") or ""),
|
||||
}
|
||||
|
||||
async def ha_rule_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
@@ -588,7 +675,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
log = job.get("log")
|
||||
if isinstance(log, list):
|
||||
return [dict(item) for item in log if isinstance(item, dict)]
|
||||
return [{"t": int(time.time()), "n": 0, "msg": f"replication idle for {job.get('id')}"}]
|
||||
return []
|
||||
|
||||
async def node_replication_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
job = await node_replication_get(request, inputs)
|
||||
|
||||
@@ -76,7 +76,11 @@ async def require_node(request: Request, node: str) -> None:
|
||||
node,
|
||||
)
|
||||
if not exists:
|
||||
raise ApiError(404, "node does not exist")
|
||||
raise ApiError(
|
||||
404,
|
||||
f"No such node ('{node}')",
|
||||
errors={"node": f"No such node ('{node}')"},
|
||||
)
|
||||
|
||||
|
||||
async def cluster_metadata(request: Request) -> dict[str, Any]:
|
||||
@@ -105,7 +109,11 @@ async def node_metadata(request: Request, node: str) -> dict[str, Any]:
|
||||
node,
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "node does not exist")
|
||||
raise ApiError(
|
||||
404,
|
||||
f"No such node ('{node}')",
|
||||
errors={"node": f"No such node ('{node}')"},
|
||||
)
|
||||
return state(row["metadata"])
|
||||
|
||||
|
||||
@@ -116,7 +124,11 @@ async def save_node_metadata(request: Request, node: str, metadata: dict[str, An
|
||||
json.dumps(metadata, sort_keys=True),
|
||||
)
|
||||
if status != "UPDATE 1":
|
||||
raise ApiError(404, "node does not exist")
|
||||
raise ApiError(
|
||||
404,
|
||||
f"No such node ('{node}')",
|
||||
errors={"node": f"No such node ('{node}')"},
|
||||
)
|
||||
|
||||
|
||||
def storage_payload(row: Any) -> dict[str, Any]:
|
||||
@@ -129,13 +141,24 @@ def storage_payload(row: Any) -> dict[str, Any]:
|
||||
total = int(row["capacity_bytes"] or 0)
|
||||
used = int(row["used_bytes"] or 0)
|
||||
avail = max(total - used, 0)
|
||||
storage_type = str(row["storage_type"])
|
||||
formats = config.get("formats")
|
||||
if not isinstance(formats, str) or not formats:
|
||||
formats = {
|
||||
"dir": "raw,qcow2,vmdk",
|
||||
"lvmthin": "raw",
|
||||
"rbd": "raw",
|
||||
"zfspool": "raw,subvol",
|
||||
}.get(storage_type, "raw")
|
||||
payload: dict[str, Any] = {
|
||||
"storage": str(row["storage_id"]),
|
||||
"type": str(row["storage_type"]),
|
||||
"type": storage_type,
|
||||
"shared": int(bool(row["shared"])),
|
||||
"content": content_str,
|
||||
"active": 1,
|
||||
"enabled": 1,
|
||||
"active": int(bool(config.get("active", True))),
|
||||
"enabled": int(bool(config.get("enabled", True))),
|
||||
"formats": formats,
|
||||
"select_existing": int(bool(config.get("select_existing", False))),
|
||||
"total": total,
|
||||
"used": used,
|
||||
"avail": avail,
|
||||
|
||||
@@ -19,7 +19,7 @@ from app.handlers.ceph import register_ceph_handlers
|
||||
from app.handlers.cluster import register_cluster_handlers
|
||||
from app.handlers.cluster_config import register_cluster_config_handlers
|
||||
from app.handlers.cluster_extra import register_cluster_extra_handlers
|
||||
from app.handlers.common import require_node, subdirs
|
||||
from app.handlers.common import cluster_metadata, require_node, subdirs
|
||||
from app.handlers.firewall import register_firewall_handlers
|
||||
from app.handlers.ha import register_ha_handlers
|
||||
from app.handlers.legacy_aliases import register_legacy_aliases
|
||||
@@ -60,50 +60,358 @@ def build_core_handlers(settings: Settings) -> HandlerRegistry:
|
||||
raise ApiError(401, "authentication failure")
|
||||
key = settings.ticket_signing_key.get_secret_value().encode()
|
||||
ticket = issue_ticket(username, key)
|
||||
cluster_name = await _database(request).pool.fetchval(
|
||||
"SELECT name FROM clusters ORDER BY created_at LIMIT 1"
|
||||
)
|
||||
return {
|
||||
"username": username,
|
||||
"ticket": ticket,
|
||||
"CSRFPreventionToken": csrf_token(ticket, key),
|
||||
"clustername": str(cluster_name or "pve-simulator"),
|
||||
"cap": {"vms": {"VM.Audit": 1, "VM.PowerMgmt": 1}},
|
||||
}
|
||||
|
||||
async def nodes(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
from app.handlers.nodes import load_node_ops
|
||||
|
||||
rows = await _database(request).pool.fetch(
|
||||
"SELECT name AS node, status FROM nodes ORDER BY name"
|
||||
)
|
||||
return [{"node": str(row["node"]), "status": str(row["status"])} for row in rows]
|
||||
result: list[dict[str, Any]] = []
|
||||
for row in rows:
|
||||
name = str(row["node"])
|
||||
ops = await load_node_ops(request, name)
|
||||
status_payload = ops.get("status")
|
||||
status_dict = dict(status_payload) if isinstance(status_payload, dict) else {}
|
||||
fingerprint = status_dict.get("ssl_fingerprint") or status_dict.get("fingerprint")
|
||||
if fingerprint in (None, "", 0) or isinstance(fingerprint, dict | list):
|
||||
from app.simulation.seed import _seed_fingerprint
|
||||
|
||||
fingerprint = _seed_fingerprint(name)
|
||||
|
||||
def _as_float(value: object, default: float) -> float:
|
||||
if isinstance(value, bool) or value is None or isinstance(value, dict | list):
|
||||
return default
|
||||
try:
|
||||
return float(str(value))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
def _as_int(value: object, default: int) -> int:
|
||||
if isinstance(value, bool) or value is None or isinstance(value, dict | list):
|
||||
return default
|
||||
try:
|
||||
return int(float(str(value)))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
item: dict[str, Any] = {
|
||||
"node": name,
|
||||
"status": str(row["status"]),
|
||||
"type": "node",
|
||||
"ssl_fingerprint": str(fingerprint),
|
||||
"cpu": _as_float(status_dict.get("cpu"), 0.0),
|
||||
"maxcpu": _as_int(status_dict.get("maxcpu"), 4),
|
||||
"mem": _as_int(status_dict.get("mem"), _as_int(status_dict.get("memory"), 0)),
|
||||
"maxmem": _as_int(status_dict.get("maxmem"), 8 * 1024**3),
|
||||
"uptime": _as_int(status_dict.get("uptime"), 0),
|
||||
"level": str(status_dict.get("level") or ""),
|
||||
}
|
||||
result.append(item)
|
||||
return result
|
||||
|
||||
async def node_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
from app.handlers.nodes import load_node_ops
|
||||
|
||||
node = str(cast(dict[str, Any], inputs["values"])["node"])
|
||||
row = await _database(request).pool.fetchrow(
|
||||
"SELECT name, status FROM nodes WHERE name=$1", node
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "node does not exist")
|
||||
ops = await load_node_ops(request, node)
|
||||
status = ops.get("status")
|
||||
payload = dict(status) if isinstance(status, dict) else {}
|
||||
mem_used = int(payload.get("mem") or 0)
|
||||
mem_total = int(payload.get("maxmem") or 8 * 1024**3)
|
||||
memory = payload.get("memory")
|
||||
if isinstance(memory, dict):
|
||||
mem_used = int(memory.get("used") or mem_used)
|
||||
mem_total = int(memory.get("total") or mem_total)
|
||||
rootfs = payload.get("rootfs")
|
||||
if not isinstance(rootfs, dict):
|
||||
rootfs = {
|
||||
"used": int(payload.get("rootfs_used") or mem_used // 8),
|
||||
"total": int(payload.get("rootfs_total") or 100 * 1024**3),
|
||||
"avail": int(payload.get("rootfs_avail") or 80 * 1024**3),
|
||||
"free": int(payload.get("rootfs_free") or 80 * 1024**3),
|
||||
}
|
||||
cpuinfo = payload.get("cpuinfo")
|
||||
if not isinstance(cpuinfo, dict):
|
||||
cpuinfo = {
|
||||
"cpus": int(payload.get("maxcpu") or 4),
|
||||
"cores": int(payload.get("maxcpu") or 4),
|
||||
"sockets": 1,
|
||||
"model": str(payload.get("cpu_model") or "QEMU Virtual CPU"),
|
||||
"flags": "",
|
||||
}
|
||||
boot_info = payload.get("boot-info")
|
||||
if not isinstance(boot_info, dict):
|
||||
boot_info = {"mode": "efi", "secureboot": 0}
|
||||
current_kernel = payload.get("current-kernel")
|
||||
if not isinstance(current_kernel, dict):
|
||||
kversion = str(payload.get("kversion") or "6.8.12-1-pve")
|
||||
current_kernel = {
|
||||
"sysname": "Linux",
|
||||
"release": kversion,
|
||||
"version": f"#1 SMP PREEMPT_DYNAMIC {kversion}",
|
||||
"machine": "x86_64",
|
||||
}
|
||||
return {
|
||||
"status": str(row["status"]),
|
||||
"node": str(row["name"]),
|
||||
"uptime": 0,
|
||||
"cpu": 0.0,
|
||||
"memory": {"used": 0, "total": 0},
|
||||
"uptime": int(payload.get("uptime") or 0),
|
||||
"wait": float(payload.get("wait") or 0.0),
|
||||
"idle": float(payload.get("idle") or 0.95),
|
||||
"cpu": float(payload.get("cpu") or 0.0),
|
||||
"loadavg": list(payload.get("loadavg") or ["0.00", "0.00", "0.00"]),
|
||||
"memory": {
|
||||
"used": mem_used,
|
||||
"total": mem_total,
|
||||
"free": max(mem_total - mem_used, 0),
|
||||
"available": max(mem_total - mem_used, 0),
|
||||
},
|
||||
"rootfs": rootfs,
|
||||
"swap": payload.get("swap")
|
||||
if isinstance(payload.get("swap"), dict)
|
||||
else {"used": 0, "total": 0, "free": 0},
|
||||
"cpuinfo": cpuinfo,
|
||||
"kversion": str(payload.get("kversion") or current_kernel.get("release") or ""),
|
||||
"pveversion": str(payload.get("pveversion") or "pve-manager/9.2.3"),
|
||||
"current-kernel": current_kernel,
|
||||
"boot-info": boot_info,
|
||||
}
|
||||
|
||||
async def resources(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
async def resources(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
"""Cluster-wide inventory in Proxmox ``/cluster/resources`` shape.
|
||||
|
||||
Do not dump guest ``state`` / config blobs: QEMU ``cpu`` is a model
|
||||
string (e.g. ``qemu64``), while this endpoint's ``cpu`` is utilization
|
||||
(float). Bridged clients (bpg / pulumi-proxmoxve) decode strictly.
|
||||
"""
|
||||
|
||||
def _as_dict(raw: object) -> dict[str, Any]:
|
||||
if isinstance(raw, str):
|
||||
loaded = json.loads(raw)
|
||||
return dict(loaded) if isinstance(loaded, dict) else {}
|
||||
return dict(raw) if isinstance(raw, dict) else {}
|
||||
|
||||
def _num(value: object, default: float | int) -> float | int:
|
||||
if isinstance(value, bool) or value is None or isinstance(value, dict | list):
|
||||
return default
|
||||
try:
|
||||
if isinstance(default, float):
|
||||
return float(str(value))
|
||||
return int(float(str(value)))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
def _memory_bytes(state: dict[str, Any]) -> int:
|
||||
raw = state.get("maxmem", state.get("memory"))
|
||||
if raw in (None, ""):
|
||||
return 0
|
||||
value = _num(raw, 0)
|
||||
# QEMU config stores memory in MiB; cluster resources use bytes.
|
||||
if isinstance(raw, str) or (isinstance(value, int) and 0 < value < 10_000_000):
|
||||
return int(value) * 1024 * 1024
|
||||
return int(value)
|
||||
|
||||
def _maxcpu(state: dict[str, Any]) -> int:
|
||||
cores = int(_num(state.get("cores", state.get("cpus", 1)), 1))
|
||||
sockets = int(_num(state.get("sockets", 1), 1))
|
||||
return max(cores * sockets, 1)
|
||||
|
||||
def _cpu_util(state: dict[str, Any], *, running: bool) -> float:
|
||||
if not running:
|
||||
return 0.0
|
||||
samples = state.get("rrddata")
|
||||
if isinstance(samples, list) and samples:
|
||||
last = samples[-1]
|
||||
if isinstance(last, dict):
|
||||
return float(_num(last.get("cpu"), 0.0))
|
||||
return (
|
||||
float(_num(state.get("cpu"), 0.0)) if not isinstance(state.get("cpu"), str) else 0.0
|
||||
)
|
||||
|
||||
type_filter = cast(dict[str, Any], inputs.get("values") or {}).get("type")
|
||||
result: list[dict[str, Any]] = []
|
||||
|
||||
if type_filter in (None, "node"):
|
||||
node_rows = await _database(request).pool.fetch(
|
||||
"SELECT name AS node, status FROM nodes ORDER BY name"
|
||||
)
|
||||
from app.handlers.nodes import load_node_ops
|
||||
|
||||
for row in node_rows:
|
||||
name = str(row["node"])
|
||||
ops = await load_node_ops(request, name)
|
||||
status_payload = ops.get("status")
|
||||
status_dict = dict(status_payload) if isinstance(status_payload, dict) else {}
|
||||
mem_raw = status_dict.get("mem", status_dict.get("memory"))
|
||||
if isinstance(mem_raw, dict):
|
||||
mem_val = int(_num(mem_raw.get("used"), 0))
|
||||
maxmem_val = int(
|
||||
_num(status_dict.get("maxmem", mem_raw.get("total")), 8 * 1024**3)
|
||||
)
|
||||
else:
|
||||
mem_val = int(_num(mem_raw, 0))
|
||||
maxmem_val = int(_num(status_dict.get("maxmem"), 8 * 1024**3))
|
||||
result.append(
|
||||
{
|
||||
"type": "node",
|
||||
"id": f"node/{name}",
|
||||
"node": name,
|
||||
"status": str(row["status"]),
|
||||
"cpu": float(_num(status_dict.get("cpu"), 0.0)),
|
||||
"maxcpu": int(_num(status_dict.get("maxcpu"), 4)),
|
||||
"mem": mem_val,
|
||||
"maxmem": maxmem_val,
|
||||
"uptime": int(_num(status_dict.get("uptime"), 0)),
|
||||
"level": str(status_dict.get("level") or ""),
|
||||
}
|
||||
)
|
||||
|
||||
kind_filter: tuple[str, ...] | None
|
||||
if type_filter == "vm":
|
||||
kind_filter = ("qemu", "lxc")
|
||||
elif type_filter == "storage":
|
||||
kind_filter = ("storage",)
|
||||
elif type_filter in (None,):
|
||||
kind_filter = ("qemu", "lxc", "storage", "pool")
|
||||
elif type_filter in {"qemu", "lxc", "storage", "pool", "sdn"}:
|
||||
kind_filter = (str(type_filter),)
|
||||
else:
|
||||
kind_filter = ()
|
||||
|
||||
ha_by_guest: dict[str, str] = {}
|
||||
if kind_filter and ("qemu" in kind_filter or "lxc" in kind_filter):
|
||||
ha_rows = await _database(request).pool.fetch(
|
||||
"SELECT external_id, state FROM resources WHERE kind='ha'"
|
||||
)
|
||||
for ha_row in ha_rows:
|
||||
sid = str(ha_row["external_id"])
|
||||
ha_state = _as_dict(ha_row["state"])
|
||||
ha_by_guest[sid] = str(ha_state.get("state") or "started")
|
||||
|
||||
pool_by_vmid: dict[str, str] = {}
|
||||
if kind_filter and ("qemu" in kind_filter or "lxc" in kind_filter or "pool" in kind_filter):
|
||||
membership = await _database(request).pool.fetch(
|
||||
"""SELECT p.pool_id AS pool, r.external_id AS vmid
|
||||
FROM pool_members pm
|
||||
JOIN pools p ON p.id=pm.pool_id
|
||||
JOIN resources r ON r.id=pm.resource_id"""
|
||||
)
|
||||
for row in membership:
|
||||
pool_by_vmid[str(row["vmid"])] = str(row["pool"])
|
||||
|
||||
node_names = [
|
||||
str(item["node"])
|
||||
for item in await _database(request).pool.fetch(
|
||||
"SELECT name AS node FROM nodes ORDER BY name"
|
||||
)
|
||||
]
|
||||
|
||||
if kind_filter:
|
||||
rows = await _database(request).pool.fetch(
|
||||
"""SELECT r.kind AS type, r.external_id, r.state, n.name AS node
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
ORDER BY r.kind, r.external_id"""
|
||||
WHERE r.kind = ANY($1::text[])
|
||||
ORDER BY r.kind, r.external_id""",
|
||||
list(kind_filter),
|
||||
)
|
||||
result: list[dict[str, Any]] = []
|
||||
for row in rows:
|
||||
raw_state = row["state"]
|
||||
state = json.loads(raw_state) if isinstance(raw_state, str) else dict(raw_state)
|
||||
kind = str(row["type"])
|
||||
external_id = str(row["external_id"])
|
||||
node = str(row["node"])
|
||||
state = _as_dict(row["state"])
|
||||
if kind in {"qemu", "lxc"}:
|
||||
status = str(state.get("status") or "stopped")
|
||||
running = status in {"running", "paused"}
|
||||
vmid = int(external_id)
|
||||
sid = f"{'vm' if kind == 'qemu' else 'ct'}:{external_id}"
|
||||
item: dict[str, Any] = {
|
||||
"type": kind,
|
||||
"id": f"{kind}/{external_id}",
|
||||
"node": node,
|
||||
"vmid": vmid,
|
||||
"name": str(state.get("name") or f"{kind}-{external_id}"),
|
||||
"status": status,
|
||||
"template": bool(state.get("template") in (True, "1", 1)),
|
||||
"cpu": _cpu_util(state, running=running),
|
||||
"maxcpu": _maxcpu(state),
|
||||
"mem": int(_num(state.get("mem"), 0)) if running else 0,
|
||||
"maxmem": _memory_bytes(state),
|
||||
"disk": int(_num(state.get("disk"), 0)),
|
||||
"maxdisk": int(_num(state.get("maxdisk"), 0)),
|
||||
"uptime": int(_num(state.get("uptime"), 0)) if running else 0,
|
||||
}
|
||||
if sid in ha_by_guest:
|
||||
item["hastate"] = ha_by_guest[sid]
|
||||
if external_id in pool_by_vmid:
|
||||
item["pool"] = pool_by_vmid[external_id]
|
||||
result.append(item)
|
||||
elif kind == "storage":
|
||||
content = state.get("content")
|
||||
if isinstance(content, list):
|
||||
content_text = ",".join(str(part) for part in content)
|
||||
else:
|
||||
content_text = str(content or "")
|
||||
shared = bool(_num(state.get("shared"), 0))
|
||||
# Schema allows one storages row per storage_id; expand to every
|
||||
# node like a real PVE dump (local + shared storages).
|
||||
targets = node_names if (shared or len(node_names) > 1) else [node]
|
||||
if not shared and external_id not in {"local", "local-lvm", "ceph"}:
|
||||
targets = [node]
|
||||
for target in targets:
|
||||
result.append(
|
||||
{
|
||||
"type": str(row["type"]),
|
||||
"id": f"{row['type']}/{row['external_id']}",
|
||||
"node": str(row["node"]),
|
||||
**state,
|
||||
"type": "storage",
|
||||
"id": f"storage/{target}/{external_id}",
|
||||
"node": target,
|
||||
"storage": external_id,
|
||||
"status": str(state.get("status") or "available"),
|
||||
"content": content_text,
|
||||
"disk": int(_num(state.get("disk"), 0)),
|
||||
"maxdisk": int(_num(state.get("maxdisk"), 1 * 1024**3)),
|
||||
"shared": shared,
|
||||
"plugintype": str(
|
||||
state.get("plugintype") or state.get("type") or "dir"
|
||||
),
|
||||
}
|
||||
)
|
||||
elif kind == "pool":
|
||||
result.append(
|
||||
{
|
||||
"type": "pool",
|
||||
"id": f"pool/{external_id}",
|
||||
"pool": external_id,
|
||||
"comment": str(state.get("comment") or ""),
|
||||
}
|
||||
)
|
||||
|
||||
if type_filter in (None, "sdn"):
|
||||
metadata = await cluster_metadata(request)
|
||||
sdn = metadata.get("sdn")
|
||||
zones = sdn.get("zones") if isinstance(sdn, dict) else None
|
||||
if isinstance(zones, dict):
|
||||
for zone_name, zone in sorted(zones.items()):
|
||||
if not isinstance(zone, dict):
|
||||
continue
|
||||
result.append(
|
||||
{
|
||||
"type": "sdn",
|
||||
"id": f"sdn/{zone_name}",
|
||||
"sdn": zone_name,
|
||||
"status": str(zone.get("status") or "ok"),
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
@@ -13,20 +13,6 @@ from app.api.registry import HandlerRegistry
|
||||
from app.handlers.common import database, require_node, state, subdirs, values
|
||||
from app.simulation.seed import CLUSTER_ID
|
||||
|
||||
DEFAULT_OPTIONS = {
|
||||
"enable": 1,
|
||||
"policy_in": "DROP",
|
||||
"policy_out": "ACCEPT",
|
||||
"log_level_in": "nolog",
|
||||
"log_level_out": "nolog",
|
||||
}
|
||||
|
||||
DEFAULT_MACROS = [
|
||||
{"macro": "SSH", "descr": "Secure Shell"},
|
||||
{"macro": "HTTPS", "descr": "Secure web server"},
|
||||
{"macro": "HTTP", "descr": "Web server"},
|
||||
]
|
||||
|
||||
ScopeFn = Callable[[dict[str, Any]], str]
|
||||
|
||||
|
||||
@@ -50,19 +36,36 @@ async def _save_firewall(request: Request, firewall: dict[str, Any]) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _scope_data(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
|
||||
scopes = firewall.setdefault("scopes", {})
|
||||
if scope not in scopes or not isinstance(scopes[scope], dict):
|
||||
scopes[scope] = {
|
||||
"options": dict(DEFAULT_OPTIONS),
|
||||
def _empty_scope() -> dict[str, Any]:
|
||||
return {
|
||||
"options": {},
|
||||
"rules": [],
|
||||
"aliases": {},
|
||||
"ipset": {},
|
||||
"groups": {},
|
||||
"log": [],
|
||||
}
|
||||
section = scopes[scope]
|
||||
section.setdefault("options", dict(DEFAULT_OPTIONS))
|
||||
|
||||
|
||||
def _get_scope(firewall: dict[str, Any], scope: str) -> dict[str, Any] | None:
|
||||
scopes = firewall.get("scopes")
|
||||
if not isinstance(scopes, dict):
|
||||
return None
|
||||
section = scopes.get(scope)
|
||||
return cast(dict[str, Any], section) if isinstance(section, dict) else None
|
||||
|
||||
|
||||
def _ensure_scope(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
|
||||
"""Create an empty durable scope on mutation; never injects catalog defaults."""
|
||||
scopes = firewall.setdefault("scopes", {})
|
||||
if not isinstance(scopes, dict):
|
||||
scopes = {}
|
||||
firewall["scopes"] = scopes
|
||||
section = scopes.get(scope)
|
||||
if not isinstance(section, dict):
|
||||
section = _empty_scope()
|
||||
scopes[scope] = section
|
||||
section.setdefault("options", {})
|
||||
section.setdefault("rules", [])
|
||||
section.setdefault("aliases", {})
|
||||
section.setdefault("ipset", {})
|
||||
@@ -71,6 +74,11 @@ def _scope_data(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
|
||||
return cast(dict[str, Any], section)
|
||||
|
||||
|
||||
def _scope_data(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
|
||||
"""Mutation helper — ensure scope exists without template defaults."""
|
||||
return _ensure_scope(firewall, scope)
|
||||
|
||||
|
||||
def register_firewall_handlers(registry: HandlerRegistry) -> None:
|
||||
def register_scope(
|
||||
base: str,
|
||||
@@ -98,25 +106,31 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
|
||||
async def options_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
return dict(_scope_data(firewall, scope_fn(payload)).get("options", DEFAULT_OPTIONS))
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
if section is None:
|
||||
return {}
|
||||
options = section.get("options", {})
|
||||
return dict(options) if isinstance(options, dict) else {}
|
||||
|
||||
async def options_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
async def options_put(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
section = _scope_data(firewall, scope_fn(payload))
|
||||
current = dict(section.get("options", DEFAULT_OPTIONS))
|
||||
section = _ensure_scope(firewall, scope_fn(payload))
|
||||
current = dict(section.get("options") or {})
|
||||
for key, value in payload.items():
|
||||
if key in {"node", "vmid", "delete", "digest"}:
|
||||
continue
|
||||
current[key] = value
|
||||
section["options"] = current
|
||||
await _save_firewall(request, firewall)
|
||||
return current
|
||||
|
||||
async def rules_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
rules = _scope_data(firewall, scope_fn(payload)).get("rules", [])
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
if section is None:
|
||||
return []
|
||||
rules = section.get("rules", [])
|
||||
return list(rules) if isinstance(rules, list) else []
|
||||
|
||||
async def rules_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
@@ -169,7 +183,8 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
|
||||
async def aliases_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
aliases = _scope_data(firewall, scope_fn(payload)).get("aliases", {})
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
aliases = section.get("aliases", {}) if section else {}
|
||||
if not isinstance(aliases, dict):
|
||||
return []
|
||||
return [
|
||||
@@ -236,7 +251,8 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
|
||||
async def ipset_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
ipsets = _scope_data(firewall, scope_fn(payload)).get("ipset", {})
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
ipsets = section.get("ipset", {}) if section else {}
|
||||
if not isinstance(ipsets, dict):
|
||||
return []
|
||||
return [
|
||||
@@ -352,27 +368,43 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
|
||||
async def refs_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
section = _scope_data(firewall, scope_fn(payload))
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
if section is None:
|
||||
return []
|
||||
refs: list[dict[str, Any]] = []
|
||||
for name in section.get("aliases", {}):
|
||||
for name in section.get("aliases", {}) or {}:
|
||||
refs.append({"type": "alias", "name": name})
|
||||
for name in section.get("ipset", {}):
|
||||
for name in section.get("ipset", {}) or {}:
|
||||
refs.append({"type": "ipset", "name": name})
|
||||
return refs
|
||||
|
||||
async def log_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
log = _scope_data(firewall, scope_fn(payload)).get("log", [])
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
if section is None:
|
||||
return []
|
||||
log = section.get("log", [])
|
||||
return list(log) if isinstance(log, list) else []
|
||||
|
||||
async def macros_list(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return list(DEFAULT_MACROS)
|
||||
async def macros_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
from app.handlers.common import cluster_metadata
|
||||
|
||||
metadata = await cluster_metadata(request)
|
||||
macros = metadata.get("firewall_macros")
|
||||
if isinstance(macros, list):
|
||||
return [dict(item) for item in macros if isinstance(item, dict)]
|
||||
firewall = await _load_firewall(request)
|
||||
nested = firewall.get("macros")
|
||||
if isinstance(nested, list):
|
||||
return [dict(item) for item in nested if isinstance(item, dict)]
|
||||
return []
|
||||
|
||||
async def groups_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = await _ready(request, inputs)
|
||||
firewall = await _load_firewall(request)
|
||||
groups = _scope_data(firewall, scope_fn(payload)).get("groups", {})
|
||||
section = _get_scope(firewall, scope_fn(payload))
|
||||
groups = section.get("groups", {}) if section else {}
|
||||
if not isinstance(groups, dict):
|
||||
return []
|
||||
return [
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
@@ -27,6 +28,13 @@ def _ha_groups(metadata: dict[str, Any]) -> dict[str, dict[str, Any]]:
|
||||
return {str(key): dict(value) for key, value in groups.items() if isinstance(value, dict)}
|
||||
|
||||
|
||||
def _ha_digest(payload: dict[str, Any]) -> str:
|
||||
material = {key: value for key, value in payload.items() if key != "digest"}
|
||||
return hashlib.sha1( # noqa: S324 - Proxmox HA digests use SHA-1
|
||||
json.dumps(material, sort_keys=True, separators=(",", ":")).encode()
|
||||
).hexdigest()
|
||||
|
||||
|
||||
def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
async def ha_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("groups", "resources", "rules", "status")
|
||||
@@ -41,8 +49,7 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
for row in rows:
|
||||
payload = state(row["state"])
|
||||
sid = str(row["external_id"])
|
||||
result.append(
|
||||
{
|
||||
entry = {
|
||||
"sid": sid,
|
||||
"type": "vm" if sid.startswith("vm:") else "ct",
|
||||
"state": payload.get("state", "started"),
|
||||
@@ -51,7 +58,8 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
"max_relocate": payload.get("max_relocate", 1),
|
||||
"max_restart": payload.get("max_restart", 1),
|
||||
}
|
||||
)
|
||||
entry["digest"] = _ha_digest(entry)
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
async def ha_resource_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -142,7 +150,7 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
async def ha_groups(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
configured = _ha_groups(metadata)
|
||||
result = [
|
||||
return [
|
||||
{
|
||||
"group": group_id,
|
||||
"nodes": str(payload.get("nodes", "")),
|
||||
@@ -153,26 +161,6 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
}
|
||||
for group_id, payload in sorted(configured.items())
|
||||
]
|
||||
if result:
|
||||
return result
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT DISTINCT state->>'group' AS group_id
|
||||
FROM resources WHERE kind='ha' AND state ? 'group'
|
||||
ORDER BY 1"""
|
||||
)
|
||||
node_names = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
nodes = ",".join(str(row["name"]) for row in node_names) or "pve01"
|
||||
return [
|
||||
{
|
||||
"group": str(row["group_id"]),
|
||||
"nodes": nodes,
|
||||
"nofailback": 0,
|
||||
"restricted": 0,
|
||||
"type": "group",
|
||||
}
|
||||
for row in rows
|
||||
if row["group_id"]
|
||||
]
|
||||
|
||||
async def ha_group_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
group = str(values(inputs)["group"])
|
||||
@@ -228,50 +216,97 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
async def ha_status(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("current", "manager_status")
|
||||
|
||||
async def ha_status_current(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
row = await database(request).pool.fetchrow(
|
||||
"""SELECT count(*) FILTER (WHERE state->>'state' = 'started') AS started,
|
||||
count(*) AS total
|
||||
FROM resources WHERE kind='ha'"""
|
||||
)
|
||||
async def ha_status_current(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
master = await database(request).pool.fetchval(
|
||||
"SELECT name FROM nodes WHERE status='online' ORDER BY name LIMIT 1"
|
||||
)
|
||||
metadata = await cluster_metadata(request)
|
||||
ha = metadata.get("ha", {}) if isinstance(metadata.get("ha"), dict) else {}
|
||||
return {
|
||||
"quorate": 1,
|
||||
"mode": "active" if ha.get("armed", True) else "disabled",
|
||||
"master_node": str(master or "pve01"),
|
||||
"ha_started": int(row["started"] or 0),
|
||||
"ha_total": int(row["total"] or 0),
|
||||
"armed": 1 if ha.get("armed", True) else 0,
|
||||
ha_raw = metadata.get("ha")
|
||||
ha: dict[str, Any] = dict(ha_raw) if isinstance(ha_raw, dict) else {}
|
||||
armed = bool(ha.get("armed")) if "armed" in ha else False
|
||||
status_raw = ha.get("status_current")
|
||||
status: dict[str, Any] = dict(status_raw) if isinstance(status_raw, dict) else {}
|
||||
quorate = bool(int(status.get("quorate", metadata.get("quorate", 1)) or 1))
|
||||
master_node = str(status.get("master_node") or master or "")
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT r.external_id, r.state, n.name AS node
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
WHERE r.kind='ha' ORDER BY r.external_id"""
|
||||
)
|
||||
nodes = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
|
||||
result: list[dict[str, Any]] = [
|
||||
{
|
||||
"id": "quorum",
|
||||
"type": "quorum",
|
||||
"node": master_node,
|
||||
"status": "OK" if quorate else "offline",
|
||||
"quorate": quorate,
|
||||
},
|
||||
{
|
||||
"id": "master",
|
||||
"type": "master",
|
||||
"node": master_node,
|
||||
"status": "active" if armed else "disabled",
|
||||
},
|
||||
{
|
||||
"id": "fencing",
|
||||
"type": "fencing",
|
||||
"node": master_node,
|
||||
"status": "armed" if armed else "disarmed",
|
||||
"armed-state": "armed" if armed else "disarmed",
|
||||
},
|
||||
]
|
||||
for node_row in nodes:
|
||||
node_name = str(node_row["name"])
|
||||
result.append(
|
||||
{
|
||||
"id": f"lrm:{node_name}",
|
||||
"type": "lrm",
|
||||
"node": node_name,
|
||||
"status": "active" if quorate else "offline",
|
||||
}
|
||||
)
|
||||
for row in rows:
|
||||
payload = state(row["state"])
|
||||
sid = str(row["external_id"])
|
||||
crm = str(payload.get("state") or "started")
|
||||
result.append(
|
||||
{
|
||||
"id": f"service:{sid}",
|
||||
"type": "service",
|
||||
"sid": sid,
|
||||
"node": str(row["node"]),
|
||||
"crm_state": crm,
|
||||
"state": crm,
|
||||
"status": crm,
|
||||
"request_state": str(payload.get("request_state") or crm),
|
||||
"max_relocate": int(payload.get("max_relocate") or 1),
|
||||
"max_restart": int(payload.get("max_restart") or 1),
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
async def ha_manager_status(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata = await cluster_metadata(request)
|
||||
ha = metadata.get("ha", {}) if isinstance(metadata.get("ha"), dict) else {}
|
||||
armed = bool(ha.get("armed", True))
|
||||
ha_raw = metadata.get("ha")
|
||||
ha: dict[str, Any] = dict(ha_raw) if isinstance(ha_raw, dict) else {}
|
||||
manager_raw = ha.get("manager_status")
|
||||
manager: dict[str, Any] = dict(manager_raw) if isinstance(manager_raw, dict) else {}
|
||||
armed = bool(ha.get("armed")) if "armed" in ha else False
|
||||
return {
|
||||
"manager_status": "active" if armed else "disabled",
|
||||
"quorum": "OK",
|
||||
"manager_status": manager.get("manager_status", "active" if armed else "disabled"),
|
||||
"quorum": manager.get("quorum", ""),
|
||||
"armed": 1 if armed else 0,
|
||||
}
|
||||
|
||||
async def ha_rules(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
rules = metadata.get("ha_rules")
|
||||
if isinstance(rules, list) and rules:
|
||||
if isinstance(rules, list):
|
||||
return [dict(item) for item in rules if isinstance(item, dict)]
|
||||
defaults = [
|
||||
{"rule": "node-fencing", "type": "node", "action": "restart"},
|
||||
{"rule": "service-ha", "type": "resource", "action": "failover"},
|
||||
]
|
||||
metadata["ha_rules"] = defaults
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return list(defaults)
|
||||
return []
|
||||
|
||||
async def ha_relocate(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def ha_relocate(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
sid = str(payload["sid"])
|
||||
target = str(payload.get("node") or payload.get("target") or "")
|
||||
@@ -311,9 +346,14 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
|
||||
ha_row["id"],
|
||||
json.dumps(current, sort_keys=True),
|
||||
)
|
||||
return {
|
||||
"sid": sid,
|
||||
"requested-node": target,
|
||||
"comigrated-resources": [],
|
||||
}
|
||||
|
||||
async def ha_migrate(request: Request, inputs: dict[str, Any]) -> None:
|
||||
await ha_relocate(request, inputs)
|
||||
async def ha_migrate(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
return await ha_relocate(request, inputs)
|
||||
|
||||
async def ha_arm(request: Request, _inputs: dict[str, Any]) -> None:
|
||||
metadata = await cluster_metadata(request)
|
||||
|
||||
@@ -19,7 +19,7 @@ def register_legacy_aliases(registry: HandlerRegistry) -> None:
|
||||
|
||||
alias("/access/tfa", "POST", "/access/tfa/{userid}", "POST")
|
||||
alias("/access/tfa", "PUT", "/access/tfa/{userid}/{id}", "PUT")
|
||||
alias("/cluster/backupinfo", "GET", "/cluster/backup-info", "GET")
|
||||
# /cluster/backupinfo is registered as a PVE 6 stub string in backup.py
|
||||
alias(
|
||||
"/cluster/backupinfo/not_backed_up",
|
||||
"GET",
|
||||
|
||||
@@ -38,27 +38,120 @@ def _state(value: object) -> dict[str, Any]:
|
||||
return dict(cast(Mapping[str, Any], value))
|
||||
|
||||
|
||||
# Runtime-only keys stored in ``resources.state``; never echo on GET /config.
|
||||
_LXC_CONFIG_INTERNAL = frozenset(
|
||||
{
|
||||
"agent",
|
||||
"cloudinit_dump",
|
||||
"config",
|
||||
"consoles",
|
||||
"interfaces",
|
||||
"migrate_preconditions",
|
||||
"name",
|
||||
"rrd",
|
||||
"rrddata",
|
||||
"status",
|
||||
"volume_moves",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _public_lxc_config(
|
||||
config: dict[str, Any], state: dict[str, Any], *, vmid: int
|
||||
) -> dict[str, Any]:
|
||||
import hashlib
|
||||
|
||||
merged = {**state, **config}
|
||||
payload: dict[str, Any] = {"vmid": vmid}
|
||||
for key, value in merged.items():
|
||||
if key in _LXC_CONFIG_INTERNAL:
|
||||
continue
|
||||
if isinstance(value, dict | list):
|
||||
continue
|
||||
payload[key] = value
|
||||
digest_src = {key: value for key, value in payload.items() if key != "digest"}
|
||||
payload["digest"] = hashlib.sha1( # noqa: S324 - Proxmox config digests use SHA-1
|
||||
json.dumps(digest_src, sort_keys=True, separators=(",", ":")).encode()
|
||||
).hexdigest()
|
||||
return payload
|
||||
|
||||
|
||||
def _lxc_index_item(vmid: int, vm_state: dict[str, Any], config: dict[str, Any]) -> dict[str, Any]:
|
||||
status = str(vm_state.get("status", "stopped"))
|
||||
running = status in {"running", "paused"}
|
||||
memory_mb = int(config.get("memory", config.get("mem", 512)))
|
||||
maxmem = memory_mb * 2**20
|
||||
return {
|
||||
"vmid": vmid,
|
||||
"name": str(config.get("hostname", config.get("name", f"ct-{vmid}"))),
|
||||
"status": status,
|
||||
"cpus": int(config.get("cores", config.get("cpus", 1))),
|
||||
"cpu": float(vm_state.get("cpu", 0.05 if running else 0.0)),
|
||||
"maxmem": maxmem,
|
||||
"mem": int(vm_state.get("mem", maxmem // 2 if running else 0)),
|
||||
"maxdisk": int(vm_state.get("maxdisk", config.get("maxdisk", 8 * 1024**3))),
|
||||
"disk": int(vm_state.get("disk", 0)),
|
||||
"diskread": int(vm_state.get("diskread", 0)),
|
||||
"diskwrite": int(vm_state.get("diskwrite", 0)),
|
||||
"netin": int(vm_state.get("netin", 0)),
|
||||
"netout": int(vm_state.get("netout", 0)),
|
||||
"maxswap": int(config.get("swap", 512)) * 2**20,
|
||||
"uptime": int(vm_state.get("uptime", 0)) if running else 0,
|
||||
"lock": str(vm_state.get("lock", "")),
|
||||
"tags": str(config.get("tags", "")),
|
||||
"template": int(bool(vm_state.get("template", False))),
|
||||
"pressurecpusome": float(vm_state.get("pressurecpusome", 0.0)),
|
||||
"pressurecpufull": float(vm_state.get("pressurecpufull", 0.0)),
|
||||
"pressureiosome": float(vm_state.get("pressureiosome", 0.0)),
|
||||
"pressureiofull": float(vm_state.get("pressureiofull", 0.0)),
|
||||
"pressurememorysome": float(vm_state.get("pressurememorysome", 0.0)),
|
||||
"pressurememoryfull": float(vm_state.get("pressurememoryfull", 0.0)),
|
||||
}
|
||||
|
||||
|
||||
def register_lxc_handlers(registry: HandlerRegistry) -> None:
|
||||
async def lxc_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(_values(inputs)["node"])
|
||||
rows = await _database(request).pool.fetch(
|
||||
"""SELECT r.external_id::integer AS vmid, r.state
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
"""SELECT r.external_id::integer AS vmid, r.state, c.config
|
||||
FROM resources r
|
||||
JOIN nodes n ON n.id=r.node_id
|
||||
JOIN containers c ON c.resource_id=r.id
|
||||
WHERE n.name=$1 AND r.kind='lxc' ORDER BY r.external_id::integer""",
|
||||
node,
|
||||
)
|
||||
return [{"vmid": int(row["vmid"]), **_state(row["state"])} for row in rows]
|
||||
return [
|
||||
_lxc_index_item(int(row["vmid"]), _state(row["state"]), _state(row["config"]))
|
||||
for row in rows
|
||||
]
|
||||
|
||||
async def lxc_config(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node, vmid = str(_values(inputs)["node"]), str(_values(inputs)["vmid"])
|
||||
row = await _lxc_resource(request, node, vmid)
|
||||
return {"vmid": int(vmid), **_state(row["config"]), **_state(row["state"])}
|
||||
return _public_lxc_config(_state(row["config"]), _state(row["state"]), vmid=int(vmid))
|
||||
|
||||
async def lxc_current(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
return await lxc_config(request, inputs)
|
||||
payload = _values(inputs)
|
||||
node, vmid = str(payload["node"]), int(payload["vmid"])
|
||||
row = await _lxc_resource(request, node, str(vmid))
|
||||
vm_state = _state(row["state"])
|
||||
config = _state(row["config"])
|
||||
item = _lxc_index_item(vmid, vm_state, config)
|
||||
item["ha"] = {"managed": int(vm_state.get("ha_managed", 0))}
|
||||
return item
|
||||
|
||||
async def lxc_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
return await lxc_config(request, inputs)
|
||||
async def lxc_status(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
payload = _values(inputs)
|
||||
await _lxc_resource(request, str(payload["node"]), str(payload["vmid"]))
|
||||
return subdirs(
|
||||
"current",
|
||||
"reboot",
|
||||
"resume",
|
||||
"shutdown",
|
||||
"start",
|
||||
"stop",
|
||||
"suspend",
|
||||
)
|
||||
|
||||
async def mutate(operation: str, request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
@@ -323,7 +416,7 @@ def register_lxc_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def migrate_preconditions(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
values = _values(inputs)
|
||||
await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
target = values.get("target")
|
||||
if target in {None, ""}:
|
||||
raise ApiError(400, "parameter 'target' is required")
|
||||
@@ -333,7 +426,13 @@ def register_lxc_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if not exists:
|
||||
raise ApiError(404, "target node does not exist")
|
||||
return {"local_disks": [], "local_resources": [], "running": False}
|
||||
state = _state(resource["state"])
|
||||
pre = state.get("migrate_preconditions")
|
||||
payload = dict(pre) if isinstance(pre, dict) else {}
|
||||
payload["running"] = str(state.get("status") or "") == "running"
|
||||
payload.setdefault("local_disks", [])
|
||||
payload.setdefault("local_resources", [])
|
||||
return payload
|
||||
|
||||
async def migrate(request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
@@ -415,7 +514,7 @@ def register_lxc_handlers(registry: HandlerRegistry) -> None:
|
||||
}
|
||||
}
|
||||
|
||||
async def lxc_resize(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def lxc_resize(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = _values(inputs)
|
||||
node, vmid = str(payload["node"]), str(payload["vmid"])
|
||||
disk = str(payload.get("disk") or "rootfs")
|
||||
@@ -440,6 +539,13 @@ def register_lxc_handlers(registry: HandlerRegistry) -> None:
|
||||
resource["id"],
|
||||
json.dumps({disk: config[disk]}, sort_keys=True),
|
||||
)
|
||||
return await _create_task(
|
||||
request,
|
||||
node=node,
|
||||
vmid=vmid,
|
||||
task_type="lxc-resize",
|
||||
payload={"resource_id": str(resource["id"]), "disk": disk, "size": size},
|
||||
)
|
||||
|
||||
async def lxc_template(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = _values(inputs)
|
||||
@@ -531,6 +637,8 @@ async def _create_task(
|
||||
"lxc-clone": "pctclone",
|
||||
"lxc-migrate": "pctmigrate",
|
||||
"lxc-remote-migrate": "pctremote",
|
||||
"lxc-resize": "pctresize",
|
||||
"lxc-move-volume": "pctmove",
|
||||
}[task_type]
|
||||
upid = str(Upid.allocate(node, worker_type, vmid, str(request.state.principal)))
|
||||
try:
|
||||
|
||||
@@ -39,16 +39,15 @@ def register_lxc_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
values = _values(inputs)
|
||||
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
state = _state(resource["state"])
|
||||
ifaces = state.setdefault(
|
||||
"interfaces",
|
||||
[{"name": "eth0", "hwaddr": "02:00:00:00:00:11", "inet": "192.0.2.20/24"}],
|
||||
)
|
||||
await _save_state(request, resource["id"], state)
|
||||
return list(ifaces) if isinstance(ifaces, list) else []
|
||||
ifaces = state.get("interfaces")
|
||||
return [dict(item) for item in ifaces] if isinstance(ifaces, list) else []
|
||||
|
||||
async def move_volume(request: Request, inputs: dict[str, Any]) -> str:
|
||||
from app.handlers.lxc import _create_task
|
||||
|
||||
values = _values(inputs)
|
||||
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
node, vmid = str(values["node"]), str(values["vmid"])
|
||||
resource = await _lxc_resource(request, node, vmid)
|
||||
volume = str(values.get("volume") or values.get("disk") or "rootfs")
|
||||
storage = str(values.get("storage") or "local-lvm")
|
||||
config = _state(resource["config"])
|
||||
@@ -64,29 +63,31 @@ def register_lxc_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
moves = state["volume_moves"] = []
|
||||
moves.append({"volume": volume, "storage": storage})
|
||||
await _save_state(request, resource["id"], state)
|
||||
return f"UPID:{values['node']}:lxc-move-volume:{values['vmid']}"
|
||||
return await _create_task(
|
||||
request,
|
||||
node=node,
|
||||
vmid=vmid,
|
||||
task_type="lxc-move-volume",
|
||||
payload={
|
||||
"resource_id": str(resource["id"]),
|
||||
"volume": volume,
|
||||
"storage": storage,
|
||||
},
|
||||
)
|
||||
|
||||
async def rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
values = _values(inputs)
|
||||
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
state = _state(resource["state"])
|
||||
rrd_state = state.setdefault("rrd", {"filename": f"pve-ct-{values['vmid']}.rrd"})
|
||||
await _save_state(request, resource["id"], state)
|
||||
return dict(rrd_state)
|
||||
rrd_state = state.get("rrd")
|
||||
return dict(rrd_state) if isinstance(rrd_state, dict) else {}
|
||||
|
||||
async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
values = _values(inputs)
|
||||
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
state = _state(resource["state"])
|
||||
series = state.setdefault(
|
||||
"rrddata",
|
||||
[
|
||||
{"time": 1_700_000_000, "cpu": 0.02, "mem": 64 * 1024 * 1024},
|
||||
{"time": 1_700_000_060, "cpu": 0.03, "mem": 66 * 1024 * 1024},
|
||||
],
|
||||
)
|
||||
await _save_state(request, resource["id"], state)
|
||||
return list(series)
|
||||
series = state.get("rrddata")
|
||||
return [dict(item) for item in series] if isinstance(series, list) else []
|
||||
|
||||
async def _console(request: Request, inputs: dict[str, Any], kind: str) -> dict[str, Any]:
|
||||
values = _values(inputs)
|
||||
|
||||
@@ -12,13 +12,14 @@ from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs
|
||||
|
||||
|
||||
def _mappings(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
current = metadata.setdefault("mapping", {"dir": {}, "pci": {}, "usb": {}})
|
||||
current = metadata.get("mapping")
|
||||
if not isinstance(current, dict):
|
||||
current = {"dir": {}, "pci": {}, "usb": {}}
|
||||
metadata["mapping"] = current
|
||||
for kind in ("dir", "pci", "usb"):
|
||||
current.setdefault(kind, {})
|
||||
return current
|
||||
return {"dir": {}, "pci": {}, "usb": {}}
|
||||
return {
|
||||
"dir": dict(current["dir"]) if isinstance(current.get("dir"), dict) else {},
|
||||
"pci": dict(current["pci"]) if isinstance(current.get("pci"), dict) else {},
|
||||
"usb": dict(current["usb"]) if isinstance(current.get("usb"), dict) else {},
|
||||
}
|
||||
|
||||
|
||||
def register_mapping_handlers(registry: HandlerRegistry) -> None:
|
||||
@@ -42,12 +43,14 @@ def register_mapping_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
item_id = str(payload["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _mappings(metadata)[kind]
|
||||
mapping = _mappings(metadata)
|
||||
store = mapping[kind]
|
||||
if item_id in store:
|
||||
raise ApiError(400, f"{kind} mapping '{item_id}' already exists")
|
||||
store[item_id] = {
|
||||
key: value for key, value in payload.items() if key not in {"delete", "digest"}
|
||||
}
|
||||
metadata["mapping"] = mapping
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -62,7 +65,8 @@ def register_mapping_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
item_id = str(payload["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _mappings(metadata)[kind]
|
||||
mapping = _mappings(metadata)
|
||||
store = mapping[kind]
|
||||
if item_id not in store:
|
||||
raise ApiError(404, f"{kind} mapping does not exist")
|
||||
current = dict(store[item_id])
|
||||
@@ -76,15 +80,18 @@ def register_mapping_handlers(registry: HandlerRegistry) -> None:
|
||||
current[key] = value
|
||||
current["id"] = item_id
|
||||
store[item_id] = current
|
||||
metadata["mapping"] = mapping
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
item_id = str(values(inputs)["id"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _mappings(metadata)[kind]
|
||||
mapping = _mappings(metadata)
|
||||
store = mapping[kind]
|
||||
if item_id not in store:
|
||||
raise ApiError(404, f"{kind} mapping does not exist")
|
||||
del store[item_id]
|
||||
metadata["mapping"] = mapping
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
registry.register(base, "GET", list_items)
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
from typing import Any, cast
|
||||
|
||||
from fastapi import Request
|
||||
@@ -20,92 +19,13 @@ from app.handlers.common import (
|
||||
from app.tasks.repository import TaskRepository
|
||||
from app.tasks.upid import Upid
|
||||
|
||||
DEFAULT_NODE_OPS: dict[str, Any] = {
|
||||
"network": [
|
||||
{
|
||||
"iface": "vmbr0",
|
||||
"type": "bridge",
|
||||
"active": 1,
|
||||
"method": "static",
|
||||
"address": "10.0.0.10/24",
|
||||
},
|
||||
{
|
||||
"iface": "vmbr1",
|
||||
"type": "bridge",
|
||||
"active": 1,
|
||||
"method": "static",
|
||||
"address": "10.10.0.10/24",
|
||||
},
|
||||
{"iface": "eno1", "type": "eth", "active": 1, "method": "manual"},
|
||||
],
|
||||
"disks": {
|
||||
"list": [
|
||||
{
|
||||
"devpath": "/dev/sda",
|
||||
"size": 1_000_000_000_000,
|
||||
"model": "SIM-DISK-01",
|
||||
"serial": "SIM0001",
|
||||
"gpt": 1,
|
||||
},
|
||||
{
|
||||
"devpath": "/dev/sdb",
|
||||
"size": 2_000_000_000_000,
|
||||
"model": "SIM-SSD-01",
|
||||
"serial": "SIM0002",
|
||||
"gpt": 0,
|
||||
},
|
||||
],
|
||||
"directory": [],
|
||||
"lvm": [],
|
||||
"lvmthin": [],
|
||||
"zfs": [],
|
||||
"smart": {},
|
||||
},
|
||||
"services": {
|
||||
"pveproxy": {"state": "running", "enabled": 1},
|
||||
"pvedaemon": {"state": "running", "enabled": 1},
|
||||
"pvestatd": {"state": "running", "enabled": 1},
|
||||
"corosync": {"state": "running", "enabled": 1},
|
||||
},
|
||||
"apt": {
|
||||
"packages": [
|
||||
{
|
||||
"Package": "pve-manager",
|
||||
"Version": "9.2.3",
|
||||
"OldVersion": "9.2.2",
|
||||
"Status": "upgradable",
|
||||
},
|
||||
{"Package": "libpve-common-perl", "Version": "9.0.3", "Status": "installed"},
|
||||
],
|
||||
"repositories": [
|
||||
{
|
||||
"path": "/etc/apt/sources.list.d/pve-enterprise.list",
|
||||
"enabled": 1,
|
||||
"types": "deb",
|
||||
"uri": "http://download.proxmox.com/debian/pve",
|
||||
"suites": "bookworm",
|
||||
"components": "pve-no-subscription",
|
||||
}
|
||||
],
|
||||
"update": {"status": "stopped", "exitstatus": "OK"},
|
||||
"changelogs": {},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def default_node_ops() -> dict[str, Any]:
|
||||
return copy.deepcopy(DEFAULT_NODE_OPS)
|
||||
|
||||
|
||||
async def load_node_ops(request: Request, node: str) -> dict[str, Any]:
|
||||
metadata = await node_metadata(request, node)
|
||||
ops = metadata.get("ops")
|
||||
if isinstance(ops, dict) and ops:
|
||||
return ops
|
||||
ops = default_node_ops()
|
||||
metadata["ops"] = ops
|
||||
await save_node_metadata(request, node, metadata)
|
||||
if isinstance(ops, dict):
|
||||
return ops
|
||||
return {}
|
||||
|
||||
|
||||
async def save_node_ops(request: Request, node: str, ops: dict[str, Any]) -> None:
|
||||
@@ -123,31 +43,76 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
node = str(values(inputs)["node"])
|
||||
ops = await load_node_ops(request, node)
|
||||
packages = ops.get("apt", {}).get("packages", [])
|
||||
return list(packages) if isinstance(packages, list) else []
|
||||
if not isinstance(packages, list):
|
||||
return []
|
||||
result: list[dict[str, Any]] = []
|
||||
for item in packages:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
entry = dict(item)
|
||||
status = str(entry.pop("Status", entry.get("CurrentState", "Installed")))
|
||||
if status.lower() == "upgradable":
|
||||
entry["CurrentState"] = "Installed"
|
||||
entry.setdefault("OldVersion", entry.get("Version", ""))
|
||||
else:
|
||||
entry["CurrentState"] = "Installed" if status.lower() == "installed" else status
|
||||
entry.setdefault("Package", entry.get("Package") or "unknown")
|
||||
entry.setdefault("Version", entry.get("Version") or "")
|
||||
entry.setdefault("Arch", "amd64")
|
||||
entry.setdefault("Origin", "Proxmox")
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
async def apt_repositories(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
async def apt_repositories(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node = str(values(inputs)["node"])
|
||||
ops = await load_node_ops(request, node)
|
||||
repositories = ops.get("apt", {}).get("repositories", [])
|
||||
return list(repositories) if isinstance(repositories, list) else []
|
||||
repositories = ops.get("apt", {}).get("repositories")
|
||||
if isinstance(repositories, dict) and "files" in repositories:
|
||||
return {
|
||||
"digest": str(repositories.get("digest") or "seedapt"),
|
||||
"files": list(repositories.get("files") or []),
|
||||
"errors": list(repositories.get("errors") or []),
|
||||
"infos": list(repositories.get("infos") or []),
|
||||
"standard-repos": list(repositories.get("standard-repos") or []),
|
||||
}
|
||||
legacy = list(repositories) if isinstance(repositories, list) else []
|
||||
return {
|
||||
"digest": "seedapt",
|
||||
"files": [
|
||||
{
|
||||
"path": "/etc/apt/sources.list.d/pve-enterprise.list",
|
||||
"file-type": "list",
|
||||
"digest": "seedapt-file0",
|
||||
"repositories": legacy,
|
||||
}
|
||||
],
|
||||
"errors": [],
|
||||
"infos": [],
|
||||
"standard-repos": [
|
||||
{
|
||||
"handle": "no-subscription",
|
||||
"name": "Proxmox VE No-Subscription Repository",
|
||||
"status": 1,
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
async def apt_changelog(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
name = str(values(inputs).get("name") or "pve-manager")
|
||||
ops = await load_node_ops(request, node)
|
||||
changelogs = ops.setdefault("apt", {}).setdefault("changelogs", {})
|
||||
if name not in changelogs:
|
||||
changelogs[name] = f"simulated changelog for {name}\n\n * emulator build\n"
|
||||
await save_node_ops(request, node, ops)
|
||||
return str(changelogs[name])
|
||||
apt = ops.get("apt")
|
||||
changelogs = apt.get("changelogs") if isinstance(apt, dict) else None
|
||||
if not isinstance(changelogs, dict):
|
||||
return ""
|
||||
return str(changelogs.get(name) or "")
|
||||
|
||||
async def apt_update_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node = str(values(inputs)["node"])
|
||||
ops = await load_node_ops(request, node)
|
||||
update = ops.get("apt", {}).get("update", {"status": "stopped", "exitstatus": "OK"})
|
||||
if isinstance(update, dict):
|
||||
return dict(update)
|
||||
return {"status": "stopped", "exitstatus": "OK"}
|
||||
apt = ops.get("apt")
|
||||
update = apt.get("update") if isinstance(apt, dict) else None
|
||||
return dict(update) if isinstance(update, dict) else {}
|
||||
|
||||
async def apt_update_start(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
@@ -170,7 +135,7 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
return item
|
||||
raise ApiError(404, "interface does not exist")
|
||||
|
||||
async def network_mutate(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def network_mutate(request: Request, inputs: dict[str, Any]) -> str | None:
|
||||
node = str(values(inputs)["node"])
|
||||
payload = values(inputs)
|
||||
ops = await load_node_ops(request, node)
|
||||
@@ -221,8 +186,14 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
raise ApiError(404, "interface does not exist")
|
||||
ops["network"] = updated
|
||||
else:
|
||||
# PUT /nodes/{node}/network without iface → reload (returns UPID string).
|
||||
ops["network_applied"] = True
|
||||
await save_node_ops(request, node, ops)
|
||||
return await _node_task(
|
||||
request, node=node, task_type="network-reload", worker="srvreload"
|
||||
)
|
||||
await save_node_ops(request, node, ops)
|
||||
return None
|
||||
|
||||
async def disks_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -230,11 +201,9 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def _disks(request: Request, node: str) -> dict[str, Any]:
|
||||
ops = await load_node_ops(request, node)
|
||||
disks = ops.setdefault("disks", default_node_ops()["disks"])
|
||||
disks = ops.get("disks")
|
||||
if not isinstance(disks, dict):
|
||||
disks = default_node_ops()["disks"]
|
||||
ops["disks"] = disks
|
||||
await save_node_ops(request, node, ops)
|
||||
return {}
|
||||
return cast(dict[str, Any], disks)
|
||||
|
||||
async def disks_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
@@ -247,17 +216,9 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
node = str(values(inputs)["node"])
|
||||
disk = str(values(inputs).get("disk") or "/dev/sda")
|
||||
disks = await _disks(request, node)
|
||||
smart = disks.setdefault("smart", {})
|
||||
if disk not in smart:
|
||||
smart[disk] = {
|
||||
"health": "PASSED",
|
||||
"type": "scsi",
|
||||
"model": "SIM-DISK",
|
||||
"serial": disk.rsplit("/", 1)[-1],
|
||||
}
|
||||
ops = await load_node_ops(request, node)
|
||||
ops["disks"] = disks
|
||||
await save_node_ops(request, node, ops)
|
||||
smart = disks.get("smart")
|
||||
if not isinstance(smart, dict) or disk not in smart:
|
||||
return {}
|
||||
return dict(smart[disk])
|
||||
|
||||
async def disks_collection(
|
||||
@@ -280,13 +241,16 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
async def disks_zfs(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return await disks_collection(request, inputs, "zfs")
|
||||
|
||||
async def disks_initgpt(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def disks_initgpt(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
disk = str(values(inputs).get("disk") or values(inputs).get("device") or "")
|
||||
if not disk:
|
||||
raise ApiError(400, "parameter verification failed - 'disk' missing")
|
||||
ops = await load_node_ops(request, node)
|
||||
disks = ops.setdefault("disks", default_node_ops()["disks"])
|
||||
disks = ops.get("disks")
|
||||
if not isinstance(disks, dict):
|
||||
disks = {}
|
||||
ops["disks"] = disks
|
||||
items = list(disks.get("list") or [])
|
||||
found = False
|
||||
for item in items:
|
||||
@@ -307,14 +271,17 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
disks["list"] = items
|
||||
ops["disks"] = disks
|
||||
await save_node_ops(request, node, ops)
|
||||
return await _node_task(request, node=node, task_type="disk-initgpt", worker="diskinit")
|
||||
|
||||
async def disks_wipedisk(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def disks_wipedisk(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
disk = str(values(inputs).get("disk") or values(inputs).get("device") or "")
|
||||
if not disk:
|
||||
raise ApiError(400, "parameter verification failed - 'disk' missing")
|
||||
ops = await load_node_ops(request, node)
|
||||
disks = ops.setdefault("disks", default_node_ops()["disks"])
|
||||
disks = ops.get("disks")
|
||||
if not isinstance(disks, dict):
|
||||
raise ApiError(404, "disk does not exist")
|
||||
items = list(disks.get("list") or [])
|
||||
for item in items:
|
||||
if item.get("devpath") == disk:
|
||||
@@ -324,18 +291,35 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
else:
|
||||
raise ApiError(404, "disk does not exist")
|
||||
disks["list"] = items
|
||||
smart = disks.setdefault("smart", {})
|
||||
smart = disks.get("smart")
|
||||
if isinstance(smart, dict):
|
||||
smart.pop(disk, None)
|
||||
ops["disks"] = disks
|
||||
await save_node_ops(request, node, ops)
|
||||
return await _node_task(request, node=node, task_type="disk-wipe", worker="diskwipe")
|
||||
|
||||
async def services_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
async def services_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(values(inputs)["node"])
|
||||
ops = await load_node_ops(request, node)
|
||||
services = ops.get("services") or {}
|
||||
return [{"subdir": name} for name in sorted(services)]
|
||||
result: list[dict[str, Any]] = []
|
||||
for name in sorted(services):
|
||||
payload = dict(services[name]) if isinstance(services[name], dict) else {}
|
||||
state_name = str(payload.get("state", "stopped"))
|
||||
enabled = bool(int(payload.get("enabled", 0) or 0))
|
||||
result.append(
|
||||
{
|
||||
"service": name,
|
||||
"name": f"{name}.service",
|
||||
"desc": str(payload.get("desc") or name),
|
||||
"state": state_name,
|
||||
"active-state": "active" if state_name == "running" else "inactive",
|
||||
"unit-state": "enabled" if enabled else "disabled",
|
||||
}
|
||||
)
|
||||
return result
|
||||
|
||||
async def service_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
async def service_get(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
node = str(values(inputs)["node"])
|
||||
service = str(values(inputs)["service"])
|
||||
ops = await load_node_ops(request, node)
|
||||
@@ -343,12 +327,25 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
if service not in services:
|
||||
services[service] = {"state": "stopped", "enabled": 0}
|
||||
await save_node_ops(request, node, ops)
|
||||
payload = dict(services[service])
|
||||
payload["service"] = service
|
||||
return payload
|
||||
return subdirs("state", "start", "stop", "restart", "reload")
|
||||
|
||||
async def service_state(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
return await service_get(request, inputs)
|
||||
node = str(values(inputs)["node"])
|
||||
service = str(values(inputs)["service"])
|
||||
ops = await load_node_ops(request, node)
|
||||
services = ops.setdefault("services", {})
|
||||
if service not in services:
|
||||
services[service] = {"state": "stopped", "enabled": 0}
|
||||
await save_node_ops(request, node, ops)
|
||||
payload = dict(services[service]) if isinstance(services[service], dict) else {}
|
||||
state_name = str(payload.get("state", "stopped"))
|
||||
enabled = bool(int(payload.get("enabled", 0) or 0))
|
||||
return {
|
||||
"service": service,
|
||||
"state": state_name,
|
||||
"active-state": "active" if state_name == "running" else "inactive",
|
||||
"unit-state": "enabled" if enabled else "disabled",
|
||||
}
|
||||
|
||||
async def service_action(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
@@ -371,7 +368,12 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
services[service] = current
|
||||
ops["services"] = services
|
||||
await save_node_ops(request, node, ops)
|
||||
return "OK"
|
||||
return await _node_task(
|
||||
request,
|
||||
node=node,
|
||||
task_type=f"service-{action}",
|
||||
worker=f"srv{action}",
|
||||
)
|
||||
|
||||
registry.register("/nodes/{node}/apt", "GET", apt_index)
|
||||
registry.register("/nodes/{node}/apt/versions", "GET", apt_versions)
|
||||
@@ -404,6 +406,8 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
|
||||
async def _node_task(request: Request, *, node: str, task_type: str, worker: str) -> str:
|
||||
import secrets
|
||||
|
||||
from app.api.errors import ApiError
|
||||
from app.db.primitives import ConflictError
|
||||
|
||||
@@ -414,7 +418,7 @@ async def _node_task(request: Request, *, node: str, task_type: str, worker: str
|
||||
upid=upid,
|
||||
task_type=task_type,
|
||||
payload={"node": node},
|
||||
resource_key=f"node:{node}",
|
||||
resource_key=f"node:{node}:{task_type}:{secrets.token_hex(4)}",
|
||||
)
|
||||
except ConflictError as error:
|
||||
raise ApiError(409, str(error)) from error
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import secrets
|
||||
import time
|
||||
@@ -13,128 +12,49 @@ from fastapi import Request
|
||||
from app.api.errors import ApiError
|
||||
from app.api.registry import HandlerRegistry
|
||||
from app.handlers.common import database, require_node, subdirs, values
|
||||
from app.handlers.nodes import default_node_ops, load_node_ops, save_node_ops
|
||||
from app.handlers.nodes import load_node_ops, save_node_ops
|
||||
from app.tasks.repository import TaskRepository
|
||||
from app.tasks.upid import Upid
|
||||
|
||||
DEFAULT_HARDWARE: dict[str, Any] = {
|
||||
"pci": [
|
||||
{
|
||||
"id": "0000:00:1f.2",
|
||||
"vendor_name": "Intel Corporation",
|
||||
"device_name": "SATA Controller",
|
||||
"iommugroup": 0,
|
||||
},
|
||||
{
|
||||
"id": "0000:01:00.0",
|
||||
"vendor_name": "NVIDIA Corporation",
|
||||
"device_name": "GP102 [GeForce GTX 1080 Ti]",
|
||||
"iommugroup": 1,
|
||||
"mdev": 1,
|
||||
},
|
||||
],
|
||||
"usb": [
|
||||
{"busnum": 1, "devnum": 1, "level": 0, "port": "1", "prodid": "0002", "vendid": "1d6b"},
|
||||
{"busnum": 2, "devnum": 2, "level": 1, "port": "2", "prodid": "5591", "vendid": "0781"},
|
||||
],
|
||||
"mdev": {
|
||||
"0000:01:00.0": [
|
||||
{"type": "nvidia-11", "available": 4, "description": "GRID profile"},
|
||||
]
|
||||
},
|
||||
}
|
||||
|
||||
DEFAULT_SCAN: dict[str, list[dict[str, Any]]] = {
|
||||
"cifs": [{"server": "files.local", "share": "backups"}],
|
||||
"iscsi": [{"portal": "10.0.0.50:3260", "target": "iqn.2024-01.local:storage"}],
|
||||
"lvm": [{"vg": "pve", "size": 500_000_000_000, "free": 100_000_000_000}],
|
||||
"lvmthin": [{"lv": "data", "vg": "pve", "lv_size": 400_000_000_000}],
|
||||
"nfs": [{"server": "nfs.local", "path": "/export/pve", "options": "vers=4"}],
|
||||
"pbs": [{"server": "pbs.local", "datastore": "store1"}],
|
||||
"zfs": [{"pool": "rpool", "name": "rpool/data", "size": 800_000_000_000}],
|
||||
}
|
||||
|
||||
DEFAULT_SUBSCRIPTION: dict[str, Any] = {
|
||||
"status": "notfound",
|
||||
"message": "There is no subscription key",
|
||||
"serverid": "SIMULATOR",
|
||||
"sockets": 1,
|
||||
"productname": "Proxmox VE",
|
||||
"url": "https://www.proxmox.com/en/proxmox-virtual-environment/pricing",
|
||||
}
|
||||
|
||||
DEFAULT_CONFIG: dict[str, Any] = {
|
||||
"description": "Simulator node",
|
||||
"startall-onboot-delay": 0,
|
||||
"wakeonlan": "",
|
||||
}
|
||||
|
||||
DEFAULT_DNS: dict[str, Any] = {
|
||||
"search": "local",
|
||||
"dns1": "1.1.1.1",
|
||||
"dns2": "8.8.8.8",
|
||||
"dns3": "",
|
||||
}
|
||||
|
||||
DEFAULT_TIME: dict[str, Any] = {
|
||||
"timezone": "UTC",
|
||||
"time": 0,
|
||||
"localtime": 0,
|
||||
}
|
||||
|
||||
|
||||
def _certificates(ops: dict[str, Any]) -> dict[str, Any]:
|
||||
certs = ops.setdefault(
|
||||
"certificates",
|
||||
{
|
||||
"custom": None,
|
||||
"acme": {"account": "default", "domains": [], "certificate": None},
|
||||
"info": [],
|
||||
},
|
||||
)
|
||||
certs = ops.get("certificates")
|
||||
if not isinstance(certs, dict):
|
||||
certs = {"custom": None, "acme": {}, "info": []}
|
||||
ops["certificates"] = certs
|
||||
certs.setdefault("acme", {"account": "default", "domains": [], "certificate": None})
|
||||
certs.setdefault("info", [])
|
||||
return {"custom": None, "acme": {}, "info": []}
|
||||
return certs
|
||||
|
||||
|
||||
def _hardware(ops: dict[str, Any]) -> dict[str, Any]:
|
||||
hardware = ops.get("hardware")
|
||||
if not isinstance(hardware, dict) or not hardware:
|
||||
hardware = copy.deepcopy(DEFAULT_HARDWARE)
|
||||
ops["hardware"] = hardware
|
||||
hardware.setdefault("pci", copy.deepcopy(DEFAULT_HARDWARE["pci"]))
|
||||
hardware.setdefault("usb", copy.deepcopy(DEFAULT_HARDWARE["usb"]))
|
||||
hardware.setdefault("mdev", copy.deepcopy(DEFAULT_HARDWARE["mdev"]))
|
||||
return hardware
|
||||
if not isinstance(hardware, dict):
|
||||
return {"pci": [], "usb": [], "mdev": {}}
|
||||
return {
|
||||
"pci": list(hardware.get("pci") or []) if isinstance(hardware.get("pci"), list) else [],
|
||||
"usb": list(hardware.get("usb") or []) if isinstance(hardware.get("usb"), list) else [],
|
||||
"mdev": dict(hardware.get("mdev") or {}) if isinstance(hardware.get("mdev"), dict) else {},
|
||||
}
|
||||
|
||||
|
||||
def _scan_cache(ops: dict[str, Any]) -> dict[str, Any]:
|
||||
scan = ops.get("scan")
|
||||
if not isinstance(scan, dict) or not scan:
|
||||
scan = copy.deepcopy(DEFAULT_SCAN)
|
||||
ops["scan"] = scan
|
||||
for key, value in DEFAULT_SCAN.items():
|
||||
scan.setdefault(key, copy.deepcopy(value))
|
||||
if not isinstance(scan, dict):
|
||||
return {}
|
||||
return scan
|
||||
|
||||
|
||||
def _subscription(ops: dict[str, Any]) -> dict[str, Any]:
|
||||
subscription = ops.get("subscription")
|
||||
if not isinstance(subscription, dict) or not subscription:
|
||||
subscription = copy.deepcopy(DEFAULT_SUBSCRIPTION)
|
||||
ops["subscription"] = subscription
|
||||
if not isinstance(subscription, dict):
|
||||
return {}
|
||||
return subscription
|
||||
|
||||
|
||||
def _disk_items(ops: dict[str, Any], kind: str) -> list[dict[str, Any]]:
|
||||
disks = ops.setdefault("disks", default_node_ops()["disks"])
|
||||
disks = ops.get("disks")
|
||||
if not isinstance(disks, dict):
|
||||
disks = default_node_ops()["disks"]
|
||||
disks = {}
|
||||
ops["disks"] = disks
|
||||
items = disks.setdefault(kind, [])
|
||||
items = disks.get(kind)
|
||||
if not isinstance(items, list):
|
||||
items = []
|
||||
disks[kind] = items
|
||||
@@ -188,7 +108,9 @@ async def _migrate_guests(request: Request, node: str, target: str) -> None:
|
||||
|
||||
|
||||
def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def disks_create(request: Request, inputs: dict[str, Any], kind: str) -> dict[str, Any]:
|
||||
async def disks_create(request: Request, inputs: dict[str, Any], kind: str) -> str:
|
||||
from app.handlers.nodes import _node_task
|
||||
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
await require_node(request, node)
|
||||
@@ -208,11 +130,19 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
}
|
||||
entry["name"] = name
|
||||
items.append(entry)
|
||||
ops.setdefault("disks", default_node_ops()["disks"])[kind] = items
|
||||
disks = ops.get("disks")
|
||||
if not isinstance(disks, dict):
|
||||
disks = {}
|
||||
ops["disks"] = disks
|
||||
disks[kind] = items
|
||||
await save_node_ops(request, node, ops)
|
||||
return entry
|
||||
return await _node_task(
|
||||
request, node=node, task_type=f"disk-{kind}-create", worker=f"disk{kind}"
|
||||
)
|
||||
|
||||
async def disks_delete(request: Request, inputs: dict[str, Any], kind: str) -> str:
|
||||
from app.handlers.nodes import _node_task
|
||||
|
||||
async def disks_delete(request: Request, inputs: dict[str, Any], kind: str) -> None:
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
name = str(payload["name"])
|
||||
@@ -224,6 +154,9 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
raise ApiError(404, f"{kind} does not exist")
|
||||
ops.setdefault("disks", {})[kind] = remaining
|
||||
await save_node_ops(request, node, ops)
|
||||
return await _node_task(
|
||||
request, node=node, task_type=f"disk-{kind}-delete", worker=f"diskdel{kind}"
|
||||
)
|
||||
|
||||
async def disks_zfs_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
@@ -270,11 +203,11 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
certs["acme"] = acme
|
||||
ops["certificates"] = certs
|
||||
await save_node_ops(request, node, ops)
|
||||
if method == "DELETE":
|
||||
return None
|
||||
return await _node_task(request, node=node, task_type="acme", worker="acme")
|
||||
|
||||
async def certificates_custom(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def certificates_custom(
|
||||
request: Request, inputs: dict[str, Any]
|
||||
) -> dict[str, Any] | None:
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
await require_node(request, node)
|
||||
@@ -282,23 +215,18 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
certs = _certificates(ops)
|
||||
if request.method.upper() == "DELETE":
|
||||
certs["custom"] = None
|
||||
else:
|
||||
ops["certificates"] = certs
|
||||
await save_node_ops(request, node, ops)
|
||||
return None
|
||||
certificates = str(payload.get("certificates") or payload.get("cert") or "")
|
||||
if not certificates:
|
||||
raise ApiError(400, "parameter verification failed - 'certificates' missing")
|
||||
key = str(payload.get("key") or payload.get("private-key") or "")
|
||||
certs["custom"] = {
|
||||
"certificates": certificates,
|
||||
"key": key,
|
||||
"restart": int(payload.get("restart") or 0),
|
||||
"filename": str(payload.get("filename") or "pveproxy-ssl.pem"),
|
||||
}
|
||||
info = list(certs.get("info") or [])
|
||||
info = [item for item in info if item.get("filename") != certs["custom"]["filename"]]
|
||||
info.append(
|
||||
{
|
||||
"filename": certs["custom"]["filename"],
|
||||
"fingerprint": secrets.token_hex(20),
|
||||
filename = str(payload.get("filename") or "pveproxy-ssl.pem")
|
||||
fingerprint = ":".join(secrets.token_hex(1) for _ in range(32)) # SHA-256 fingerprint style
|
||||
summary = {
|
||||
"filename": filename,
|
||||
"fingerprint": fingerprint,
|
||||
"issuer": "CN=Simulator",
|
||||
"subject": "CN=pve.local",
|
||||
"notbefore": int(time.time()) - 86_400,
|
||||
@@ -306,11 +234,21 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
"san": ["DNS:pve.local"],
|
||||
"public-key-type": "rsa",
|
||||
"public-key-bits": 2048,
|
||||
"pem": certificates,
|
||||
}
|
||||
)
|
||||
certs["custom"] = {
|
||||
"certificates": certificates,
|
||||
"key": key,
|
||||
"restart": int(payload.get("restart") or 0),
|
||||
"filename": filename,
|
||||
}
|
||||
info = list(certs.get("info") or [])
|
||||
info = [item for item in info if item.get("filename") != filename]
|
||||
info.append({key: value for key, value in summary.items() if key != "pem"})
|
||||
certs["info"] = info
|
||||
ops["certificates"] = certs
|
||||
await save_node_ops(request, node, ops)
|
||||
return summary
|
||||
|
||||
async def certificates_info(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(values(inputs)["node"])
|
||||
@@ -333,7 +271,39 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
certs["info"] = info
|
||||
ops["certificates"] = certs
|
||||
await save_node_ops(request, node, ops)
|
||||
return [dict(item) for item in info]
|
||||
if not info:
|
||||
from app.simulation.seed import _seed_fingerprint
|
||||
|
||||
info = [
|
||||
{
|
||||
"filename": "pve-ssl.pem",
|
||||
"fingerprint": _seed_fingerprint(node),
|
||||
"issuer": f"CN={node}.proxmox.local",
|
||||
"subject": f"CN={node}.proxmox.local",
|
||||
"notbefore": 1_700_000_000,
|
||||
"notafter": 2_000_000_000,
|
||||
"pem": (
|
||||
"-----BEGIN CERTIFICATE-----\n"
|
||||
"SIMULATOR-SEED-CERT\n"
|
||||
"-----END CERTIFICATE-----\n"
|
||||
),
|
||||
"san": [f"{node}.proxmox.local"],
|
||||
"public-key-type": "rsa",
|
||||
"public-key-bits": 4096,
|
||||
}
|
||||
]
|
||||
certs["info"] = info
|
||||
ops["certificates"] = certs
|
||||
await save_node_ops(request, node, ops)
|
||||
result: list[dict[str, Any]] = []
|
||||
for item in info:
|
||||
entry = dict(item)
|
||||
entry.setdefault("pem", "-----BEGIN CERTIFICATE-----\nSIM\n-----END CERTIFICATE-----\n")
|
||||
entry.setdefault("san", [str(entry.get("subject", "CN=pve.local")).removeprefix("CN=")])
|
||||
entry.setdefault("public-key-type", "rsa")
|
||||
entry.setdefault("public-key-bits", 4096)
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
async def scan_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -360,35 +330,40 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
return subdirs("cpu", "cpu-flags", "machines", "migration")
|
||||
|
||||
async def capabilities_cpu(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
return [
|
||||
{"name": "host", "vendor": "QEMU", "custom": 0},
|
||||
{"name": "x86-64-v2-AES", "vendor": "QEMU", "custom": 0},
|
||||
{"name": "kvm64", "vendor": "QEMU", "custom": 0},
|
||||
]
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
caps = ops.get("capabilities")
|
||||
items = caps.get("cpu") if isinstance(caps, dict) else None
|
||||
return [dict(item) for item in items] if isinstance(items, list) else []
|
||||
|
||||
async def capabilities_cpu_flags(
|
||||
request: Request, inputs: dict[str, Any]
|
||||
) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
return [
|
||||
{"name": "aes", "introduces": "Westmere"},
|
||||
{"name": "avx", "introduces": "SandyBridge"},
|
||||
{"name": "avx2", "introduces": "Haswell"},
|
||||
]
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
caps = ops.get("capabilities")
|
||||
items = caps.get("cpu_flags") if isinstance(caps, dict) else None
|
||||
return [dict(item) for item in items] if isinstance(items, list) else []
|
||||
|
||||
async def capabilities_machines(
|
||||
request: Request, inputs: dict[str, Any]
|
||||
) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
return [
|
||||
{"id": "pc-i440fx-9.0", "type": "i440fx", "version": "9.0"},
|
||||
{"id": "pc-q35-9.0", "type": "q35", "version": "9.0"},
|
||||
]
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
caps = ops.get("capabilities")
|
||||
items = caps.get("machines") if isinstance(caps, dict) else None
|
||||
return [dict(item) for item in items] if isinstance(items, list) else []
|
||||
|
||||
async def capabilities_migration(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
return {"network": "", "type": "secure", "enabled": 1}
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
caps = ops.get("capabilities")
|
||||
migration = caps.get("migration") if isinstance(caps, dict) else None
|
||||
return dict(migration) if isinstance(migration, dict) else {}
|
||||
|
||||
async def hardware_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -399,8 +374,6 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
hardware = _hardware(ops)
|
||||
ops["hardware"] = hardware
|
||||
await save_node_ops(request, node, ops)
|
||||
return [dict(item) for item in hardware.get("pci", [])]
|
||||
|
||||
async def hardware_pci_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -430,8 +403,6 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
hardware = _hardware(ops)
|
||||
ops["hardware"] = hardware
|
||||
await save_node_ops(request, node, ops)
|
||||
return [dict(item) for item in hardware.get("usb", [])]
|
||||
|
||||
async def subscription_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -452,7 +423,10 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
current = _subscription(ops)
|
||||
method = request.method.upper()
|
||||
if method == "DELETE":
|
||||
ops["subscription"] = copy.deepcopy(DEFAULT_SUBSCRIPTION)
|
||||
ops["subscription"] = {
|
||||
"status": "notfound",
|
||||
"message": "There is no subscription key",
|
||||
}
|
||||
await save_node_ops(request, node, ops)
|
||||
return None
|
||||
if method == "POST":
|
||||
@@ -481,17 +455,8 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
ops = await load_node_ops(request, node)
|
||||
items = ops.get("aplinfo")
|
||||
if not isinstance(items, list):
|
||||
items = [
|
||||
{
|
||||
"package": "alpine-3-standard",
|
||||
"section": "system",
|
||||
"type": "lxc",
|
||||
"version": "3.20",
|
||||
}
|
||||
]
|
||||
ops["aplinfo"] = items
|
||||
await save_node_ops(request, node, ops)
|
||||
return [dict(item) for item in items]
|
||||
return []
|
||||
return [dict(item) for item in items if isinstance(item, dict)]
|
||||
|
||||
async def aplinfo_download(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = values(inputs)
|
||||
@@ -514,45 +479,140 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
node = str(payload["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
apt = ops.setdefault("apt", copy.deepcopy(default_node_ops()["apt"]))
|
||||
repositories = list(apt.get("repositories") or [])
|
||||
apt = ops.get("apt")
|
||||
if not isinstance(apt, dict):
|
||||
apt = {}
|
||||
ops["apt"] = apt
|
||||
envelope = apt.get("repositories")
|
||||
if isinstance(envelope, dict) and "files" in envelope:
|
||||
files = [
|
||||
dict(item) for item in list(envelope.get("files") or []) if isinstance(item, dict)
|
||||
]
|
||||
standard = [
|
||||
dict(item)
|
||||
for item in list(envelope.get("standard-repos") or [])
|
||||
if isinstance(item, dict)
|
||||
]
|
||||
else:
|
||||
legacy = list(envelope) if isinstance(envelope, list) else []
|
||||
files = [
|
||||
{
|
||||
"path": "/etc/apt/sources.list.d/pve-enterprise.list",
|
||||
"file-type": "list",
|
||||
"digest": "seedapt-file0",
|
||||
"repositories": legacy,
|
||||
}
|
||||
]
|
||||
standard = []
|
||||
method = request.method.upper()
|
||||
if method == "POST":
|
||||
entry = {
|
||||
handle = payload.get("handle")
|
||||
if handle is not None:
|
||||
updated_standard: list[dict[str, Any]] = []
|
||||
found = False
|
||||
for item in standard:
|
||||
if item.get("handle") == handle:
|
||||
found = True
|
||||
merged = dict(item)
|
||||
if "enabled" in payload:
|
||||
merged["status"] = int(bool(payload.get("enabled")))
|
||||
updated_standard.append(merged)
|
||||
else:
|
||||
updated_standard.append(item)
|
||||
if not found:
|
||||
updated_standard.append(
|
||||
{
|
||||
"handle": str(handle),
|
||||
"name": str(payload.get("name") or handle),
|
||||
"status": int(bool(payload.get("enabled", 1))),
|
||||
}
|
||||
)
|
||||
standard = updated_standard
|
||||
elif method == "POST":
|
||||
repo_entry = {
|
||||
key: value
|
||||
for key, value in payload.items()
|
||||
if key not in {"node", "delete", "digest"}
|
||||
if key not in {"node", "delete", "digest", "path", "handle", "index", "file-type"}
|
||||
}
|
||||
entry.setdefault("path", f"/etc/apt/sources.list.d/sim-{secrets.token_hex(2)}.list")
|
||||
entry.setdefault("enabled", 1)
|
||||
repositories.append(entry)
|
||||
path = str(
|
||||
payload.get("path") or f"/etc/apt/sources.list.d/sim-{secrets.token_hex(2)}.list"
|
||||
)
|
||||
matched = False
|
||||
for file_entry in files:
|
||||
if file_entry.get("path") == path:
|
||||
repos = list(file_entry.get("repositories") or [])
|
||||
repos.append(repo_entry)
|
||||
file_entry["repositories"] = repos
|
||||
matched = True
|
||||
break
|
||||
if not matched:
|
||||
files.append(
|
||||
{
|
||||
"path": path,
|
||||
"file-type": str(payload.get("file-type") or "list"),
|
||||
"digest": secrets.token_hex(4),
|
||||
"repositories": [repo_entry],
|
||||
}
|
||||
)
|
||||
else:
|
||||
path = payload.get("path")
|
||||
handle = payload.get("handle")
|
||||
target_path = payload.get("path")
|
||||
index = payload.get("index")
|
||||
updated: list[dict[str, Any]] = []
|
||||
for idx, item in enumerate(repositories):
|
||||
match = False
|
||||
if path is not None and item.get("path") == path:
|
||||
match = True
|
||||
if handle is not None and item.get("handle") == handle:
|
||||
match = True
|
||||
if index is not None and idx == int(index):
|
||||
match = True
|
||||
if match or (path is None and handle is None and index is None and idx == 0):
|
||||
merged = {
|
||||
**item,
|
||||
for file_idx, file_entry in enumerate(files):
|
||||
if target_path is not None and file_entry.get("path") != target_path:
|
||||
continue
|
||||
repos = [
|
||||
dict(item)
|
||||
for item in list(file_entry.get("repositories") or [])
|
||||
if isinstance(item, dict)
|
||||
]
|
||||
if index is not None:
|
||||
idx = int(index)
|
||||
if 0 <= idx < len(repos):
|
||||
repos[idx] = {
|
||||
**repos[idx],
|
||||
**{
|
||||
key: value
|
||||
for key, value in payload.items()
|
||||
if key not in {"node", "delete", "digest", "path", "handle", "index"}
|
||||
if key
|
||||
not in {
|
||||
"node",
|
||||
"delete",
|
||||
"digest",
|
||||
"path",
|
||||
"handle",
|
||||
"index",
|
||||
"file-type",
|
||||
}
|
||||
},
|
||||
}
|
||||
updated.append(merged)
|
||||
else:
|
||||
updated.append(item)
|
||||
repositories = updated
|
||||
apt["repositories"] = repositories
|
||||
elif target_path is not None or file_idx == 0:
|
||||
if repos:
|
||||
repos[0] = {
|
||||
**repos[0],
|
||||
**{
|
||||
key: value
|
||||
for key, value in payload.items()
|
||||
if key
|
||||
not in {
|
||||
"node",
|
||||
"delete",
|
||||
"digest",
|
||||
"path",
|
||||
"handle",
|
||||
"index",
|
||||
"file-type",
|
||||
}
|
||||
},
|
||||
}
|
||||
file_entry["repositories"] = repos
|
||||
if target_path is not None:
|
||||
break
|
||||
apt["repositories"] = {
|
||||
"digest": secrets.token_hex(4),
|
||||
"errors": [],
|
||||
"infos": [],
|
||||
"standard-repos": standard,
|
||||
"files": files,
|
||||
}
|
||||
ops["apt"] = apt
|
||||
await save_node_ops(request, node, ops)
|
||||
|
||||
@@ -561,18 +621,14 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
config = ops.get("config")
|
||||
if not isinstance(config, dict):
|
||||
config = copy.deepcopy(DEFAULT_CONFIG)
|
||||
ops["config"] = config
|
||||
await save_node_ops(request, node, ops)
|
||||
return dict(config)
|
||||
return dict(config) if isinstance(config, dict) else {}
|
||||
|
||||
async def node_config_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
config = dict(ops.get("config") or DEFAULT_CONFIG)
|
||||
config = dict(ops.get("config") or {})
|
||||
config.update(
|
||||
{
|
||||
key: value
|
||||
@@ -589,18 +645,14 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
dns = ops.get("dns")
|
||||
if not isinstance(dns, dict):
|
||||
dns = copy.deepcopy(DEFAULT_DNS)
|
||||
ops["dns"] = dns
|
||||
await save_node_ops(request, node, ops)
|
||||
return dict(dns)
|
||||
return dict(dns) if isinstance(dns, dict) else {}
|
||||
|
||||
async def dns_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
node = str(payload["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
dns = dict(ops.get("dns") or DEFAULT_DNS)
|
||||
dns = dict(ops.get("dns") or {})
|
||||
dns.update(
|
||||
{
|
||||
key: value
|
||||
@@ -616,13 +668,10 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
current = dict(ops.get("time") or DEFAULT_TIME)
|
||||
current = dict(ops.get("time") or {})
|
||||
now = int(time.time())
|
||||
current["time"] = now
|
||||
current["localtime"] = now
|
||||
current.setdefault("timezone", "UTC")
|
||||
ops["time"] = current
|
||||
await save_node_ops(request, node, ops)
|
||||
return current
|
||||
|
||||
async def time_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -630,7 +679,7 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
node = str(payload["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
current = dict(ops.get("time") or DEFAULT_TIME)
|
||||
current = dict(ops.get("time") or {})
|
||||
if "timezone" in payload:
|
||||
current["timezone"] = str(payload["timezone"])
|
||||
now = int(time.time())
|
||||
@@ -668,12 +717,7 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
ops = await load_node_ops(request, node)
|
||||
hosts = ops.get("hosts")
|
||||
if not isinstance(hosts, dict):
|
||||
hosts = {
|
||||
"data": f"127.0.0.1 localhost\n10.0.0.10 {node}\n",
|
||||
"digest": secrets.token_hex(8),
|
||||
}
|
||||
ops["hosts"] = hosts
|
||||
await save_node_ops(request, node, ops)
|
||||
return {"data": "", "digest": ""}
|
||||
return {"data": str(hosts.get("data", "")), "digest": str(hosts.get("digest", ""))}
|
||||
|
||||
async def hosts_post(request: Request, inputs: dict[str, Any]) -> None:
|
||||
@@ -692,46 +736,50 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
start = int(values(inputs).get("startcursor") or values(inputs).get("start") or 0)
|
||||
limit = int(values(inputs).get("limit") or 50)
|
||||
lines = [
|
||||
f"{index}: {node} systemd[1]: Started simulated service {index}."
|
||||
for index in range(start, start + limit)
|
||||
]
|
||||
return lines
|
||||
ops = await load_node_ops(request, node)
|
||||
lines = ops.get("journal")
|
||||
if not isinstance(lines, list):
|
||||
return []
|
||||
sliced = lines[start : start + limit]
|
||||
return [str(item) for item in sliced]
|
||||
|
||||
async def syslog(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
limit = int(values(inputs).get("limit") or 50)
|
||||
return [
|
||||
{"n": index, "t": f"{node} kernel: simulated syslog line {index}"}
|
||||
for index in range(limit)
|
||||
]
|
||||
ops = await load_node_ops(request, node)
|
||||
lines = ops.get("syslog")
|
||||
if not isinstance(lines, list):
|
||||
return []
|
||||
return [dict(item) for item in lines[:limit] if isinstance(item, dict)]
|
||||
|
||||
async def netstat(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
return [
|
||||
{"in": 1_000_000, "out": 900_000, "vnet": "vmbr0", "hwaddr": "bc:24:11:00:00:01"},
|
||||
{"in": 500_000, "out": 450_000, "vnet": "vmbr1", "hwaddr": "bc:24:11:00:00:02"},
|
||||
]
|
||||
ops = await load_node_ops(request, node)
|
||||
items = ops.get("netstat")
|
||||
return [dict(item) for item in items] if isinstance(items, list) else []
|
||||
|
||||
async def report(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
return f"==== Proxmox node report for {node} ====\nuptime: simulated\n"
|
||||
ops = await load_node_ops(request, node)
|
||||
report_text = ops.get("report")
|
||||
return str(report_text) if report_text is not None else ""
|
||||
|
||||
async def rrd(_request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
await require_node(_request, str(values(inputs)["node"]))
|
||||
return {"filename": "/var/lib/rrdcached/db/pve-node.rrd"}
|
||||
async def rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
rrd_state = ops.get("rrd")
|
||||
return dict(rrd_state) if isinstance(rrd_state, dict) else {}
|
||||
|
||||
async def rrddata(_request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(_request, str(values(inputs)["node"]))
|
||||
now = int(time.time())
|
||||
return [
|
||||
{"time": now - 120, "cpu": 0.05, "memused": 1_000_000_000},
|
||||
{"time": now - 60, "cpu": 0.07, "memused": 1_100_000_000},
|
||||
{"time": now, "cpu": 0.04, "memused": 1_050_000_000},
|
||||
]
|
||||
async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
series = ops.get("rrddata")
|
||||
return [dict(item) for item in series] if isinstance(series, list) else []
|
||||
|
||||
async def startall(request: Request, inputs: dict[str, Any]) -> str:
|
||||
node = str(values(inputs)["node"])
|
||||
@@ -809,7 +857,7 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
ops = await load_node_ops(request, node)
|
||||
if not isinstance(ops.get("network"), list):
|
||||
ops["network"] = copy.deepcopy(default_node_ops()["network"])
|
||||
ops["network"] = []
|
||||
ops["network_applied"] = False
|
||||
await save_node_ops(request, node, ops)
|
||||
|
||||
@@ -818,28 +866,24 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
await require_node(request, node)
|
||||
repo = str(values(inputs).get("repo") or "library/alpine")
|
||||
ops = await load_node_ops(request, node)
|
||||
cache = ops.setdefault("oci_tags", {})
|
||||
if repo not in cache:
|
||||
cache[repo] = [{"tag": "latest"}, {"tag": "3.20"}]
|
||||
ops["oci_tags"] = cache
|
||||
await save_node_ops(request, node, ops)
|
||||
return [dict(item) for item in cache[repo]]
|
||||
cache = ops.get("oci_tags")
|
||||
if not isinstance(cache, dict):
|
||||
return []
|
||||
items = cache.get(repo)
|
||||
if not isinstance(items, list):
|
||||
return []
|
||||
return [dict(item) for item in items if isinstance(item, dict)]
|
||||
|
||||
async def query_url_metadata(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node = str(values(inputs)["node"])
|
||||
await require_node(request, node)
|
||||
url = str(values(inputs).get("url") or "")
|
||||
ops = await load_node_ops(request, node)
|
||||
cache = ops.setdefault("url_metadata", {})
|
||||
if url not in cache:
|
||||
cache[url] = {
|
||||
"filename": url.rsplit("/", 1)[-1] or "download.bin",
|
||||
"mimetype": "application/octet-stream",
|
||||
"size": 1024,
|
||||
}
|
||||
ops["url_metadata"] = cache
|
||||
await save_node_ops(request, node, ops)
|
||||
return dict(cache[url])
|
||||
cache = ops.get("url_metadata")
|
||||
if not isinstance(cache, dict):
|
||||
return {}
|
||||
payload = cache.get(url)
|
||||
return dict(payload) if isinstance(payload, dict) else {}
|
||||
|
||||
async def vncwebsocket(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node = str(values(inputs)["node"])
|
||||
|
||||
@@ -13,53 +13,40 @@ from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs
|
||||
|
||||
_SECRET_KEYS = frozenset({"token", "password", "secret"})
|
||||
|
||||
DEFAULT_MATCHER_FIELDS = [
|
||||
{"name": "type", "type": "string"},
|
||||
{"name": "hostname", "type": "string"},
|
||||
{"name": "job-id", "type": "string"},
|
||||
{"name": "severity", "type": "string"},
|
||||
]
|
||||
|
||||
DEFAULT_MATCHER_FIELD_VALUES = [
|
||||
{"field": "type", "value": "fencing"},
|
||||
{"field": "type", "value": "package-updates"},
|
||||
{"field": "type", "value": "replication"},
|
||||
{"field": "type", "value": "system-mail"},
|
||||
]
|
||||
|
||||
|
||||
def _public(endpoint: dict[str, Any]) -> dict[str, Any]:
|
||||
return {key: value for key, value in endpoint.items() if key not in _SECRET_KEYS}
|
||||
|
||||
|
||||
def _notifications(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
current = metadata.setdefault(
|
||||
"notifications",
|
||||
{
|
||||
"endpoints": {
|
||||
"gotify": {},
|
||||
"sendmail": {},
|
||||
"smtp": {},
|
||||
"webhook": {},
|
||||
},
|
||||
"matchers": {},
|
||||
"tests": [],
|
||||
},
|
||||
)
|
||||
current = metadata.get("notifications")
|
||||
if not isinstance(current, dict):
|
||||
current = {
|
||||
return {
|
||||
"endpoints": {"gotify": {}, "sendmail": {}, "smtp": {}, "webhook": {}},
|
||||
"matchers": {},
|
||||
"tests": [],
|
||||
}
|
||||
metadata["notifications"] = current
|
||||
current.setdefault(
|
||||
"endpoints",
|
||||
{"gotify": {}, "sendmail": {}, "smtp": {}, "webhook": {}},
|
||||
endpoints = current.get("endpoints")
|
||||
if not isinstance(endpoints, dict):
|
||||
endpoints = {}
|
||||
result = {
|
||||
key: value
|
||||
for key, value in current.items()
|
||||
if key not in {"endpoints", "matchers", "tests"}
|
||||
}
|
||||
result["endpoints"] = {
|
||||
"gotify": dict(endpoints["gotify"]) if isinstance(endpoints.get("gotify"), dict) else {},
|
||||
"sendmail": (
|
||||
dict(endpoints["sendmail"]) if isinstance(endpoints.get("sendmail"), dict) else {}
|
||||
),
|
||||
"smtp": dict(endpoints["smtp"]) if isinstance(endpoints.get("smtp"), dict) else {},
|
||||
"webhook": dict(endpoints["webhook"]) if isinstance(endpoints.get("webhook"), dict) else {},
|
||||
}
|
||||
result["matchers"] = (
|
||||
dict(current["matchers"]) if isinstance(current.get("matchers"), dict) else {}
|
||||
)
|
||||
current.setdefault("matchers", {})
|
||||
current.setdefault("tests", [])
|
||||
return current
|
||||
result["tests"] = list(current["tests"]) if isinstance(current.get("tests"), list) else []
|
||||
return result
|
||||
|
||||
|
||||
def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
@@ -80,26 +67,28 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def list_endpoints(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["endpoints"].setdefault(kind, {})
|
||||
store = _notifications(metadata)["endpoints"].get(kind) or {}
|
||||
return [_public({"name": name, **item}) for name, item in sorted(store.items())]
|
||||
|
||||
async def create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
name = str(payload["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["endpoints"].setdefault(kind, {})
|
||||
notifications = _notifications(metadata)
|
||||
store = notifications["endpoints"].setdefault(kind, {})
|
||||
if name in store:
|
||||
raise ApiError(400, f"{kind} endpoint '{name}' already exists")
|
||||
entry = {key: payload[key] for key in create_keys if key in payload}
|
||||
entry["name"] = name
|
||||
entry.setdefault("disable", 0)
|
||||
store[name] = entry
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
name = str(values(inputs)["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["endpoints"].setdefault(kind, {})
|
||||
store = _notifications(metadata)["endpoints"].get(kind) or {}
|
||||
if name not in store:
|
||||
raise ApiError(404, f"{kind} endpoint does not exist")
|
||||
return _public({"name": name, **store[name]})
|
||||
@@ -108,7 +97,8 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
name = str(payload["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["endpoints"].setdefault(kind, {})
|
||||
notifications = _notifications(metadata)
|
||||
store = notifications["endpoints"].setdefault(kind, {})
|
||||
if name not in store:
|
||||
raise ApiError(404, f"{kind} endpoint does not exist")
|
||||
current = dict(store[name])
|
||||
@@ -123,15 +113,18 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
current[key] = value
|
||||
current["name"] = name
|
||||
store[name] = current
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
name = str(values(inputs)["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["endpoints"].setdefault(kind, {})
|
||||
notifications = _notifications(metadata)
|
||||
store = notifications["endpoints"].setdefault(kind, {})
|
||||
if name not in store:
|
||||
raise ApiError(404, f"{kind} endpoint does not exist")
|
||||
del store[name]
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
registry.register(base, "GET", list_endpoints)
|
||||
@@ -149,12 +142,14 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
name = str(payload["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["matchers"]
|
||||
notifications = _notifications(metadata)
|
||||
store = notifications["matchers"]
|
||||
if name in store:
|
||||
raise ApiError(400, f"matcher '{name}' already exists")
|
||||
store[name] = {
|
||||
key: value for key, value in payload.items() if key not in {"delete", "digest"}
|
||||
}
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def matchers_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -169,7 +164,8 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
name = str(payload["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["matchers"]
|
||||
notifications = _notifications(metadata)
|
||||
store = notifications["matchers"]
|
||||
if name not in store:
|
||||
raise ApiError(404, "matcher does not exist")
|
||||
current = dict(store[name])
|
||||
@@ -183,24 +179,35 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
current[key] = value
|
||||
current["name"] = name
|
||||
store[name] = current
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def matchers_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
name = str(values(inputs)["name"])
|
||||
metadata = await cluster_metadata(request)
|
||||
store = _notifications(metadata)["matchers"]
|
||||
notifications = _notifications(metadata)
|
||||
store = notifications["matchers"]
|
||||
if name not in store:
|
||||
raise ApiError(404, "matcher does not exist")
|
||||
del store[name]
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def matcher_fields(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
return list(DEFAULT_MATCHER_FIELDS)
|
||||
async def matcher_fields(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
fields = _notifications(metadata).get("matcher_fields")
|
||||
if isinstance(fields, list):
|
||||
return [dict(item) for item in fields if isinstance(item, dict)]
|
||||
return []
|
||||
|
||||
async def matcher_field_values(
|
||||
_request: Request, _inputs: dict[str, Any]
|
||||
request: Request, _inputs: dict[str, Any]
|
||||
) -> list[dict[str, Any]]:
|
||||
return list(DEFAULT_MATCHER_FIELD_VALUES)
|
||||
metadata = await cluster_metadata(request)
|
||||
values_list = _notifications(metadata).get("matcher_field_values")
|
||||
if isinstance(values_list, list):
|
||||
return [dict(item) for item in values_list if isinstance(item, dict)]
|
||||
return []
|
||||
|
||||
async def targets(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
@@ -231,10 +238,10 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
|
||||
break
|
||||
if not found:
|
||||
raise ApiError(404, "notification target does not exist")
|
||||
tests = notifications.setdefault("tests", [])
|
||||
if not isinstance(tests, list):
|
||||
tests = notifications["tests"] = []
|
||||
tests = list(notifications.get("tests") or [])
|
||||
tests.append({"name": name, "tested_at": int(time.time()), "ok": True})
|
||||
notifications["tests"] = tests
|
||||
metadata["notifications"] = notifications
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
registry.register("/cluster/notifications", "GET", index)
|
||||
|
||||
@@ -10,18 +10,35 @@ from fastapi import Request
|
||||
|
||||
from app.api.errors import ApiError
|
||||
from app.api.registry import HandlerRegistry
|
||||
from app.handlers.common import database, state, values
|
||||
from app.handlers.common import database, require_value, state, values
|
||||
from app.simulation.seed import CLUSTER_ID, stable_id
|
||||
|
||||
|
||||
async def _pool_members(request: Request, pool_id: uuid.UUID) -> list[str]:
|
||||
async def _pool_members(request: Request, pool_id: uuid.UUID) -> list[dict[str, Any]]:
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT r.external_id FROM pool_members pm
|
||||
"""SELECT r.external_id, r.kind, r.state, n.name AS node
|
||||
FROM pool_members pm
|
||||
JOIN resources r ON r.id = pm.resource_id
|
||||
WHERE pm.pool_id=$1 ORDER BY r.external_id::integer""",
|
||||
JOIN nodes n ON n.id = r.node_id
|
||||
WHERE pm.pool_id=$1
|
||||
ORDER BY r.external_id::integer""",
|
||||
pool_id,
|
||||
)
|
||||
return [str(row["external_id"]) for row in rows]
|
||||
members: list[dict[str, Any]] = []
|
||||
for row in rows:
|
||||
guest_state = state(row["state"])
|
||||
kind = str(row["kind"])
|
||||
vmid = int(row["external_id"])
|
||||
members.append(
|
||||
{
|
||||
"id": f"{kind}/{vmid}",
|
||||
"node": str(row["node"]),
|
||||
"type": kind,
|
||||
"vmid": vmid,
|
||||
"name": str(guest_state.get("name") or f"{kind}-{vmid}"),
|
||||
}
|
||||
)
|
||||
return members
|
||||
|
||||
|
||||
def register_pool_handlers(registry: HandlerRegistry) -> None:
|
||||
@@ -39,16 +56,25 @@ def register_pool_handlers(registry: HandlerRegistry) -> None:
|
||||
metadata = state(row["metadata"])
|
||||
members = await _pool_members(request, row["id"])
|
||||
if not members and isinstance(metadata.get("members"), list):
|
||||
members = [str(item) for item in metadata["members"]]
|
||||
item: dict[str, Any] = {
|
||||
# Fallback for metadata-only membership before pool_members rows exist.
|
||||
for item in metadata["members"]:
|
||||
members.append(
|
||||
{
|
||||
"id": f"qemu/{item}",
|
||||
"node": "",
|
||||
"type": "qemu",
|
||||
"vmid": int(item) if str(item).isdigit() else 0,
|
||||
}
|
||||
)
|
||||
pool_item: dict[str, Any] = {
|
||||
"poolid": str(row["pool_id"]),
|
||||
"members": members,
|
||||
}
|
||||
if row["comment"] is not None:
|
||||
item["comment"] = str(row["comment"])
|
||||
pool_item["comment"] = str(row["comment"])
|
||||
elif metadata.get("comment"):
|
||||
item["comment"] = str(metadata["comment"])
|
||||
result.append(item)
|
||||
pool_item["comment"] = str(metadata["comment"])
|
||||
result.append(pool_item)
|
||||
return result
|
||||
|
||||
async def pool_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -117,7 +143,7 @@ def register_pool_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
|
||||
async def pool_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
poolid = str(values(inputs)["poolid"])
|
||||
poolid = str(require_value(values(inputs), "poolid"))
|
||||
status = await database(request).pool.execute(
|
||||
"DELETE FROM pools WHERE pool_id=$1",
|
||||
poolid,
|
||||
|
||||
@@ -33,16 +33,124 @@ def _state(value: object) -> dict[str, Any]:
|
||||
return dict(cast(Mapping[str, Any], value))
|
||||
|
||||
|
||||
# Runtime-only keys stored in ``resources.state``; never echo on GET /config.
|
||||
_QEMU_CONFIG_INTERNAL = frozenset(
|
||||
{
|
||||
"agent",
|
||||
"cloudinit_dump",
|
||||
"config",
|
||||
"interfaces",
|
||||
"migrate_preconditions",
|
||||
"rrd",
|
||||
"rrddata",
|
||||
"sendkey",
|
||||
"status",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _agent_config_string(value: object) -> str:
|
||||
"""Proxmox QEMU config ``agent`` is a string (e.g. ``1`` / ``0``), not an object."""
|
||||
|
||||
if isinstance(value, dict):
|
||||
return "1" if value.get("enabled", True) else "0"
|
||||
if isinstance(value, bool):
|
||||
return "1" if value else "0"
|
||||
if value in (None, ""):
|
||||
return "0"
|
||||
text = str(value).strip()
|
||||
return text or "0"
|
||||
|
||||
|
||||
def _public_qemu_config(config: dict[str, Any], state: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Wire-format QEMU config for clients (bpg / pulumi-proxmoxve)."""
|
||||
|
||||
import hashlib
|
||||
|
||||
merged = {**state, **config}
|
||||
payload: dict[str, Any] = {}
|
||||
for key, value in merged.items():
|
||||
if key in _QEMU_CONFIG_INTERNAL:
|
||||
continue
|
||||
if isinstance(value, dict | list):
|
||||
continue
|
||||
payload[key] = value
|
||||
agent = config.get("agent", state.get("agent", "0"))
|
||||
payload["agent"] = _agent_config_string(agent)
|
||||
digest_src = {key: value for key, value in payload.items() if key != "digest"}
|
||||
payload["digest"] = hashlib.sha1( # noqa: S324 - Proxmox config digests use SHA-1
|
||||
json.dumps(digest_src, sort_keys=True, separators=(",", ":")).encode()
|
||||
).hexdigest()
|
||||
return payload
|
||||
|
||||
|
||||
def _cpu_utilization(vm_state: Mapping[str, Any], *, running: bool, idle: float = 0.1) -> float:
|
||||
"""Status ``cpu`` is utilization; config may store model strings like ``qemu64``."""
|
||||
|
||||
raw = vm_state.get("cpu")
|
||||
if isinstance(raw, (int, float)):
|
||||
return float(raw)
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
return float(raw)
|
||||
except ValueError:
|
||||
pass
|
||||
return idle if running else 0.0
|
||||
|
||||
|
||||
def _qemu_index_item(vmid: int, vm_state: dict[str, Any], config: dict[str, Any]) -> dict[str, Any]:
|
||||
status = str(vm_state.get("status", "stopped"))
|
||||
running = status in {"running", "paused"}
|
||||
memory_mb = int(config.get("memory", config.get("mem", 2048)))
|
||||
maxmem = memory_mb * 2**20
|
||||
mem = int(vm_state.get("mem", maxmem // 2 if running else 0))
|
||||
return {
|
||||
"vmid": vmid,
|
||||
"name": str(config.get("name", f"vm-{vmid}")),
|
||||
"status": status,
|
||||
"qmpstatus": status if running else "stopped",
|
||||
"cpus": int(config.get("cores", config.get("cpus", 1))),
|
||||
"cpu": _cpu_utilization(vm_state, running=running),
|
||||
"maxmem": maxmem,
|
||||
"mem": mem,
|
||||
"memhost": int(vm_state.get("memhost", mem)),
|
||||
"maxdisk": int(vm_state.get("maxdisk", config.get("maxdisk", 32 * 1024**3))),
|
||||
"disk": int(vm_state.get("disk", 0)),
|
||||
"diskread": int(vm_state.get("diskread", 0)),
|
||||
"diskwrite": int(vm_state.get("diskwrite", 0)),
|
||||
"netin": int(vm_state.get("netin", 0)),
|
||||
"netout": int(vm_state.get("netout", 0)),
|
||||
"uptime": int(vm_state.get("uptime", 0)) if running else 0,
|
||||
"pid": int(vm_state.get("pid", 12_345 if running else 0)),
|
||||
"lock": str(vm_state.get("lock", "")),
|
||||
"tags": str(config.get("tags", "")),
|
||||
"template": int(bool(vm_state.get("template", False))),
|
||||
"running-qemu": str(vm_state.get("running-qemu", "9.2.0")) if running else "",
|
||||
"running-machine": str(vm_state.get("running-machine", "pc-i440fx-9.2")) if running else "",
|
||||
"pressurecpusome": float(vm_state.get("pressurecpusome", 0.0)),
|
||||
"pressurecpufull": float(vm_state.get("pressurecpufull", 0.0)),
|
||||
"pressureiosome": float(vm_state.get("pressureiosome", 0.0)),
|
||||
"pressureiofull": float(vm_state.get("pressureiofull", 0.0)),
|
||||
"pressurememorysome": float(vm_state.get("pressurememorysome", 0.0)),
|
||||
"pressurememoryfull": float(vm_state.get("pressurememoryfull", 0.0)),
|
||||
}
|
||||
|
||||
|
||||
def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
async def qemu_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
node = str(_values(inputs)["node"])
|
||||
rows = await _database(request).pool.fetch(
|
||||
"""SELECT r.external_id::integer AS vmid, r.state
|
||||
FROM resources r JOIN nodes n ON n.id=r.node_id
|
||||
"""SELECT r.external_id::integer AS vmid, r.state, v.config
|
||||
FROM resources r
|
||||
JOIN nodes n ON n.id=r.node_id
|
||||
JOIN virtual_machines v ON v.resource_id=r.id
|
||||
WHERE n.name=$1 AND r.kind='qemu' ORDER BY r.external_id::integer""",
|
||||
node,
|
||||
)
|
||||
return [{"vmid": int(row["vmid"]), **_state(row["state"])} for row in rows]
|
||||
return [
|
||||
_qemu_index_item(int(row["vmid"]), _state(row["state"]), _state(row["config"]))
|
||||
for row in rows
|
||||
]
|
||||
|
||||
async def qemu_status_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
payload = _values(inputs)
|
||||
@@ -82,15 +190,11 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
else 0,
|
||||
)
|
||||
)
|
||||
return {
|
||||
"vmid": vmid,
|
||||
"name": str(config.get("name", f"vm-{vmid}")),
|
||||
"status": status,
|
||||
"qmpstatus": status if running else "stopped",
|
||||
"lock": str(vm_state.get("lock", "")),
|
||||
"pid": int(vm_state.get("pid", 12_345 if running else 0)),
|
||||
"cpus": int(config.get("cores", config.get("cpus", 1))),
|
||||
"maxmem": maxmem,
|
||||
agent_enabled = str(config.get("agent", "0")).startswith("1")
|
||||
item = _qemu_index_item(vmid, vm_state, config)
|
||||
item.update(
|
||||
{
|
||||
"uptime": uptime,
|
||||
"mem": mem_used,
|
||||
"balloon": int(vm_state.get("balloon", 0)),
|
||||
"ballooninfo": {
|
||||
@@ -99,11 +203,15 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
"mem_swapped_in": 0,
|
||||
"mem_swapped_out": 0,
|
||||
},
|
||||
"uptime": uptime,
|
||||
"template": int(bool(vm_state.get("template", False))),
|
||||
"template": bool(vm_state.get("template", False)),
|
||||
"ha": {"managed": int(vm_state.get("ha_managed", 0))},
|
||||
"agent": 1 if running and str(config.get("agent", "0")).startswith("1") else 0,
|
||||
"agent": bool(running and agent_enabled),
|
||||
"clipboard": str(vm_state.get("clipboard", "")),
|
||||
"spice": bool(vm_state.get("spice", False)),
|
||||
"serial": bool(vm_state.get("serial", False)),
|
||||
}
|
||||
)
|
||||
return item
|
||||
|
||||
async def qemu_config(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
node, vmid = str(_values(inputs)["node"]), str(_values(inputs)["vmid"])
|
||||
@@ -117,7 +225,10 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if row is None:
|
||||
raise ApiError(404, "virtual machine does not exist")
|
||||
return {"vmid": int(vmid), **_state(row["config"]), **_state(row["state"])}
|
||||
return {
|
||||
"vmid": int(vmid),
|
||||
**_public_qemu_config(_state(row["config"]), _state(row["state"])),
|
||||
}
|
||||
|
||||
async def mutate(operation: str, request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
@@ -291,24 +402,30 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
values = _values(inputs)
|
||||
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
rows = await _database(request).pool.fetch(
|
||||
"""SELECT name, parent_name, description, created_at FROM snapshots
|
||||
"""SELECT name, parent_name, description, created_at, state FROM snapshots
|
||||
WHERE resource_id=$1 ORDER BY created_at, name""",
|
||||
resource["id"],
|
||||
)
|
||||
return [
|
||||
result: list[dict[str, Any]] = []
|
||||
for row in rows:
|
||||
raw_state = row["state"] if "state" in row else None
|
||||
snap_state = _state(raw_state) if raw_state is not None else {}
|
||||
result.append(
|
||||
{
|
||||
"name": row["name"],
|
||||
"parent": row["parent_name"],
|
||||
"description": row["description"] or "",
|
||||
"snaptime": int(row["created_at"].timestamp()),
|
||||
"vmstate": int(bool(snap_state.get("vmstate"))),
|
||||
}
|
||||
for row in rows
|
||||
]
|
||||
)
|
||||
return result
|
||||
|
||||
async def snapshot_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
values = _values(inputs)
|
||||
row = await _snapshot(request, values)
|
||||
state = _state(row["state"])
|
||||
raw_state = row["state"] if "state" in row else None
|
||||
state = _state(raw_state) if raw_state is not None else {}
|
||||
return {
|
||||
"name": row["name"],
|
||||
"parent": row["parent_name"],
|
||||
@@ -406,7 +523,7 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def migrate_preconditions(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
values = _values(inputs)
|
||||
await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
target = values.get("target")
|
||||
if target in {None, ""}:
|
||||
raise ApiError(400, "parameter 'target' is required")
|
||||
@@ -416,7 +533,13 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
)
|
||||
if not exists:
|
||||
raise ApiError(404, "target node does not exist")
|
||||
return {"local_disks": [], "local_resources": [], "running": False}
|
||||
state = _state(resource["state"])
|
||||
pre = state.get("migrate_preconditions")
|
||||
payload = dict(pre) if isinstance(pre, dict) else {}
|
||||
payload["running"] = str(state.get("status") or "") == "running"
|
||||
payload.setdefault("local_disks", [])
|
||||
payload.setdefault("local_resources", [])
|
||||
return payload
|
||||
|
||||
async def migrate(request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
@@ -474,7 +597,7 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
},
|
||||
)
|
||||
|
||||
async def resize(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def resize(request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
node, vmid, disk = str(values["node"]), str(values["vmid"]), str(values["disk"])
|
||||
resource = await _qemu_resource(request, node, vmid)
|
||||
@@ -507,6 +630,13 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
str(config[disk]).split(":", 1)[0],
|
||||
size,
|
||||
)
|
||||
return await _create_task(
|
||||
request,
|
||||
node=node,
|
||||
vmid=vmid,
|
||||
task_type="qemu-resize",
|
||||
payload={"resource_id": str(resource["id"]), "disk": disk, "size": size},
|
||||
)
|
||||
|
||||
async def move_disk(request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
@@ -544,39 +674,16 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
command: str, request: Request, inputs: dict[str, Any]
|
||||
) -> dict[str, Any]:
|
||||
resource = await _agent_resource(request, _values(inputs))
|
||||
config = _state(resource["config"])
|
||||
vmid = str(_values(inputs)["vmid"])
|
||||
results: dict[str, Any] = {
|
||||
"info": {
|
||||
"version": "9.2.0-simulator",
|
||||
"supported_commands": [
|
||||
{"name": name, "enabled": True, "success-response": True}
|
||||
for name in ("guest-ping", "guest-info", "guest-get-osinfo")
|
||||
],
|
||||
},
|
||||
"get-osinfo": {
|
||||
"name": str(config.get("ostype", "linux")),
|
||||
"pretty-name": "Proxmox Simulator Guest",
|
||||
"version": "1.0",
|
||||
"machine": "x86_64",
|
||||
},
|
||||
"get-host-name": {"host-name": str(config.get("name", f"vm-{vmid}"))},
|
||||
"network-get-interfaces": [
|
||||
{
|
||||
"name": "eth0",
|
||||
"hardware-address": "02:00:00:00:00:01",
|
||||
"ip-addresses": [
|
||||
{"ip-address": "192.0.2.10", "ip-address-type": "ipv4", "prefix": 24}
|
||||
],
|
||||
}
|
||||
],
|
||||
"ping": {},
|
||||
}
|
||||
if command == "get-time":
|
||||
seconds = int(
|
||||
await _database(request).pool.fetchval("SELECT extract(epoch from now())::bigint")
|
||||
)
|
||||
return {"result": {"seconds": seconds, "nanoseconds": 0}}
|
||||
state = _state(resource["state"])
|
||||
agent = state.get("agent")
|
||||
results = agent.get("results") if isinstance(agent, dict) else None
|
||||
if not isinstance(results, dict) or command not in results:
|
||||
raise ApiError(404, f"agent command '{command}' has no stored result")
|
||||
return {"result": results[command]}
|
||||
|
||||
async def agent_info(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -586,6 +693,12 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
return await agent_result("get-osinfo", request, inputs)
|
||||
|
||||
async def agent_hostname(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
resource = await _agent_resource(request, _values(inputs))
|
||||
state = _state(resource["state"])
|
||||
config = _state(resource["config"])
|
||||
name = state.get("name") or config.get("name")
|
||||
if name:
|
||||
return {"result": {"host-name": str(name)}}
|
||||
return await agent_result("get-host-name", request, inputs)
|
||||
|
||||
async def agent_network(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -598,12 +711,32 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
return await agent_result("ping", request, inputs)
|
||||
|
||||
async def task_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
tasks = await TaskRepository(_database(request).pool).list_for_node(
|
||||
str(_values(inputs)["node"])
|
||||
)
|
||||
return [
|
||||
{"upid": task.upid, "status": task.status, "type": task.task_type} for task in tasks
|
||||
]
|
||||
from app.tasks.upid import Upid
|
||||
|
||||
node = str(_values(inputs)["node"])
|
||||
tasks = await TaskRepository(_database(request).pool).list_for_node(node)
|
||||
result: list[dict[str, Any]] = []
|
||||
for task in tasks:
|
||||
try:
|
||||
parsed = Upid.parse(task.upid)
|
||||
except ValueError:
|
||||
continue
|
||||
finished = task.status in {"success", "error", "cancelled"}
|
||||
entry: dict[str, Any] = {
|
||||
"upid": task.upid,
|
||||
"id": parsed.task_id,
|
||||
"node": parsed.node,
|
||||
"pid": parsed.pid,
|
||||
"pstart": parsed.process_start,
|
||||
"starttime": parsed.start_time,
|
||||
"type": parsed.task_type,
|
||||
"user": parsed.user,
|
||||
}
|
||||
if finished:
|
||||
entry["status"] = "OK" if task.status == "success" else str(task.status).upper()
|
||||
entry["endtime"] = parsed.start_time + max(task.progress, 1)
|
||||
result.append(entry)
|
||||
return result
|
||||
|
||||
async def task_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
task = await TaskRepository(_database(request).pool).get_by_upid(
|
||||
@@ -644,7 +777,7 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
}
|
||||
}
|
||||
|
||||
async def qemu_template(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def qemu_template(request: Request, inputs: dict[str, Any]) -> str:
|
||||
payload = _values(inputs)
|
||||
node, vmid = str(payload["node"]), str(payload["vmid"])
|
||||
resource = await _qemu_resource(request, node, vmid)
|
||||
@@ -661,6 +794,13 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
|
||||
resource["id"],
|
||||
json.dumps(state, sort_keys=True),
|
||||
)
|
||||
return await _create_task(
|
||||
request,
|
||||
node=node,
|
||||
vmid=vmid,
|
||||
task_type="qemu-template",
|
||||
payload={"resource_id": str(resource["id"])},
|
||||
)
|
||||
|
||||
async def qemu_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
payload = _values(inputs)
|
||||
@@ -778,6 +918,8 @@ async def _create_task(
|
||||
"qemu-migrate": "qmigrate",
|
||||
"qemu-remote-migrate": "qmremote",
|
||||
"qemu-move-disk": "qmmove",
|
||||
"qemu-resize": "qmresize",
|
||||
"qemu-template": "qmtemplate",
|
||||
}[task_type]
|
||||
upid = str(Upid.allocate(node, worker_type, vmid, str(request.state.principal)))
|
||||
try:
|
||||
@@ -825,7 +967,14 @@ async def _agent_resource(request: Request, values: dict[str, Any]) -> Any:
|
||||
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
config = _state(resource["config"])
|
||||
state = _state(resource["state"])
|
||||
if str(config.get("agent", "0")).split(",", 1)[0].lower() not in {"1", "true", "yes"}:
|
||||
agent = state.get("agent")
|
||||
if not isinstance(agent, dict):
|
||||
agent = config.get("agent")
|
||||
if isinstance(agent, dict):
|
||||
enabled = bool(agent.get("enabled", True))
|
||||
else:
|
||||
enabled = str(config.get("agent", "0")).split(",", 1)[0].lower() in {"1", "true", "yes"}
|
||||
if not enabled:
|
||||
raise ApiError(409, "QEMU guest agent is not enabled")
|
||||
if state.get("status") != "running":
|
||||
raise ApiError(409, "QEMU guest agent is not running")
|
||||
|
||||
@@ -79,20 +79,10 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def _agent_blob(command: str, request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
resource = await _agent_resource(request, _values(inputs))
|
||||
state = _state(resource["state"])
|
||||
agent = state.setdefault("agent", {})
|
||||
blobs = agent.setdefault("results", {})
|
||||
defaults: dict[str, Any] = {
|
||||
"get-users": [{"user": "root", "login-time": 0}],
|
||||
"get-fsinfo": [{"name": "/", "type": "ext4", "total-bytes": 32 * 1024**3}],
|
||||
"get-memory-block-info": {"size": 1024**3},
|
||||
"get-memory-blocks": [{"start": 0, "size": 1024**3}],
|
||||
"get-timezone": {"zone": "UTC", "offset": 0},
|
||||
"get-vcpus": [{"online": True, "can-offline": False}],
|
||||
"fsfreeze-status": "thawed",
|
||||
}
|
||||
if command not in blobs:
|
||||
blobs[command] = defaults.get(command, {})
|
||||
await _save_guest_state(request, resource["id"], state)
|
||||
agent = state.get("agent")
|
||||
blobs = agent.get("results") if isinstance(agent, dict) else None
|
||||
if not isinstance(blobs, dict) or command not in blobs:
|
||||
raise ApiError(404, f"agent command '{command}' has no stored result")
|
||||
return {"result": blobs[command]}
|
||||
|
||||
async def agent_users(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -146,10 +136,10 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
resource = await _agent_resource(request, payload)
|
||||
path = str(payload.get("file") or payload.get("path") or "/etc/hostname")
|
||||
state = _state(resource["state"])
|
||||
files = state.setdefault("agent", {}).setdefault("files", {})
|
||||
if path not in files:
|
||||
files[path] = f"simulated:{path}\n"
|
||||
await _save_guest_state(request, resource["id"], state)
|
||||
agent = state.get("agent")
|
||||
files = agent.get("files") if isinstance(agent, dict) else None
|
||||
if not isinstance(files, dict) or path not in files:
|
||||
raise ApiError(404, "file does not exist")
|
||||
content = str(files[path])
|
||||
return {"result": {"content": content, "truncated": True}}
|
||||
|
||||
@@ -188,9 +178,15 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def agent_fstrim(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
resource = await _agent_resource(request, _values(inputs))
|
||||
state = _state(resource["state"])
|
||||
state.setdefault("agent", {})["last_fstrim"] = True
|
||||
agent = dict(state.get("agent") or {})
|
||||
agent["last_fstrim"] = True
|
||||
state["agent"] = agent
|
||||
await _save_guest_state(request, resource["id"], state)
|
||||
return {"result": {"paths": [{"path": "/", "trimmed": 0}]}}
|
||||
results = agent.get("results") if isinstance(agent.get("results"), dict) else {}
|
||||
fstrim = results.get("fstrim") if isinstance(results, dict) else None
|
||||
if isinstance(fstrim, dict):
|
||||
return {"result": fstrim}
|
||||
return {"result": {}}
|
||||
|
||||
async def agent_set_password(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = _values(inputs)
|
||||
@@ -258,45 +254,23 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
|
||||
async def cloudinit_dump(request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
config = _state(resource["config"])
|
||||
return (
|
||||
f"#cloud-config\nhostname: {config.get('name', values['vmid'])}\n"
|
||||
f"manage_etc_hosts: true\n"
|
||||
)
|
||||
state = _state(resource["state"])
|
||||
dump = state.get("cloudinit_dump")
|
||||
return str(dump) if dump is not None else ""
|
||||
|
||||
async def rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
values = _values(inputs)
|
||||
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
state = _state(resource["state"])
|
||||
rrd_state = state.setdefault("rrd", {"filename": f"pve-vm-{values['vmid']}.rrd"})
|
||||
await _save_guest_state(request, resource["id"], state)
|
||||
return dict(rrd_state)
|
||||
rrd_state = state.get("rrd")
|
||||
return dict(rrd_state) if isinstance(rrd_state, dict) else {}
|
||||
|
||||
async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
values = _values(inputs)
|
||||
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
|
||||
state = _state(resource["state"])
|
||||
series = state.setdefault(
|
||||
"rrddata",
|
||||
[
|
||||
{
|
||||
"time": 1_700_000_000,
|
||||
"cpu": 0.05,
|
||||
"mem": 256 * 1024 * 1024,
|
||||
"netin": 0,
|
||||
"netout": 0,
|
||||
},
|
||||
{
|
||||
"time": 1_700_000_060,
|
||||
"cpu": 0.08,
|
||||
"mem": 260 * 1024 * 1024,
|
||||
"netin": 100,
|
||||
"netout": 80,
|
||||
},
|
||||
],
|
||||
)
|
||||
await _save_guest_state(request, resource["id"], state)
|
||||
return list(series)
|
||||
series = state.get("rrddata")
|
||||
return [dict(item) for item in series] if isinstance(series, list) else []
|
||||
|
||||
async def monitor(request: Request, inputs: dict[str, Any]) -> str:
|
||||
values = _values(inputs)
|
||||
|
||||
@@ -21,65 +21,77 @@ _SECRET_KEYS = frozenset({"key", "token", "fingerprint"})
|
||||
|
||||
|
||||
def _sdn(metadata: dict[str, Any]) -> dict[str, Any]:
|
||||
current = metadata.setdefault(
|
||||
"sdn",
|
||||
current = metadata.get("sdn")
|
||||
return current if isinstance(current, dict) else {}
|
||||
|
||||
|
||||
_VNET_LIST_KEYS = frozenset(
|
||||
{
|
||||
"zones": {},
|
||||
"vnets": {},
|
||||
"controllers": {},
|
||||
"dns": {},
|
||||
"ipams": {},
|
||||
"fabrics": {},
|
||||
"fabric_nodes": {},
|
||||
"prefix_lists": {},
|
||||
"route_maps": {},
|
||||
"lock": None,
|
||||
"pending": False,
|
||||
"running_version": 1,
|
||||
},
|
||||
)
|
||||
if not isinstance(current, dict):
|
||||
current = {
|
||||
"zones": {},
|
||||
"vnets": {},
|
||||
"controllers": {},
|
||||
"dns": {},
|
||||
"ipams": {},
|
||||
"fabrics": {},
|
||||
"fabric_nodes": {},
|
||||
"prefix_lists": {},
|
||||
"route_maps": {},
|
||||
"lock": None,
|
||||
"pending": False,
|
||||
"running_version": 1,
|
||||
"vnet",
|
||||
"type",
|
||||
"zone",
|
||||
"alias",
|
||||
"tag",
|
||||
"vlanaware",
|
||||
"digest",
|
||||
"isolate-ports",
|
||||
"pending",
|
||||
"state",
|
||||
}
|
||||
metadata["sdn"] = current
|
||||
for key in (
|
||||
"zones",
|
||||
"vnets",
|
||||
"controllers",
|
||||
)
|
||||
|
||||
_ZONE_LIST_KEYS = frozenset(
|
||||
{
|
||||
"zone",
|
||||
"type",
|
||||
"bridge",
|
||||
"mtu",
|
||||
"nodes",
|
||||
"ipam",
|
||||
"dns",
|
||||
"ipams",
|
||||
"fabrics",
|
||||
"fabric_nodes",
|
||||
"prefix_lists",
|
||||
"route_maps",
|
||||
):
|
||||
current.setdefault(key, {})
|
||||
return current
|
||||
"reversedns",
|
||||
"vlan-protocol",
|
||||
"digest",
|
||||
"pending",
|
||||
"state",
|
||||
"comment",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _public(item: dict[str, Any]) -> dict[str, Any]:
|
||||
return {key: value for key, value in item.items() if key not in _SECRET_KEYS}
|
||||
|
||||
|
||||
def _vnet_list_item(name: str, item: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = {key: value for key, value in item.items() if key in _VNET_LIST_KEYS}
|
||||
payload["vnet"] = str(payload.get("vnet") or name)
|
||||
payload.setdefault("type", "vnet")
|
||||
return _public(payload)
|
||||
|
||||
|
||||
def _zone_list_item(name: str, item: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = {key: value for key, value in item.items() if key in _ZONE_LIST_KEYS}
|
||||
payload["zone"] = str(payload.get("zone") or name)
|
||||
return _public(payload)
|
||||
|
||||
|
||||
def _store_list(store: dict[str, Any], *, id_key: str) -> list[dict[str, Any]]:
|
||||
if id_key == "vnet":
|
||||
return [_vnet_list_item(name, item) for name, item in sorted(store.items())]
|
||||
if id_key == "zone":
|
||||
return [_zone_list_item(name, item) for name, item in sorted(store.items())]
|
||||
return [_public({id_key: name, **item}) for name, item in sorted(store.items())]
|
||||
|
||||
|
||||
async def _load(request: Request) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
async def _load(
|
||||
request: Request, *, for_write: bool = False
|
||||
) -> tuple[dict[str, Any], dict[str, Any]]:
|
||||
metadata = await cluster_metadata(request)
|
||||
return metadata, _sdn(metadata)
|
||||
sdn = _sdn(metadata)
|
||||
if for_write and metadata.get("sdn") is not sdn:
|
||||
metadata["sdn"] = sdn
|
||||
return metadata, sdn
|
||||
|
||||
|
||||
def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
@@ -98,9 +110,11 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
"zones",
|
||||
)
|
||||
|
||||
async def apply(request: Request, inputs: dict[str, Any]) -> None:
|
||||
async def apply(request: Request, inputs: dict[str, Any]) -> str:
|
||||
from app.handlers.cluster_extra import _cluster_task
|
||||
|
||||
payload = values(inputs)
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
lock = sdn.get("lock")
|
||||
token = payload.get("lock-token")
|
||||
if lock and token and lock.get("token") != token:
|
||||
@@ -110,19 +124,20 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
if payload.get("release-lock"):
|
||||
sdn["lock"] = None
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return await _cluster_task(request, task_type="sdn-apply", worker="sdnapply")
|
||||
|
||||
async def lock_create(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
metadata, sdn = await _load(request)
|
||||
async def lock_create(request: Request, inputs: dict[str, Any]) -> str:
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
if sdn.get("lock") and not values(inputs).get("allow-pending"):
|
||||
raise ApiError(400, "SDN is already locked")
|
||||
token = secrets.token_hex(8)
|
||||
sdn["lock"] = {"token": token}
|
||||
await save_cluster_metadata(request, metadata)
|
||||
return {"digest": token, "token": token}
|
||||
return token
|
||||
|
||||
async def lock_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
lock = sdn.get("lock")
|
||||
if lock is None:
|
||||
return None
|
||||
@@ -133,14 +148,14 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def rollback(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
sdn["pending"] = False
|
||||
if payload.get("release-lock"):
|
||||
sdn["lock"] = None
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def dry_run(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
return [
|
||||
{"type": "zone", "name": name, "action": "noop"}
|
||||
for name in sorted(sdn.get("zones") or {})
|
||||
@@ -154,7 +169,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
create_required: str | None = None,
|
||||
) -> None:
|
||||
async def list_items(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
items = _store_list(sdn.get(store_key) or {}, id_key=id_param)
|
||||
type_filter = values(inputs).get("type")
|
||||
if type_filter:
|
||||
@@ -164,7 +179,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
item_id = str(payload[create_required or id_param])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault(store_key, {})
|
||||
if item_id in store:
|
||||
raise ApiError(400, f"{store_key} '{item_id}' already exists")
|
||||
@@ -179,7 +194,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
item_id = str(values(inputs)[id_param])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get(store_key) or {}).get(item_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, f"{store_key} entry does not exist")
|
||||
@@ -188,7 +203,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
item_id = str(payload[id_param])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault(store_key, {})
|
||||
if item_id not in store:
|
||||
raise ApiError(404, f"{store_key} entry does not exist")
|
||||
@@ -208,7 +223,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
item_id = str(values(inputs)[id_param])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault(store_key, {})
|
||||
if item_id not in store:
|
||||
raise ApiError(404, f"{store_key} entry does not exist")
|
||||
@@ -230,20 +245,20 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def ipam_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
ipam = str(values(inputs)["ipam"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
if ipam not in (sdn.get("ipams") or {}):
|
||||
raise ApiError(404, "ipam does not exist")
|
||||
return {"status": "ok", "ipam": ipam}
|
||||
|
||||
# vnets + nested
|
||||
async def vnets_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
return _store_list(sdn.get("vnets") or {}, id_key="vnet")
|
||||
|
||||
async def vnets_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("vnets", {})
|
||||
if vnet in store:
|
||||
raise ApiError(400, f"vnet '{vnet}' already exists")
|
||||
@@ -259,7 +274,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def vnet_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
vnet = str(values(inputs)["vnet"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("vnets") or {}).get(vnet)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "vnet does not exist")
|
||||
@@ -268,7 +283,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def vnet_update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("vnets", {})
|
||||
if vnet not in store:
|
||||
raise ApiError(404, "vnet does not exist")
|
||||
@@ -288,7 +303,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def vnet_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
vnet = str(values(inputs)["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("vnets", {})
|
||||
if vnet not in store:
|
||||
raise ApiError(404, "vnet does not exist")
|
||||
@@ -307,7 +322,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def subnets_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
vnet = str(values(inputs)["vnet"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
return _store_list(item.get("subnets") or {}, id_key="subnet")
|
||||
|
||||
@@ -315,7 +330,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
subnet = str(payload["subnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
subnets = item.setdefault("subnets", {})
|
||||
if subnet in subnets:
|
||||
@@ -331,7 +346,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
subnet = str(payload["subnet"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
data = (item.get("subnets") or {}).get(subnet)
|
||||
if not isinstance(data, dict):
|
||||
@@ -342,7 +357,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
subnet = str(payload["subnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
subnets = item.setdefault("subnets", {})
|
||||
if subnet not in subnets:
|
||||
@@ -365,7 +380,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
subnet = str(payload["subnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
subnets = item.setdefault("subnets", {})
|
||||
if subnet not in subnets:
|
||||
@@ -377,7 +392,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def ips_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
ips = item.setdefault("ips", [])
|
||||
if not isinstance(ips, list):
|
||||
@@ -398,7 +413,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def ips_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
ips = item.setdefault("ips", [])
|
||||
if not isinstance(ips, list):
|
||||
@@ -416,14 +431,14 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def fw_options_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
vnet = str(values(inputs)["vnet"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
return dict(item.get("firewall", {}).get("options") or {"enable": 0})
|
||||
|
||||
async def fw_options_put(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
options = dict(item.setdefault("firewall", {}).setdefault("options", {"enable": 0}))
|
||||
for key, value in payload.items():
|
||||
@@ -435,7 +450,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def fw_rules_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
vnet = str(values(inputs)["vnet"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
rules = item.get("firewall", {}).get("rules") or []
|
||||
return list(rules) if isinstance(rules, list) else []
|
||||
@@ -443,7 +458,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def fw_rules_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
rules = item.setdefault("firewall", {}).setdefault("rules", [])
|
||||
if not isinstance(rules, list):
|
||||
@@ -464,7 +479,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
pos = int(payload["pos"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
rules = item.setdefault("firewall", {}).setdefault("rules", [])
|
||||
if not isinstance(rules, list) or pos < 0 or pos >= len(rules):
|
||||
@@ -479,7 +494,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
vnet = str(payload["vnet"])
|
||||
pos = int(payload["pos"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = await _vnet(sdn, vnet)
|
||||
rules = item.setdefault("firewall", {}).setdefault("rules", [])
|
||||
if not isinstance(rules, list) or pos < 0 or pos >= len(rules):
|
||||
@@ -492,7 +507,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
return subdirs("all", "fabric", "node")
|
||||
|
||||
async def fabrics_all(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
return _store_list(sdn.get("fabrics") or {}, id_key="id")
|
||||
|
||||
async def fabric_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
@@ -501,7 +516,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def fabric_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
fabric_id = str(payload["id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("fabrics", {})
|
||||
if fabric_id in store:
|
||||
raise ApiError(400, f"fabric '{fabric_id}' already exists")
|
||||
@@ -514,7 +529,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def fabric_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
fabric_id = str(values(inputs)["id"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("fabrics") or {}).get(fabric_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "fabric does not exist")
|
||||
@@ -523,7 +538,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def fabric_update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
fabric_id = str(payload["id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("fabrics", {})
|
||||
if fabric_id not in store:
|
||||
raise ApiError(404, "fabric does not exist")
|
||||
@@ -543,7 +558,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def fabric_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
fabric_id = str(values(inputs)["id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("fabrics", {})
|
||||
if fabric_id not in store:
|
||||
raise ApiError(404, "fabric does not exist")
|
||||
@@ -554,7 +569,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
await save_cluster_metadata(request, metadata)
|
||||
|
||||
async def fabric_nodes_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
fabric_id = values(inputs).get("fabric_id")
|
||||
nodes = sdn.get("fabric_nodes") or {}
|
||||
result: list[dict[str, Any]] = []
|
||||
@@ -571,7 +586,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
fabric_id = str(payload["fabric_id"])
|
||||
node_id = str(payload["node_id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
if fabric_id not in (sdn.get("fabrics") or {}):
|
||||
raise ApiError(404, "fabric does not exist")
|
||||
store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {})
|
||||
@@ -588,7 +603,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
fabric_id = str(payload["fabric_id"])
|
||||
node_id = str(payload["node_id"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = ((sdn.get("fabric_nodes") or {}).get(fabric_id) or {}).get(node_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "fabric node does not exist")
|
||||
@@ -598,7 +613,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
fabric_id = str(payload["fabric_id"])
|
||||
node_id = str(payload["node_id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {})
|
||||
if node_id not in store:
|
||||
raise ApiError(404, "fabric node does not exist")
|
||||
@@ -620,7 +635,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
fabric_id = str(payload["fabric_id"])
|
||||
node_id = str(payload["node_id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {})
|
||||
if node_id not in store:
|
||||
raise ApiError(404, "fabric node does not exist")
|
||||
@@ -630,13 +645,13 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
# prefix lists
|
||||
async def prefix_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
return _store_list(sdn.get("prefix_lists") or {}, id_key="id")
|
||||
|
||||
async def prefix_create(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
list_id = str(payload["id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("prefix_lists", {})
|
||||
if list_id in store:
|
||||
raise ApiError(400, f"prefix-list '{list_id}' already exists")
|
||||
@@ -654,7 +669,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def prefix_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
list_id = str(values(inputs)["id"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("prefix_lists") or {}).get(list_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -663,7 +678,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def prefix_update(request: Request, inputs: dict[str, Any]) -> None:
|
||||
payload = values(inputs)
|
||||
list_id = str(payload["id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("prefix_lists", {})
|
||||
if list_id not in store:
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -676,7 +691,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def prefix_delete(request: Request, inputs: dict[str, Any]) -> None:
|
||||
list_id = str(values(inputs)["id"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
store = sdn.setdefault("prefix_lists", {})
|
||||
if list_id not in store:
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -686,7 +701,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def prefix_entries(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
list_id = str(values(inputs)["id"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("prefix_lists") or {}).get(list_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -699,7 +714,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
list_id = str(payload["id"])
|
||||
seq = str(payload.get("seq") or secrets.randbelow(10000))
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("prefix_lists") or {}).get(list_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -720,7 +735,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
list_id = str(payload["id"])
|
||||
seq = str(payload["url_seq"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("prefix_lists") or {}).get(list_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -733,7 +748,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
list_id = str(payload["id"])
|
||||
seq = str(payload["url_seq"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("prefix_lists") or {}).get(list_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -752,7 +767,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
list_id = str(payload["id"])
|
||||
seq = str(payload["url_seq"])
|
||||
metadata, sdn = await _load(request)
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("prefix_lists") or {}).get(list_id)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "prefix-list does not exist")
|
||||
@@ -767,16 +782,44 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
async def route_maps_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
|
||||
return subdirs("entries")
|
||||
|
||||
def _route_map_entries(item: object) -> dict[str, Any]:
|
||||
if not isinstance(item, dict):
|
||||
return {}
|
||||
nested = item.get("entries")
|
||||
if isinstance(nested, dict):
|
||||
return {
|
||||
str(key): value
|
||||
for key, value in nested.items()
|
||||
if isinstance(value, dict) and str(key).isdigit()
|
||||
}
|
||||
return {
|
||||
str(key): value
|
||||
for key, value in item.items()
|
||||
if isinstance(value, dict) and str(key).isdigit()
|
||||
}
|
||||
|
||||
def _route_map_entries_mutable(sdn: dict[str, Any], map_id: str) -> dict[str, Any]:
|
||||
bucket = sdn.setdefault("route_maps", {}).setdefault(map_id, {})
|
||||
if not isinstance(bucket, dict):
|
||||
bucket = {}
|
||||
sdn["route_maps"][map_id] = bucket
|
||||
if "entries" in bucket or "name" in bucket:
|
||||
entries = bucket.setdefault("entries", {})
|
||||
if not isinstance(entries, dict):
|
||||
entries = {}
|
||||
bucket["entries"] = entries
|
||||
return entries
|
||||
return bucket
|
||||
|
||||
async def route_entries_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
maps = sdn.get("route_maps") or {}
|
||||
route_map_id = values(inputs).get("route-map-id")
|
||||
result: list[dict[str, Any]] = []
|
||||
for map_id, entries in sorted(maps.items()):
|
||||
for map_id, raw_entries in sorted(maps.items()):
|
||||
if route_map_id and map_id != route_map_id:
|
||||
continue
|
||||
if not isinstance(entries, dict):
|
||||
continue
|
||||
entries = _route_map_entries(raw_entries)
|
||||
for order, entry in sorted(entries.items(), key=lambda pair: int(pair[0])):
|
||||
result.append({"route-map-id": map_id, "order": int(order), **entry})
|
||||
return result
|
||||
@@ -785,8 +828,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
map_id = str(payload["route-map-id"])
|
||||
order = str(payload.get("order") or 10)
|
||||
metadata, sdn = await _load(request)
|
||||
entries = sdn.setdefault("route_maps", {}).setdefault(map_id, {})
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
entries = _route_map_entries_mutable(sdn, map_id)
|
||||
if order in entries:
|
||||
raise ApiError(400, f"route-map entry '{order}' already exists")
|
||||
entries[order] = {
|
||||
@@ -809,8 +852,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
map_id = str(payload["route-map-id"])
|
||||
order = str(payload["order"])
|
||||
_metadata, sdn = await _load(request)
|
||||
entry = ((sdn.get("route_maps") or {}).get(map_id) or {}).get(order)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
entry = _route_map_entries((sdn.get("route_maps") or {}).get(map_id)).get(order)
|
||||
if not isinstance(entry, dict):
|
||||
raise ApiError(404, "route-map entry does not exist")
|
||||
return {"route-map-id": map_id, "order": int(order), **entry}
|
||||
@@ -819,8 +862,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
map_id = str(payload["route-map-id"])
|
||||
order = str(payload["order"])
|
||||
metadata, sdn = await _load(request)
|
||||
entries = sdn.setdefault("route_maps", {}).setdefault(map_id, {})
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
entries = _route_map_entries_mutable(sdn, map_id)
|
||||
if order not in entries:
|
||||
raise ApiError(404, "route-map entry does not exist")
|
||||
current = dict(entries[order])
|
||||
@@ -837,8 +880,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
map_id = str(payload["route-map-id"])
|
||||
order = str(payload["order"])
|
||||
metadata, sdn = await _load(request)
|
||||
entries = sdn.setdefault("route_maps", {}).setdefault(map_id, {})
|
||||
metadata, sdn = await _load(request, for_write=True)
|
||||
entries = _route_map_entries_mutable(sdn, map_id)
|
||||
if order not in entries:
|
||||
raise ApiError(404, "route-map entry does not exist")
|
||||
del entries[order]
|
||||
@@ -852,13 +895,13 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def node_zones(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
return _store_list(sdn.get("zones") or {}, id_key="zone")
|
||||
|
||||
async def node_zone(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
zone = str(values(inputs)["zone"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
item = (sdn.get("zones") or {}).get(zone)
|
||||
if not isinstance(item, dict):
|
||||
raise ApiError(404, "zone does not exist")
|
||||
@@ -866,13 +909,16 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def node_zone_bridges(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
zone = await node_zone(request, inputs)
|
||||
bridge = zone.get("bridge") or f"vmbr-{zone.get('zone')}"
|
||||
return [{"iface": bridge, "active": 1}]
|
||||
bridges = zone.get("bridges")
|
||||
if isinstance(bridges, list):
|
||||
return [dict(item) for item in bridges if isinstance(item, dict)]
|
||||
bridge = zone.get("bridge")
|
||||
return [{"iface": bridge, "active": 1}] if bridge else []
|
||||
|
||||
async def node_zone_content(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
zone = str(values(inputs)["zone"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
return [
|
||||
{"vnet": name, **item}
|
||||
for name, item in sorted((sdn.get("vnets") or {}).items())
|
||||
@@ -881,7 +927,11 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def node_zone_ip_vrf(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
zone = await node_zone(request, inputs)
|
||||
return {"zone": zone.get("zone"), "vrf": f"vrf-{zone.get('zone')}", "table": 100}
|
||||
return {
|
||||
"zone": zone.get("zone"),
|
||||
"vrf": zone.get("vrf") or "",
|
||||
"table": zone.get("table") if zone.get("table") is not None else 0,
|
||||
}
|
||||
|
||||
async def node_vnet(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -889,7 +939,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
|
||||
async def node_vnet_mac_vrf(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
vnet = await node_vnet(request, inputs)
|
||||
return {"vnet": vnet.get("vnet"), "mac-vrf": f"macvrf-{vnet.get('vnet')}"}
|
||||
return {"vnet": vnet.get("vnet"), "mac-vrf": vnet.get("mac-vrf") or ""}
|
||||
|
||||
async def node_fabric(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
@@ -901,7 +951,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
fabric = str(values(inputs)["fabric"])
|
||||
_metadata, sdn = await _load(request)
|
||||
_metadata, sdn = await _load(request, for_write=True)
|
||||
nodes = (sdn.get("fabric_nodes") or {}).get(fabric) or {}
|
||||
result = []
|
||||
for node_id, item in nodes.items():
|
||||
@@ -919,15 +969,23 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
|
||||
fabric = str(values(inputs)["fabric"])
|
||||
_metadata, sdn = await _load(request)
|
||||
nodes = (sdn.get("fabric_nodes") or {}).get(fabric) or {}
|
||||
return [{"node": node_id, "state": "up"} for node_id in sorted(nodes)]
|
||||
return [
|
||||
{
|
||||
"node": node_id,
|
||||
"state": item.get("state") if isinstance(item, dict) else "",
|
||||
}
|
||||
for node_id, item in sorted(nodes.items())
|
||||
]
|
||||
|
||||
async def node_fabric_routes(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
await require_node(request, str(values(inputs)["node"]))
|
||||
fabric = str(values(inputs)["fabric"])
|
||||
_metadata, sdn = await _load(request)
|
||||
item = (sdn.get("fabrics") or {}).get(fabric) or {}
|
||||
prefix = item.get("ip_prefix") or "10.0.0.0/24"
|
||||
return [{"dst": prefix, "protocol": item.get("protocol") or "ospf"}]
|
||||
routes = item.get("routes") if isinstance(item, dict) else None
|
||||
if isinstance(routes, list):
|
||||
return [dict(route) for route in routes if isinstance(route, dict)]
|
||||
return []
|
||||
|
||||
# registrations
|
||||
registry.register("/cluster/sdn", "GET", index)
|
||||
|
||||
@@ -9,7 +9,15 @@ from fastapi import Request
|
||||
|
||||
from app.api.errors import ApiError
|
||||
from app.api.registry import HandlerRegistry
|
||||
from app.handlers.common import database, require_node, state, storage_payload, subdirs, values
|
||||
from app.handlers.common import (
|
||||
database,
|
||||
require_node,
|
||||
require_value,
|
||||
state,
|
||||
storage_payload,
|
||||
subdirs,
|
||||
values,
|
||||
)
|
||||
from app.simulation.seed import CLUSTER_ID, stable_id
|
||||
|
||||
|
||||
@@ -140,6 +148,7 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
JOIN resources r ON r.id = s.resource_id
|
||||
JOIN nodes n ON n.id = r.node_id
|
||||
WHERE n.name=$1 OR s.shared = true
|
||||
OR s.storage_id IN ('local', 'local-lvm')
|
||||
ORDER BY s.storage_id""",
|
||||
node,
|
||||
)
|
||||
@@ -286,7 +295,55 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
if status != "DELETE 1":
|
||||
raise ApiError(404, "volume does not exist")
|
||||
|
||||
async def node_storage_upload(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
async def node_storage_allocate(request: Request, inputs: dict[str, Any]) -> str:
|
||||
from app.handlers.nodes import _node_task
|
||||
|
||||
node = str(values(inputs)["node"])
|
||||
storage_id = str(values(inputs)["storage"])
|
||||
await require_node(request, node)
|
||||
await _storage_row(request, None, storage_id)
|
||||
payload = values(inputs)
|
||||
filename = str(payload.get("filename") or f"vm-{payload.get('vmid', '0')}-disk-0")
|
||||
content_type = str(payload.get("content") or "images")
|
||||
size_raw = str(payload.get("size") or "1G")
|
||||
# Accept either integer bytes or Proxmox size strings like 32G.
|
||||
try:
|
||||
size = int(size_raw)
|
||||
except ValueError:
|
||||
from app.handlers.common import parse_size_bytes
|
||||
|
||||
try:
|
||||
size = parse_size_bytes(size_raw)
|
||||
except ValueError as error:
|
||||
raise ApiError(400, f"invalid size '{size_raw}'") from error
|
||||
format_name = str(payload.get("format") or "raw")
|
||||
volume_id = f"{storage_id}:{content_type}/{filename}"
|
||||
resource_id = await database(request).pool.fetchval(
|
||||
"SELECT resource_id FROM storages WHERE storage_id=$1",
|
||||
storage_id,
|
||||
)
|
||||
await database(request).pool.execute(
|
||||
"""INSERT INTO storage_contents(
|
||||
id, storage_resource_id, volume_id, content_type, size_bytes, metadata
|
||||
) VALUES(gen_random_uuid(), $1, $2, $3, $4, $5::jsonb)
|
||||
ON CONFLICT (storage_resource_id, volume_id) DO UPDATE
|
||||
SET size_bytes=EXCLUDED.size_bytes,
|
||||
content_type=EXCLUDED.content_type,
|
||||
metadata=EXCLUDED.metadata""",
|
||||
resource_id,
|
||||
volume_id,
|
||||
content_type,
|
||||
size,
|
||||
json.dumps(
|
||||
{"filename": filename, "format": format_name, "source": "allocate"},
|
||||
sort_keys=True,
|
||||
),
|
||||
)
|
||||
return await _node_task(request, node=node, task_type="storage-allocate", worker="imgcopy")
|
||||
|
||||
async def node_storage_upload(request: Request, inputs: dict[str, Any]) -> str:
|
||||
from app.handlers.nodes import _node_task
|
||||
|
||||
node = str(values(inputs)["node"])
|
||||
storage_id = str(values(inputs)["storage"])
|
||||
await require_node(request, node)
|
||||
@@ -318,11 +375,11 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
size,
|
||||
json.dumps({"filename": filename, "source": "upload"}, sort_keys=True),
|
||||
)
|
||||
return {"uploadid": volume_id, "filename": filename, "size": size, "volid": volume_id}
|
||||
return await _node_task(request, node=node, task_type="storage-upload", worker="download")
|
||||
|
||||
async def node_storage_prunebackups(
|
||||
request: Request, inputs: dict[str, Any]
|
||||
) -> list[dict[str, Any]] | None:
|
||||
) -> list[dict[str, Any]] | str:
|
||||
node = str(values(inputs)["node"])
|
||||
storage_id = str(values(inputs)["storage"])
|
||||
await require_node(request, node)
|
||||
@@ -332,6 +389,8 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
storage_id,
|
||||
)
|
||||
if request.method == "DELETE":
|
||||
from app.handlers.nodes import _node_task
|
||||
|
||||
keep = int(values(inputs).get("keep-last") or values(inputs).get("keep_last") or 1)
|
||||
await database(request).pool.execute(
|
||||
"""DELETE FROM backups
|
||||
@@ -344,7 +403,9 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
resource_id,
|
||||
keep,
|
||||
)
|
||||
return None
|
||||
return await _node_task(
|
||||
request, node=node, task_type="prune-backups", worker="prunebackups"
|
||||
)
|
||||
rows = await database(request).pool.fetch(
|
||||
"""SELECT volume_id, size_bytes, created_at FROM backups
|
||||
WHERE storage_resource_id=$1 ORDER BY created_at DESC""",
|
||||
@@ -485,18 +546,29 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
async def file_restore_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
await _storage_row(request, None, str(payload["storage"]))
|
||||
row = await _storage_row(request, None, str(payload["storage"]))
|
||||
filepath = str(payload.get("filepath") or "/")
|
||||
return [{"filepath": filepath.rstrip("/") + "/etc", "type": "d", "text": "etc"}]
|
||||
config = state(row["config"])
|
||||
restore = config.get("file_restore")
|
||||
if not isinstance(restore, dict):
|
||||
return []
|
||||
items = restore.get(filepath) or restore.get(filepath.rstrip("/") or "/")
|
||||
return [dict(item) for item in items] if isinstance(items, list) else []
|
||||
|
||||
async def file_restore_download(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
await _storage_row(request, None, str(payload["storage"]))
|
||||
row = await _storage_row(request, None, str(payload["storage"]))
|
||||
config = state(row["config"])
|
||||
downloads = config.get("file_restore_downloads")
|
||||
filepath = str(payload.get("filepath") or "/")
|
||||
if isinstance(downloads, dict) and filepath in downloads:
|
||||
entry = downloads[filepath]
|
||||
return dict(entry) if isinstance(entry, dict) else {"filepath": filepath}
|
||||
return {
|
||||
"download-url": f"/api2/json/nodes/{payload['node']}/storage/"
|
||||
f"{payload['storage']}/file-restore/download",
|
||||
"filepath": payload.get("filepath") or "/",
|
||||
"filepath": filepath,
|
||||
"volume": payload.get("volume"),
|
||||
}
|
||||
|
||||
@@ -504,40 +576,51 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
row = await _storage_row(request, None, str(payload["storage"]))
|
||||
config = state(row["config"])
|
||||
identity = config.get("identity")
|
||||
if isinstance(identity, dict):
|
||||
return {
|
||||
"storage": str(row["storage_id"]),
|
||||
"type": str(identity.get("type") or row["storage_type"]),
|
||||
"fingerprint": str(identity.get("fingerprint") or ""),
|
||||
}
|
||||
return {
|
||||
"storage": str(row["storage_id"]),
|
||||
"type": str(row["storage_type"]),
|
||||
"fingerprint": f"sim-{row['storage_id']}",
|
||||
"fingerprint": "",
|
||||
}
|
||||
|
||||
async def import_metadata(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
storage_id = str(payload["storage"])
|
||||
volume = str(payload["volume"])
|
||||
await _storage_row(request, None, storage_id)
|
||||
return {
|
||||
"type": "qemu",
|
||||
"source": volume,
|
||||
"disks": {"scsi0": f"{storage_id}:0/vm-import.raw"},
|
||||
"net0": "virtio,bridge=vmbr0",
|
||||
}
|
||||
volume = str(require_value(payload, "volume"))
|
||||
row = await _storage_row(request, None, storage_id)
|
||||
config = state(row["config"])
|
||||
by_volume = config.get("import_metadata_by_volume")
|
||||
if isinstance(by_volume, dict) and volume in by_volume:
|
||||
meta = by_volume[volume]
|
||||
return dict(meta) if isinstance(meta, dict) else {}
|
||||
meta = config.get("import_metadata")
|
||||
if isinstance(meta, dict):
|
||||
return {"source": volume, **meta}
|
||||
return {}
|
||||
|
||||
async def storage_rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
storage_id = str(payload["storage"])
|
||||
await _storage_row(request, None, storage_id)
|
||||
return {"filename": f"pve-storage-{storage_id}.rrd"}
|
||||
row = await _storage_row(request, None, str(payload["storage"]))
|
||||
config = state(row["config"])
|
||||
rrd_state = config.get("rrd")
|
||||
return dict(rrd_state) if isinstance(rrd_state, dict) else {}
|
||||
|
||||
async def storage_rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
|
||||
payload = values(inputs)
|
||||
await require_node(request, str(payload["node"]))
|
||||
await _storage_row(request, None, str(payload["storage"]))
|
||||
return [
|
||||
{"time": 1_700_000_000, "used": 10, "total": 100},
|
||||
{"time": 1_700_000_060, "used": 12, "total": 100},
|
||||
]
|
||||
row = await _storage_row(request, None, str(payload["storage"]))
|
||||
config = state(row["config"])
|
||||
series = config.get("rrddata")
|
||||
return [dict(item) for item in series] if isinstance(series, list) else []
|
||||
|
||||
registry.register("/storage", "GET", storage_ids)
|
||||
registry.register("/storage", "POST", storage_create)
|
||||
@@ -548,7 +631,7 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
|
||||
registry.register("/nodes/{node}/storage/{storage}", "GET", node_storage_index)
|
||||
registry.register("/nodes/{node}/storage/{storage}/status", "GET", node_storage_status)
|
||||
registry.register("/nodes/{node}/storage/{storage}/content", "GET", node_storage_content)
|
||||
registry.register("/nodes/{node}/storage/{storage}/content", "POST", node_storage_upload)
|
||||
registry.register("/nodes/{node}/storage/{storage}/content", "POST", node_storage_allocate)
|
||||
registry.register(
|
||||
"/nodes/{node}/storage/{storage}/content/{volume}", "GET", node_storage_content_get
|
||||
)
|
||||
|
||||
@@ -25,6 +25,7 @@ from app.tasks.lxc import lxc_handler
|
||||
from app.tasks.qemu import qemu_handler
|
||||
from app.tasks.repository import TaskRepository
|
||||
from app.tasks.worker import TaskWorker
|
||||
from app.version import get_app_version
|
||||
from app.web.routes import router as web_router
|
||||
|
||||
|
||||
@@ -68,12 +69,15 @@ def create_app(
|
||||
"qemu-stop": qemu,
|
||||
"qemu-suspend": qemu,
|
||||
"qemu-update": qemu,
|
||||
"qemu-resize": qemu,
|
||||
"qemu-template": qemu,
|
||||
"lxc-clone": lxc,
|
||||
"lxc-create": lxc,
|
||||
"lxc-delete": lxc,
|
||||
"lxc-migrate": lxc,
|
||||
"lxc-reboot": lxc,
|
||||
"lxc-resume": lxc,
|
||||
"lxc-resize": lxc,
|
||||
"lxc-shutdown": lxc,
|
||||
"lxc-snapshot-create": lxc,
|
||||
"lxc-snapshot-delete": lxc,
|
||||
@@ -83,6 +87,29 @@ def create_app(
|
||||
"lxc-suspend": lxc,
|
||||
"vzdump": backup,
|
||||
"aptupdate": backup,
|
||||
"network-reload": backup,
|
||||
"disk-initgpt": backup,
|
||||
"disk-wipe": backup,
|
||||
"disk-directory-create": backup,
|
||||
"disk-directory-delete": backup,
|
||||
"disk-lvm-create": backup,
|
||||
"disk-lvm-delete": backup,
|
||||
"disk-lvmthin-create": backup,
|
||||
"disk-lvmthin-delete": backup,
|
||||
"disk-zfs-create": backup,
|
||||
"disk-zfs-delete": backup,
|
||||
"service-start": backup,
|
||||
"service-stop": backup,
|
||||
"service-restart": backup,
|
||||
"service-reload": backup,
|
||||
"storage-allocate": backup,
|
||||
"storage-upload": backup,
|
||||
"prune-backups": backup,
|
||||
"acme": backup,
|
||||
"sdn-apply": backup,
|
||||
"ceph-flags": backup,
|
||||
"cluster-create": backup,
|
||||
"cluster-join": backup,
|
||||
},
|
||||
concurrency=resolved.task_worker_concurrency,
|
||||
lease_seconds=resolved.task_lease_seconds,
|
||||
@@ -91,7 +118,7 @@ def create_app(
|
||||
resolved_workers = (task_worker,)
|
||||
app = FastAPI(
|
||||
title=resolved.app_name,
|
||||
version="0.1.0",
|
||||
version=get_app_version(),
|
||||
openapi_tags=openapi_tag_metadata(),
|
||||
lifespan=create_lifespan(resolved, database_factory, resolved_workers or ()),
|
||||
)
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""Canonical Ceph cluster flag names (PVE /cluster/ceph/flags)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
# Keep in sync with Proxmox `ceph_flags` API (majors 6-9).
|
||||
CEPH_FLAG_DESCRIPTIONS: dict[str, str] = {
|
||||
"nobackfill": "Pause backfill operations",
|
||||
"nodeep-scrub": "Disable deep scrubbing",
|
||||
"nodown": "Do not mark OSDs down",
|
||||
"noin": "Do not mark OSDs in",
|
||||
"noout": "Do not mark OSDs out",
|
||||
"norebalance": "Pause rebalance",
|
||||
"norecover": "Pause recovery",
|
||||
"noscrub": "Disable scrubbing",
|
||||
"notieragent": "Pause tiering agent",
|
||||
"noup": "Do not mark OSDs up",
|
||||
"pause": "Pause I/O",
|
||||
}
|
||||
|
||||
|
||||
def default_ceph_flags(*, enabled: frozenset[str] | None = None) -> dict[str, int]:
|
||||
"""Return the full flag map; unset flags are ``0`` (PVE GET → ``false``)."""
|
||||
|
||||
active = enabled or frozenset()
|
||||
return {name: (1 if name in active else 0) for name in CEPH_FLAG_DESCRIPTIONS}
|
||||
@@ -27,7 +27,7 @@ from app.contracts.model import Method, Snapshot
|
||||
from app.db.migrations import migrate
|
||||
from app.db.pool import AsyncpgDatabase
|
||||
from app.main import create_app
|
||||
from app.simulation.seed import apply_seed, small_profile
|
||||
from app.simulation.seed import apply_seed, lab_profile
|
||||
from app.web.contract_catalog import get_major_releases
|
||||
|
||||
_BUNDLED_9 = Path(
|
||||
@@ -119,7 +119,7 @@ async def prepare_db(url: str) -> None:
|
||||
connection = await asyncpg.connect(url)
|
||||
try:
|
||||
await migrate(connection)
|
||||
await apply_seed(connection, small_profile())
|
||||
await apply_seed(connection, lab_profile())
|
||||
finally:
|
||||
await connection.close()
|
||||
|
||||
@@ -159,16 +159,16 @@ async def probe_major(
|
||||
verb = method.verb.upper()
|
||||
url = f"/api2/json{render_path(path_template)}"
|
||||
headers = {"CSRFPreventionToken": csrf} if verb != "GET" else {}
|
||||
body = body_for(method, path_template) if verb in {"PUT", "POST"} else None
|
||||
payload = body_for(method, path_template)
|
||||
try:
|
||||
if verb == "GET":
|
||||
response = await client.get(url, headers=headers)
|
||||
response = await client.get(url, headers=headers, params=payload or None)
|
||||
elif verb == "PUT":
|
||||
response = await client.put(url, data=body or {}, headers=headers)
|
||||
response = await client.put(url, data=payload or {}, headers=headers)
|
||||
elif verb == "POST":
|
||||
response = await client.post(url, data=body or {}, headers=headers)
|
||||
response = await client.post(url, data=payload or {}, headers=headers)
|
||||
elif verb == "DELETE":
|
||||
response = await client.delete(url, headers=headers)
|
||||
response = await client.request("DELETE", url, data=payload or {}, headers=headers)
|
||||
else:
|
||||
continue
|
||||
except Exception as exc:
|
||||
|
||||
@@ -43,6 +43,11 @@ def backup_handler(repository: TaskRepository, clock: Clock) -> TaskHandler:
|
||||
await repository.append_log(task.id, "apt update finished")
|
||||
return {"status": "OK"}
|
||||
|
||||
# Node/cluster ops whose durable side effects were applied by the API handler.
|
||||
if task.task_type != "vzdump":
|
||||
await repository.append_log(task.id, f"{task.task_type} completed")
|
||||
return {"status": "OK"}
|
||||
|
||||
node = str(task.payload["node"])
|
||||
vmids = [str(item) for item in task.payload.get("vmids", [])]
|
||||
storage_id = str(task.payload.get("storage") or "nfs-backup")
|
||||
|
||||
@@ -29,6 +29,9 @@ def lxc_handler(repository: TaskRepository, clock: Clock) -> TaskHandler:
|
||||
)
|
||||
if operation == "migrate" or operation == "remote-migrate":
|
||||
return await _migrate(repository, task, resource_id, clock)
|
||||
if operation == "resize":
|
||||
await repository.append_log(task.id, "container resize completed")
|
||||
return {"status": "OK"}
|
||||
async with repository.pool.acquire() as connection:
|
||||
row = await connection.fetchrow("SELECT state FROM resources WHERE id=$1", resource_id)
|
||||
if row is None:
|
||||
|
||||
@@ -33,6 +33,10 @@ def qemu_handler(repository: TaskRepository, clock: Clock) -> TaskHandler:
|
||||
return await _migrate(repository, task, resource_id, clock)
|
||||
if operation == "move-disk":
|
||||
return await _move_disk(repository, task, resource_id)
|
||||
if operation in {"resize", "template"}:
|
||||
# Side effects already persisted by the API handler; complete the worker.
|
||||
await repository.append_log(task.id, f"VM {operation} completed")
|
||||
return {"status": "OK"}
|
||||
async with repository.pool.acquire() as connection:
|
||||
row = await connection.fetchrow("SELECT state FROM resources WHERE id=$1", resource_id)
|
||||
if row is None:
|
||||
@@ -62,10 +66,12 @@ def qemu_handler(repository: TaskRepository, clock: Clock) -> TaskHandler:
|
||||
|
||||
|
||||
async def _create(repository: TaskRepository, task: Task) -> dict[str, Any]:
|
||||
from app.simulation.seed import enrich_guest_state
|
||||
|
||||
node, vmid = str(task.payload["node"]), int(task.payload["vmid"])
|
||||
config = dict(task.payload.get("config", {}))
|
||||
resource_id = uuid.uuid4()
|
||||
state = {"status": "stopped", **config}
|
||||
state = enrich_guest_state({"status": "stopped", **config}, kind="qemu", vmid=str(vmid))
|
||||
async with repository.pool.acquire() as connection:
|
||||
async with connection.transaction():
|
||||
node_row = await connection.fetchrow(
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
"""SemVer source of truth: VERSION file (synced to pyproject + Helm)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
|
||||
_VERSION_FILE = Path(__file__).resolve().parents[1] / "VERSION"
|
||||
_PROJECT_ROOT = _VERSION_FILE.parent
|
||||
_PYPROJECT_FILE = _PROJECT_ROOT / "pyproject.toml"
|
||||
_CHART_FILE = _PROJECT_ROOT / "helm" / "proxmox-api-simulator" / "Chart.yaml"
|
||||
_VALUES_FILE = _PROJECT_ROOT / "helm" / "proxmox-api-simulator" / "values.yaml"
|
||||
_DEFAULT_VERSION = "0.1.0"
|
||||
_VERSION_RE = re.compile(r"^(\d+)\.(\d+)\.(\d+)$")
|
||||
|
||||
|
||||
def _read_version_file(path: Path) -> str | None:
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8").strip()
|
||||
except OSError:
|
||||
return None
|
||||
if _VERSION_RE.match(text):
|
||||
return text
|
||||
return None
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_app_version() -> str:
|
||||
for path in (
|
||||
Path("/app/VERSION"),
|
||||
_VERSION_FILE,
|
||||
Path.cwd() / "VERSION",
|
||||
):
|
||||
found = _read_version_file(path)
|
||||
if found:
|
||||
return found
|
||||
try:
|
||||
from importlib.metadata import version
|
||||
|
||||
meta = version("proxmox-api-simulator")
|
||||
if _VERSION_RE.match(meta):
|
||||
return meta
|
||||
except Exception:
|
||||
pass
|
||||
return _DEFAULT_VERSION
|
||||
|
||||
|
||||
def get_app_version_label() -> str:
|
||||
clear_version_cache()
|
||||
return f"v{get_app_version()}"
|
||||
|
||||
|
||||
def parse_version(value: str) -> tuple[int, int, int]:
|
||||
match = _VERSION_RE.match(value.strip())
|
||||
if not match:
|
||||
raise ValueError(f"Invalid semantic version: {value!r}")
|
||||
return int(match.group(1)), int(match.group(2)), int(match.group(3))
|
||||
|
||||
|
||||
def format_version(major: int, minor: int, patch: int) -> str:
|
||||
return f"{major}.{minor}.{patch}"
|
||||
|
||||
|
||||
def clear_version_cache() -> None:
|
||||
get_app_version.cache_clear()
|
||||
|
||||
|
||||
def write_project_version(version: str) -> None:
|
||||
parse_version(version)
|
||||
_VERSION_FILE.write_text(f"{version}\n", encoding="utf-8")
|
||||
|
||||
pyproject = _PYPROJECT_FILE.read_text(encoding="utf-8")
|
||||
pyproject, count = re.subn(
|
||||
r'^version = ".*"$',
|
||||
f'version = "{version}"',
|
||||
pyproject,
|
||||
count=1,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
if count != 1:
|
||||
raise RuntimeError("Failed to update pyproject.toml version")
|
||||
_PYPROJECT_FILE.write_text(pyproject, encoding="utf-8")
|
||||
|
||||
chart = _CHART_FILE.read_text(encoding="utf-8")
|
||||
chart, chart_count = re.subn(
|
||||
r"^version: .*$",
|
||||
f"version: {version}",
|
||||
chart,
|
||||
count=1,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
chart, app_count = re.subn(
|
||||
r'^appVersion: ".*"$',
|
||||
f'appVersion: "{version}"',
|
||||
chart,
|
||||
count=1,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
if chart_count != 1 or app_count != 1:
|
||||
raise RuntimeError("Failed to update Chart.yaml version")
|
||||
_CHART_FILE.write_text(chart, encoding="utf-8")
|
||||
|
||||
values = _VALUES_FILE.read_text(encoding="utf-8")
|
||||
values, values_count = re.subn(
|
||||
r'^(\s*tag:\s*).*$',
|
||||
rf'\1"{version}"',
|
||||
values,
|
||||
count=1,
|
||||
flags=re.MULTILINE,
|
||||
)
|
||||
if values_count != 1:
|
||||
raise RuntimeError("Failed to update helm values.yaml image.tag")
|
||||
_VALUES_FILE.write_text(values, encoding="utf-8")
|
||||
|
||||
clear_version_cache()
|
||||
|
||||
|
||||
def bump_patch_version() -> str:
|
||||
major, minor, patch = parse_version(get_app_version())
|
||||
version = format_version(major, minor, patch + 1)
|
||||
write_project_version(version)
|
||||
return version
|
||||
|
||||
|
||||
def bump_minor_version() -> str:
|
||||
major, minor, patch = parse_version(get_app_version())
|
||||
version = format_version(major, minor + 1, 0)
|
||||
write_project_version(version)
|
||||
return version
|
||||
@@ -11,7 +11,7 @@ from pathlib import Path
|
||||
|
||||
from app.api.openapi import contract_openapi_tag
|
||||
from app.config import Settings
|
||||
from app.contracts.examples import path_param_example, schema_example
|
||||
from app.contracts.examples import path_param_example, schema_example, wire_param_name
|
||||
from app.contracts.importer import RemoteSourceImporter
|
||||
from app.contracts.model import Method, Parameter, Snapshot
|
||||
from app.contracts.normalize import normalize_snapshot
|
||||
@@ -102,8 +102,10 @@ def list_majors(
|
||||
*,
|
||||
runtime_version: str | None,
|
||||
settings: Settings | None = None,
|
||||
app_version: str | None = None,
|
||||
) -> dict[str, object]:
|
||||
return {
|
||||
"app_version": app_version,
|
||||
"runtime_version": runtime_version,
|
||||
"majors": [
|
||||
{
|
||||
@@ -226,16 +228,25 @@ def _path_param_names(path: str) -> tuple[str, ...]:
|
||||
return tuple(match.group(1) for match in _PATH_PARAM.finditer(path))
|
||||
|
||||
|
||||
def _parameter_payload(parameter: Parameter) -> dict[str, object]:
|
||||
def _parameter_payload(parameter: Parameter, *, wire_name: str | None = None) -> dict[str, object]:
|
||||
schema = parameter.definition
|
||||
return {
|
||||
"name": parameter.name,
|
||||
name = wire_name or parameter.name
|
||||
typetext = schema.extra.get("typetext")
|
||||
payload: dict[str, object] = {
|
||||
"name": name,
|
||||
"type": schema.type,
|
||||
"description": schema.description,
|
||||
"optional": bool(schema.optional),
|
||||
"enum": list(schema.enum),
|
||||
"example": schema_example(schema, name=parameter.name),
|
||||
}
|
||||
if wire_name and wire_name != parameter.name:
|
||||
payload["template"] = parameter.name
|
||||
if isinstance(schema.format, str):
|
||||
payload["format"] = schema.format
|
||||
if isinstance(typetext, str) and typetext:
|
||||
payload["typetext"] = typetext
|
||||
return payload
|
||||
|
||||
|
||||
def method_payload(
|
||||
@@ -274,15 +285,20 @@ def method_payload(
|
||||
"example": path_param_example(name) or name,
|
||||
}
|
||||
)
|
||||
body_fields = [
|
||||
contract_body_fields = [
|
||||
_parameter_payload(parameter)
|
||||
for parameter in method.parameters
|
||||
if parameter.name not in path_params and "[n]" not in parameter.name
|
||||
]
|
||||
# Contract uses foo[n]; the wire form is foo0..fooN — seed the console with foo0.
|
||||
indexed_fields = [
|
||||
_parameter_payload(parameter) for parameter in method.parameters if "[n]" in parameter.name
|
||||
_parameter_payload(parameter, wire_name=wire_param_name(parameter.name))
|
||||
for parameter in method.parameters
|
||||
if "[n]" in parameter.name
|
||||
]
|
||||
body_example = _body_example(method, path_params)
|
||||
# Flatten nested scalars from body_example (and nested field examples) into PARAMS.
|
||||
body_fields = _body_fields_with_nested(contract_body_fields, body_example)
|
||||
resolved_path = _resolve_path(path, path_fields)
|
||||
implemented = (
|
||||
(path, method.verb.upper()) in implemented_methods
|
||||
@@ -316,21 +332,133 @@ def _resolve_path(path: str, path_fields: list[dict[str, object]]) -> str:
|
||||
|
||||
|
||||
def _body_example(method: Method, path_params: tuple[str, ...]) -> dict[str, object]:
|
||||
"""Minimal request body: required parameters only, Proxmox wire names/values."""
|
||||
|
||||
body: dict[str, object] = {}
|
||||
for parameter in method.parameters:
|
||||
if parameter.name in path_params:
|
||||
continue
|
||||
if "[n]" in parameter.name:
|
||||
concrete = parameter.name.replace("[n]", "0")
|
||||
if not parameter.definition.optional:
|
||||
body[concrete] = schema_example(parameter.definition, name=concrete)
|
||||
continue
|
||||
if parameter.definition.optional:
|
||||
continue
|
||||
body[parameter.name] = schema_example(parameter.definition, name=parameter.name)
|
||||
wire = wire_param_name(parameter.name)
|
||||
body[wire] = schema_example(parameter.definition, name=parameter.name)
|
||||
return body
|
||||
|
||||
|
||||
def _leaf_type(value: object) -> str:
|
||||
if isinstance(value, bool):
|
||||
return "boolean"
|
||||
if isinstance(value, int) and not isinstance(value, bool):
|
||||
return "integer"
|
||||
if isinstance(value, float):
|
||||
return "number"
|
||||
return "string"
|
||||
|
||||
|
||||
def _body_fields_from_example(body_example: dict[str, object] | None) -> list[dict[str, object]]:
|
||||
"""PARAM inputs derived from body_example, including nested scalar paths."""
|
||||
|
||||
if not isinstance(body_example, dict) or not body_example:
|
||||
return []
|
||||
inner: object = body_example
|
||||
# Soft unwrap a single root object envelope (Engine-style), never a scalar/list.
|
||||
if len(body_example) == 1:
|
||||
only = next(iter(body_example.values()))
|
||||
if isinstance(only, dict):
|
||||
inner = only
|
||||
if not isinstance(inner, dict):
|
||||
return []
|
||||
|
||||
fields: list[dict[str, object]] = []
|
||||
|
||||
def _walk(prefix: str, value: object) -> None:
|
||||
if isinstance(value, dict):
|
||||
for key, child in value.items():
|
||||
path = f"{prefix}.{key}" if prefix else str(key)
|
||||
_walk(path, child)
|
||||
return
|
||||
if isinstance(value, list):
|
||||
for index, child in enumerate(value):
|
||||
path = f"{prefix}.{index}" if prefix else str(index)
|
||||
_walk(path, child)
|
||||
return
|
||||
fields.append(
|
||||
{
|
||||
"name": prefix,
|
||||
"type": _leaf_type(value),
|
||||
"description": prefix,
|
||||
"optional": True,
|
||||
"enum": [],
|
||||
"example": value,
|
||||
}
|
||||
)
|
||||
|
||||
_walk("", inner)
|
||||
return fields
|
||||
|
||||
|
||||
def _body_fields_with_nested(
|
||||
contract_fields: list[dict[str, object]],
|
||||
body_example: dict[str, object],
|
||||
) -> list[dict[str, object]]:
|
||||
"""Merge contract PARAMS with nested scalar paths from body_example."""
|
||||
|
||||
example_fields = _body_fields_from_example(body_example)
|
||||
nested_tops = {
|
||||
str(item["name"]).split(".", 1)[0] for item in example_fields if "." in str(item["name"])
|
||||
}
|
||||
contract_by_name = {str(item["name"]): item for item in contract_fields}
|
||||
merged: list[dict[str, object]] = []
|
||||
seen: set[str] = set()
|
||||
|
||||
for item in example_fields:
|
||||
name = str(item["name"])
|
||||
top = name.split(".", 1)[0]
|
||||
parent = contract_by_name.get(name) or contract_by_name.get(top)
|
||||
row = dict(item)
|
||||
if parent is not None:
|
||||
row["optional"] = bool(parent.get("optional"))
|
||||
description = parent.get("description")
|
||||
if isinstance(description, str) and description:
|
||||
row["description"] = description
|
||||
typetext = parent.get("typetext")
|
||||
if isinstance(typetext, str) and typetext:
|
||||
row["typetext"] = typetext
|
||||
fmt = parent.get("format")
|
||||
if isinstance(fmt, str) and fmt:
|
||||
row["format"] = fmt
|
||||
else:
|
||||
# body_example is required-only — treat uncovered leaves as required.
|
||||
row["optional"] = False
|
||||
merged.append(row)
|
||||
seen.add(name)
|
||||
|
||||
for field in contract_fields:
|
||||
name = str(field["name"])
|
||||
if name in nested_tops or name in seen:
|
||||
continue
|
||||
example = field.get("example")
|
||||
if isinstance(example, dict | list):
|
||||
nested = _body_fields_from_example({name: example})
|
||||
if nested:
|
||||
for item in nested:
|
||||
nested_name = str(item["name"])
|
||||
if nested_name in seen:
|
||||
continue
|
||||
row = dict(item)
|
||||
row["optional"] = bool(field.get("optional"))
|
||||
description = field.get("description")
|
||||
if isinstance(description, str) and description:
|
||||
row["description"] = description
|
||||
merged.append(row)
|
||||
seen.add(nested_name)
|
||||
continue
|
||||
# Empty object/array examples stay as a top-level PARAM.
|
||||
merged.append(field)
|
||||
seen.add(name)
|
||||
return merged
|
||||
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def default_store_root() -> Path:
|
||||
return _DEFAULT_STORE
|
||||
|
||||
@@ -16,6 +16,7 @@ from app.contracts.source import SourceError
|
||||
from app.db.pool import AsyncpgDatabase
|
||||
from app.dependencies import get_database
|
||||
from app.simulation.seed import apply_seed, build_profile, simulation_state_summary
|
||||
from app.version import get_app_version
|
||||
from app.web.assets import console_html
|
||||
from app.web.compatibility_catalog import compatibility_payload
|
||||
from app.web.contract_catalog import catalog_payload, list_majors, load_snapshot, method_payload
|
||||
@@ -37,7 +38,13 @@ async def console() -> HTMLResponse:
|
||||
async def ui_versions(request: Request) -> JSONResponse:
|
||||
settings = _settings(request)
|
||||
runtime_version = _runtime_version(request)
|
||||
return JSONResponse(list_majors(runtime_version=runtime_version, settings=settings))
|
||||
return JSONResponse(
|
||||
list_majors(
|
||||
runtime_version=runtime_version,
|
||||
settings=settings,
|
||||
app_version=get_app_version(),
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@router.get("/ui/api/catalog", include_in_schema=False)
|
||||
@@ -161,7 +168,23 @@ async def ui_demo_state(request: Request) -> JSONResponse:
|
||||
@router.post("/ui/api/demo/load", include_in_schema=False)
|
||||
async def ui_demo_load(request: Request) -> JSONResponse:
|
||||
pool = _database_pool(request)
|
||||
profile = build_profile("demo-cluster")
|
||||
requested = "small"
|
||||
if request.query_params.get("profile"):
|
||||
requested = str(request.query_params.get("profile"))
|
||||
else:
|
||||
try:
|
||||
body = await request.json()
|
||||
except Exception:
|
||||
body = None
|
||||
if isinstance(body, dict) and body.get("profile"):
|
||||
requested = str(body["profile"])
|
||||
allowed = {"small", "large", "big"}
|
||||
if requested not in allowed:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=f"unknown cluster profile '{requested}' (expected: small, large, big)",
|
||||
)
|
||||
profile = build_profile(requested)
|
||||
async with pool.acquire() as connection:
|
||||
await apply_seed(connection, profile)
|
||||
summary = await simulation_state_summary(connection)
|
||||
@@ -179,7 +202,7 @@ async def ui_demo_unload(request: Request) -> JSONResponse:
|
||||
summary = await simulation_state_summary(connection)
|
||||
except Exception as error:
|
||||
raise HTTPException(
|
||||
status_code=500, detail=f"failed to remove demo data: {error}"
|
||||
status_code=500, detail=f"failed to reset to minimal cluster: {error}"
|
||||
) from error
|
||||
return JSONResponse({"ok": True, "profile": profile.name, "summary": summary})
|
||||
|
||||
|
||||
@@ -1,12 +1,28 @@
|
||||
# Quick start with the published Docker Hub runtime image.
|
||||
#
|
||||
# WARNING: Laboratory / CI only.
|
||||
# Default TICKET_SIGNING_KEY and PostgreSQL password are intentional lab
|
||||
# defaults. Do NOT expose host :8006 to untrusted networks without replacing
|
||||
# secrets and adding your own controls. See SECURITY.md and docs/security.md.
|
||||
#
|
||||
# Requires this repository checkout (compose file).
|
||||
#
|
||||
# docker compose -f docker-compose.release.yml up -d
|
||||
# docker compose -f docker-compose.release.yml run --rm --entrypoint python simulator -m app.simulation.seed_cli
|
||||
# docker compose -f docker-compose.release.yml run --rm --entrypoint python \
|
||||
# simulator -m app.simulation.seed_cli
|
||||
# curl -sS http://localhost:8006/health/ready
|
||||
#
|
||||
# Override the image tag:
|
||||
# IMAGE_TAG=0.1.0 docker compose -f docker-compose.release.yml up -d
|
||||
#
|
||||
# Change TICKET_SIGNING_KEY before exposing the stack beyond a local lab.
|
||||
# Rotate lab secrets before any shared or networked demo:
|
||||
# TICKET_SIGNING_KEY=$(openssl rand -hex 32) \
|
||||
# POSTGRES_PASSWORD=$(openssl rand -hex 16) \
|
||||
# docker compose -f docker-compose.release.yml up -d
|
||||
#
|
||||
# Optional HTTPS for proxmoxer-style clients (separate port):
|
||||
# docker compose -f docker-compose.release.yml --profile tls up -d
|
||||
# curl -sk https://localhost:8443/health/ready
|
||||
|
||||
name: proxmox-api-simulator-release
|
||||
|
||||
@@ -18,6 +34,7 @@ x-app-env: &app-env
|
||||
CONTRACT_SNAPSHOT: /app/contracts/pve-9.2.3.json
|
||||
COMPATIBILITY_EVIDENCE: /app/evidence/pve-9.2.3.json
|
||||
LOG_LEVEL: ${LOG_LEVEL:-INFO}
|
||||
# Lab default — replace for any shared or networked use.
|
||||
TICKET_SIGNING_KEY: ${TICKET_SIGNING_KEY:-development-only-signing-key-change-me}
|
||||
TASK_WORKER_CONCURRENCY: ${TASK_WORKER_CONCURRENCY:-2}
|
||||
SIMULATION_TIME_SCALE: ${SIMULATION_TIME_SCALE:-10}
|
||||
@@ -86,3 +103,35 @@ services:
|
||||
start_period: 20s
|
||||
ports:
|
||||
- "${SIMULATOR_PORT:-8006}:8006"
|
||||
|
||||
tls-gateway:
|
||||
profiles: [tls]
|
||||
image: nginx:1.28.0-alpine
|
||||
restart: unless-stopped
|
||||
networks: [simulator]
|
||||
depends_on:
|
||||
simulator:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "${TLS_GATEWAY_PORT:-8443}:8443"
|
||||
volumes:
|
||||
- ./docker/tls/gateway.conf:/etc/nginx/conf.d/default.conf:ro
|
||||
- ./docker/tls/server.crt:/etc/nginx/tls/server.crt:ro
|
||||
- ./docker/tls/server.key:/etc/nginx/tls/server.key:ro
|
||||
healthcheck:
|
||||
test:
|
||||
[
|
||||
"CMD-SHELL",
|
||||
"wget -qO- --no-check-certificate https://127.0.0.1:8443/health/live || exit 1",
|
||||
]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 5
|
||||
start_period: 5s
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /var/cache/nginx
|
||||
- /tmp
|
||||
- /var/run
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
|
||||
@@ -12,8 +12,9 @@ x-dev-env: &dev-env
|
||||
TEST_DATABASE_URL: postgresql://proxmox:proxmox@postgres:5432/proxmox_simulator
|
||||
CONTRACT_SNAPSHOT: /workspace/contracts/e61a893e996d05d376579226e7dfbedbcfce8b71787adacffbc557e6e35901c1/snapshot.json
|
||||
COMPATIBILITY_EVIDENCE: /workspace/evidence/pve-9.2.3.json
|
||||
PROXMOXER_HOST: tls-gateway
|
||||
PROXMOXER_PORT: "8443"
|
||||
# proxmoxer is HTTPS-only — set when using --profile tls (see Makefile test-compatibility)
|
||||
PROXMOXER_HOST: ${PROXMOXER_HOST:-}
|
||||
PROXMOXER_PORT: ${PROXMOXER_PORT:-8443}
|
||||
LOG_LEVEL: INFO
|
||||
TICKET_SIGNING_KEY: development-only-signing-key-change-me
|
||||
|
||||
@@ -108,9 +109,12 @@ services:
|
||||
retries: 8
|
||||
start_period: 20s
|
||||
ports:
|
||||
- "8006:8006"
|
||||
- "${SIMULATOR_PORT:-8006}:8006"
|
||||
|
||||
# Optional HTTPS front for clients that cannot speak plain HTTP (proxmoxer).
|
||||
# Default lab URL is http://localhost:8006/ — K8s TLS is Ingress-only.
|
||||
tls-gateway:
|
||||
profiles: [tls]
|
||||
image: nginx:1.28.0-alpine
|
||||
restart: unless-stopped
|
||||
networks: [simulator]
|
||||
@@ -118,7 +122,7 @@ services:
|
||||
simulator:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "8007:8443"
|
||||
- "${TLS_GATEWAY_PORT:-8443}:8443"
|
||||
volumes:
|
||||
- ./docker/tls/gateway.conf:/etc/nginx/conf.d/default.conf:ro
|
||||
- ./docker/tls/server.crt:/etc/nginx/tls/server.crt:ro
|
||||
|
||||
@@ -1,5 +1,26 @@
|
||||
# Development HTTPS gateway for proxmoxer and other TLS clients.
|
||||
# Upstream hostnames are resolved at request time via Docker embedded DNS.
|
||||
# Optional / internal TLS terminator.
|
||||
#
|
||||
# - Host lab default is plain HTTP :8006 on the simulator (no TLS).
|
||||
# - Kubernetes HTTPS terminates at Ingress (cert-manager).
|
||||
# - This gateway is for HTTPS-only clients (proxmoxer, pulumi-proxmoxve).
|
||||
# - Also listen on :80: some bridged providers rewrite https://host:8443 → http://host
|
||||
# on later requests (delete); without :80 those calls fail.
|
||||
|
||||
server {
|
||||
listen 80;
|
||||
server_name _;
|
||||
resolver 127.0.0.11 valid=10s ipv6=off;
|
||||
location / {
|
||||
set $upstream simulator:8006;
|
||||
proxy_pass http://$upstream;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Request-ID $request_id;
|
||||
}
|
||||
}
|
||||
|
||||
server {
|
||||
listen 8443 ssl;
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
**Language / Язык:** [English](README.md) | [Русский](ru/README.md)
|
||||
|
||||
# Documentation
|
||||
|
||||
Guides for the Proxmox VE API simulator. Switch language with the header on each
|
||||
page. Russian mirrors live under [`ru/`](ru/README.md).
|
||||
|
||||

|
||||
|
||||
| Guide | Description |
|
||||
|---|---|
|
||||
| [Getting started](getting-started.md) | First successful lab session |
|
||||
| [Configuration](configuration.md) | Environment variables and Compose |
|
||||
| [Authentication](authentication.md) | Tickets, CSRF, API tokens, ACLs |
|
||||
| [API versions](api-versions.md) | Contracts 6–9 and hot-swap |
|
||||
| [Clients & examples](clients.md) | Python, Go, Java, Perl, Ansible, Terraform, Pulumi |
|
||||
| [Seed profiles](seed-profiles.md) | Deterministic cluster fixtures |
|
||||
| [API surface](api-surface.md) | Routing, handlers, fallbacks |
|
||||
| [Domains](domains/README.md) | QEMU, LXC, storage, HA, SDN, … |
|
||||
| [Web UI](web-ui.md) | Interactive console and catalogs |
|
||||
| [Operations](operations.md) | Migrate, reseed, upgrade, Hub publish |
|
||||
| [CI/CD (Jenkins)](ci-cd.md) | VERSION, Harbor/Hub push, deploy to proxmox.devops.org.ru |
|
||||
| [Docker Hub overview](docker-hub-overview.md) | Paste-ready Hub repository description |
|
||||
| [Kubernetes / Helm](kubernetes.md) | Hub image + Ingress + Let's Encrypt |
|
||||
| [Security](security.md) | Lab threat model and credentials |
|
||||
| [Observability](observability.md) | Health endpoints and logging |
|
||||
| [Troubleshooting](troubleshooting.md) | Common failure modes |
|
||||
| [FAQ](faq.md) | Short answers |
|
||||
| [Architecture](architecture.md) | Component boundaries |
|
||||
| [Compatibility](compatibility.md) | Evidence model and release matrix |
|
||||
| [Testing](testing.md) | Suites layout, Make targets, latest results |
|
||||
|
||||
Runnable cookbooks: [`examples/`](../examples/README.md).
|
||||
Integration suites: [`pulumi-tests/`](../pulumi-tests/README.md).
|
||||
Test map and latest pass notes: [Testing](testing.md).
|
||||
|
||||
## See also
|
||||
|
||||
- [Web UI](web-ui.md) — interactive console, catalogs, DATA panel, and screenshots
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](api-surface.md) | [Русский](ru/api-surface.md)
|
||||
|
||||
# API surface
|
||||
|
||||
## Request path
|
||||
@@ -33,7 +35,8 @@ not part of the product contract. See the workspace durable-simulator rule.
|
||||
|
||||
- Interactive FastAPI docs: `/docs`
|
||||
- Web UI method inspector: `/` → catalog → method
|
||||
- UI APIs: `/ui/api/catalog`, `/ui/api/method`, `/ui/api/compatibility`
|
||||
- UI APIs: `/ui/api/versions`, `/ui/api/catalog`, `/ui/api/method`,
|
||||
`/ui/api/compatibility`, `/ui/api/contract/apply`, `/ui/api/demo/*`
|
||||
|
||||
## Compatibility endpoints
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](api-versions.md) | [Русский](ru/api-versions.md)
|
||||
|
||||
# API versions (PVE 6–9)
|
||||
|
||||
The simulator ships authoritative imported contracts for four Proxmox VE majors.
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](architecture.md) | [Русский](ru/architecture.md)
|
||||
|
||||
# Architecture
|
||||
|
||||
## Goals
|
||||
@@ -30,7 +32,7 @@ flowchart LR
|
||||
Obs["Logs / Prometheus / OpenTelemetry"]
|
||||
|
||||
Client -->|"/api2/json"| API
|
||||
Admin -->|"CLI and /_simulator"| API
|
||||
Admin -->|"CLI, Make/Helm, Web UI /ui/api"| API
|
||||
Docs -->|"explicit import only"| Importer
|
||||
Importer --> Contract
|
||||
Contract --> DB
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](authentication.md) | [Русский](ru/authentication.md)
|
||||
|
||||
# Authentication
|
||||
|
||||
The simulator implements Proxmox-compatible ticket and API-token authentication
|
||||
@@ -54,8 +56,9 @@ beyond its owner.
|
||||
|
||||
## Seeded development principals
|
||||
|
||||
Loaded by every standard seed profile (unless replaced by UI demo unload →
|
||||
`minimal`):
|
||||
Seeded for **every** profile — including `minimal` and after Web UI demo unload.
|
||||
Unload shrinks guests/nodes/storages; lab principals and tokens are still
|
||||
inserted by `apply_seed`:
|
||||
|
||||
| Principal | Password | Token | Notes |
|
||||
|---|---|---|---|
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
**Language / Язык:** [English](ci-cd.md) | [Русский](ru/ci-cd.md)
|
||||
|
||||
# CI/CD (Jenkins + Gitea)
|
||||
|
||||
SemVer lives in [`VERSION`](../VERSION). Bump locally with `make bump-patch` /
|
||||
`make bump-minor` (also syncs `pyproject.toml` and Helm chart/values). Jenkins
|
||||
does **not** bump — it builds whatever is in the commit.
|
||||
|
||||
Public lab URL after deploy: **https://proxmox.devops.org.ru/**
|
||||
|
||||
## Pipeline
|
||||
|
||||
Single [`Jenkinsfile`](../Jenkinsfile) on `main` / `master`:
|
||||
|
||||
1. **Build & push** (agent `docker` / DinD) — multi-arch `linux/amd64,linux/arm64`
|
||||
to Harbor and Docker Hub with tags **`:{VERSION}`** and **`:latest`** only.
|
||||
2. **Deploy** — `kubectl set image` on `simulators/simulators-proxmox` (main
|
||||
container `simulator` + initContainer `migrate`), then rollout status
|
||||
(`--timeout=1200s`).
|
||||
|
||||
First cluster install (Ingress, Postgres, secrets) is **not** this job — use
|
||||
`make addon-simulators` in DevOpsTools/K3S (`simulators_proxmox_host:
|
||||
proxmox.devops.org.ru`).
|
||||
|
||||
### Images
|
||||
|
||||
| Registry | Repository |
|
||||
|---|---|
|
||||
| Harbor | `hub.antropoff.ru/devops-tools/proxmox-api-simulator` |
|
||||
| Docker Hub | `inecs/proxmox-api-simulator` |
|
||||
|
||||
### Credentials (Jenkins Global)
|
||||
|
||||
| ID | Type | Use |
|
||||
|---|---|---|
|
||||
| `harbor-devops-tools-push-pull-access` | Username/password | Harbor robot push |
|
||||
| `docker-hub` | Username/password | Docker Hub `inecs` |
|
||||
| `k3s-kubeconfig` | Secret file | Kubeconfig for deploy |
|
||||
| `ssh-gitea-key` | SSH private key | Gitea SCM checkout |
|
||||
|
||||
## Gitea setup
|
||||
|
||||
1. Ensure the repo is on Gitea (mirror or primary) with branch `main`.
|
||||
2. Deploy key / Jenkins credential `ssh-gitea-key` can clone the repo
|
||||
(`git@…:…/proxmox_api_simulator.git` or your path).
|
||||
3. Optional: webhook from Gitea → Jenkins Multibranch (or rely on Jenkins SCM
|
||||
polling / Organization Folder).
|
||||
|
||||
## Jenkins setup
|
||||
|
||||
1. **Credentials** — create the four IDs above if missing (same as Wrapped).
|
||||
2. **Agent** — Kubernetes cloud pod template with label `docker` and a
|
||||
`docker` container (DinD / buildx), same as Wrapped.
|
||||
3. **Job** — Multibranch Pipeline (or Pipeline from SCM):
|
||||
- Script Path: `Jenkinsfile`
|
||||
- Branch discover: `main` (and `master` if needed)
|
||||
- SCM: Gitea SSH URL + credential `ssh-gitea-key`
|
||||
4. **Harbor project** — `devops-tools` must allow the robot to push
|
||||
`proxmox-api-simulator` (create the repository on first push or pre-create).
|
||||
5. **Docker Hub** — `inecs/proxmox-api-simulator` exists / push rights for
|
||||
credential `docker-hub`.
|
||||
6. **Cluster** — release `simulators` in namespace `simulators` already
|
||||
installed (`make addon-simulators`). Deployment name must be
|
||||
`simulators-proxmox`.
|
||||
7. Run once on `main` after bumping `VERSION` and pushing. Confirm:
|
||||
- Harbor/Hub tags `:0.x.y` and `:latest`
|
||||
- `kubectl -n simulators get deploy,ing`
|
||||
- Help → About shows badge `v0.x.y` on https://proxmox.devops.org.ru/
|
||||
|
||||
## Local release (without Jenkins)
|
||||
|
||||
```bash
|
||||
make bump-patch # or bump-minor
|
||||
make version-commit
|
||||
git push origin HEAD
|
||||
# or manual Hub-only:
|
||||
make release # DOCKER_IMAGE=inecs/proxmox-api-simulator
|
||||
```
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](clients.md) | [Русский](ru/clients.md)
|
||||
|
||||
# Clients
|
||||
|
||||
Use the simulator from common automation stacks. Each cookbook aims for the
|
||||
@@ -13,39 +15,28 @@ same laboratory flow where the tool allows it:
|
||||
|
||||
## Connection matrix
|
||||
|
||||
| Stack | Transport | Notes | Docs | Code |
|
||||
Real Proxmox VE clients talk to **HTTPS `:8006`**. This lab’s Compose stack
|
||||
publishes plain **HTTP `:8006`** (same port number). HTTPS belongs on
|
||||
**Kubernetes Ingress** (cert-manager). Clients that cannot speak HTTP
|
||||
(proxmoxer) use the optional profile: `docker compose --profile tls` →
|
||||
`https://localhost:8443/` (see [Ports and TLS](configuration.md#ports-and-tls)).
|
||||
|
||||
| Stack | Compose transport | Notes | Docs | Code |
|
||||
|---|---|---|---|---|
|
||||
| Python (proxmoxer) | HTTPS `:8007` | Unmodified library; `verify_ssl=False` for local cert | [guide](examples/python-proxmoxer.md) | [`examples/python`](../examples/python) |
|
||||
| Python (proxmoxer) | HTTPS `:8443` (`--profile tls`) | HTTPS-only library; `verify_ssl=False` for lab cert | [guide](examples/python-proxmoxer.md) | [`examples/python`](../examples/python) |
|
||||
| Python (requests) | HTTP `:8006` | Raw `/api2/json` | [guide](examples/python-requests.md) | [`examples/python`](../examples/python) |
|
||||
| Go | HTTP `:8006` | stdlib `net/http` | [guide](examples/go.md) | [`examples/go`](../examples/go) |
|
||||
| Java | HTTP `:8006` | Java 11+ `HttpClient` | [guide](examples/java.md) | [`examples/java`](../examples/java) |
|
||||
| Perl | HTTP `:8006` | `HTTP::Tiny` + JSON | [guide](examples/perl.md) | [`examples/perl`](../examples/perl) |
|
||||
| Ansible | HTTP `:8006` | `uri` module cookbook | [guide](examples/ansible.md) | [`examples/ansible`](../examples/ansible) |
|
||||
| Terraform | HTTPS `:8007` | Provider + insecure TLS for local gateway | [guide](examples/terraform.md) | [`examples/terraform`](../examples/terraform) |
|
||||
| Pulumi | HTTPS `:8007` | Python program against the API | [guide](examples/pulumi.md) | [`examples/pulumi`](../examples/pulumi) |
|
||||
| Terraform | HTTP `:8006` (or TLS `:8443`) | Prefer HTTP; use `insecure` only with `--profile tls` | [guide](examples/terraform.md) | [`examples/terraform`](../examples/terraform) |
|
||||
| Pulumi | HTTP `:8006` | `pulumi-proxmoxve` or HTTP cookbooks | [guide](examples/pulumi.md) | [`examples/pulumi`](../examples/pulumi) |
|
||||
|
||||
Shared prerequisites: [examples overview](examples/overview.md).
|
||||
On Kubernetes with Ingress + cert-manager, point every client at
|
||||
`https://<your-host>/` instead.
|
||||
|
||||
## Credentials (seed)
|
||||
## More
|
||||
|
||||
| Use | Value |
|
||||
|---|---|
|
||||
| User | `root@pam` |
|
||||
| Password | `secret` |
|
||||
| Token | `root@pam!automation=automation-secret` |
|
||||
| Default node (`small`) | `pve01` |
|
||||
|
||||
## API major
|
||||
|
||||
Pin the major before long runs:
|
||||
|
||||
- Cold start: `CONTRACT_SNAPSHOT`
|
||||
- Runtime: Web UI apply or `POST /ui/api/contract/apply?major=N`
|
||||
|
||||
Confirm with `GET /api2/json/version`. Coverage is **100%** for declared methods
|
||||
on majors 6–9.
|
||||
|
||||
## Troubleshooting clients
|
||||
|
||||
See [troubleshooting-clients](examples/troubleshooting-clients.md) and the
|
||||
global [Troubleshooting](troubleshooting.md) guide.
|
||||
- Cookbooks index: [examples/overview.md](examples/overview.md)
|
||||
- Troubleshooting: [examples/troubleshooting-clients.md](examples/troubleshooting-clients.md)
|
||||
- Pulumi full suite: [`pulumi-tests/`](../pulumi-tests/README.md)
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](compatibility-0.1.0.md) | [Русский](ru/compatibility-0.1.0.md)
|
||||
|
||||
# Compatibility report — 0.1.0
|
||||
|
||||
This report records evidence for simulator release 0.1.0 against the bundled
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](compatibility.md) | [Русский](ru/compatibility.md)
|
||||
|
||||
# Compatibility
|
||||
|
||||
This document explains how the simulator claims compatibility with Proxmox VE
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](configuration.md) | [Русский](ru/configuration.md)
|
||||
|
||||
# Configuration
|
||||
|
||||
Application settings are loaded from the environment (see `.env.example`).
|
||||
@@ -53,14 +55,41 @@ any accidental gap surfaces as HTTP 501.
|
||||
| `SEED_LARGE_NODES` | Node count for `large` |
|
||||
| `SEED_LARGE_RESOURCES` | Guest count for `large` (default 10 000) |
|
||||
| `TEST_DATABASE_URL` | Integration-test DSN |
|
||||
| `PROXMOXER_HOST` / `PROXMOXER_PORT` | Compatibility test client target (`tls-gateway` / `8443` in Compose) |
|
||||
| `PROXMOXER_HOST` / `PROXMOXER_PORT` | Compatibility test client target with `--profile tls` (`tls-gateway` / `8443`) |
|
||||
|
||||
## Ports and TLS
|
||||
|
||||
### Real Proxmox VE (reference)
|
||||
|
||||
On a physical / production PVE node the management API listens on **HTTPS
|
||||
`:8006`** (`/api2/json/...`). Related management ports (not separate REST APIs):
|
||||
|
||||
| Port | Protocol | Role |
|
||||
|---|---|---|
|
||||
| `8006` | TCP, HTTPS | Web UI + REST API |
|
||||
| `3128` | TCP | SPICE proxy (graphical console) |
|
||||
| `5900–5999` | TCP (WebSocket) | VNC web console |
|
||||
| `22` | TCP | SSH / cluster actions |
|
||||
| `5405–5412` | UDP | Corosync cluster traffic |
|
||||
|
||||
Port **`8007`** is **not** the PVE API — it is the usual Proxmox Backup Server
|
||||
(PBS) management port. Do not point PVE clients at `:8007` on real hardware.
|
||||
|
||||
### Simulator lab endpoints
|
||||
|
||||
| Endpoint | Use |
|
||||
|---|---|
|
||||
| `http://localhost:8006` | Direct HTTP (curl, browsers, most examples) |
|
||||
| `https://localhost:8007` | TLS gateway for TLS-assuming clients (proxmoxer, etc.) |
|
||||
| `http://localhost:8006` | Primary client URL — simulator (curl, browsers, requests, Terraform, …) |
|
||||
| `https://localhost:8443` | Optional — `docker compose --profile tls` for proxmoxer-style HTTPS-only clients |
|
||||
|
||||
Compose publishes plain **HTTP on host `:8006`** (same port number as real PVE,
|
||||
which uses HTTPS). The simulator process speaks HTTP on `:8006` inside the Docker
|
||||
network as well. For Kubernetes, TLS terminates at Ingress (cert-manager). Host
|
||||
**`:8007` is not used** for the lab API (on real hardware that port is typically
|
||||
PBS, not PVE).
|
||||
|
||||
Optional Compose TLS: `docker compose --profile tls` starts an nginx gateway on
|
||||
host `:8443` (requires `docker/tls/`). It proxies to `simulator:8006`.
|
||||
|
||||
The checked-in certificate under `docker/tls/` is disposable development
|
||||
material. Never reuse it outside local labs. See [Security](security.md).
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
# Docker Hub overview (paste into Hub)
|
||||
|
||||
Copy the block below into the **Full description** of
|
||||
[`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator)
|
||||
so Hub wording matches GitHub (stateful simulator — not a thin mock).
|
||||
|
||||
---
|
||||
|
||||
**proxmox-api-simulator** — stateful asynchronous Proxmox VE API simulator for
|
||||
labs and CI. PostgreSQL-backed mutations, durable UPIDs, official API contracts
|
||||
for PVE 6–9, and the same `/api2/json` surface clients already speak.
|
||||
|
||||
**Laboratory / CI only.** Default credentials and signing keys are intentional
|
||||
lab defaults. Do **not** expose this image to the public Internet without
|
||||
replacing secrets and adding your own network controls.
|
||||
|
||||
### Quick start
|
||||
|
||||
```bash
|
||||
# from a git checkout (needs docker-compose.release.yml + docker/tls/)
|
||||
docker compose -f docker-compose.release.yml up -d
|
||||
docker compose -f docker-compose.release.yml run --rm --entrypoint python \
|
||||
simulator -m app.simulation.seed_cli
|
||||
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
curl -sS http://localhost:8006/api2/json/version
|
||||
```
|
||||
|
||||
- HTTP API + Web UI: `http://localhost:8006/`
|
||||
- Optional HTTPS for proxmoxer: `docker compose --profile tls` → `https://localhost:8443/`
|
||||
- Seeded admin: `root@pam` / `secret`
|
||||
- Source & docs: https://github.com/sergeyantropoff/proxmox-api-simulator
|
||||
- Helm chart: `helm/proxmox-api-simulator` in the same repository
|
||||
|
||||
### Tags
|
||||
|
||||
| Tag | Meaning |
|
||||
|---|---|
|
||||
| `0.1.0`, `…` | Immutable release from `pyproject.toml` / `make release` |
|
||||
| `latest` | Most recent `make release` (when `PUSH_LATEST=1`) |
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](README.md) | [Русский](../ru/domains/README.md)
|
||||
|
||||
# Domain guides
|
||||
|
||||
These pages summarize durable semantics by area. For exhaustive method lists,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](access.md) | [Русский](../ru/domains/access.md)
|
||||
|
||||
# Access
|
||||
|
||||
Durable identity and authorization: users, groups, roles, ACL entries, realms,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](ceph.md) | [Русский](../ru/domains/ceph.md)
|
||||
|
||||
# Ceph
|
||||
|
||||
Ceph-related API paths persist simulated cluster, pool, OSD, and monitor state.
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](cluster-extras.md) | [Русский](../ru/domains/cluster-extras.md)
|
||||
|
||||
# Cluster extras
|
||||
|
||||
Additional cluster-scoped domains with durable handlers:
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](core-cluster.md) | [Русский](../ru/domains/core-cluster.md)
|
||||
|
||||
# Core & cluster
|
||||
|
||||
## Version
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](firewall.md) | [Русский](../ru/domains/firewall.md)
|
||||
|
||||
# Firewall
|
||||
|
||||
Cluster, node, and guest firewall configuration — rules, aliases, IP sets,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](ha.md) | [Русский](../ru/domains/ha.md)
|
||||
|
||||
# HA
|
||||
|
||||
High-availability groups, resources, status, and rules persist in cluster
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](lxc.md) | [Русский](../ru/domains/lxc.md)
|
||||
|
||||
# LXC
|
||||
|
||||
Container APIs mirror the QEMU lifecycle patterns where the contract declares
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](pools.md) | [Русский](../ru/domains/pools.md)
|
||||
|
||||
# Pools
|
||||
|
||||
Pool CRUD and resource membership are fully covered and durable. The `medium`
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](qemu.md) | [Русский](../ru/domains/qemu.md)
|
||||
|
||||
# QEMU
|
||||
|
||||
Full contract surface for QEMU guests on the active major, including:
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](sdn.md) | [Русский](../ru/domains/sdn.md)
|
||||
|
||||
# SDN
|
||||
|
||||
Software-defined networking handlers cover declared zones, VNets, subnets,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](storage-backup.md) | [Русский](../ru/domains/storage-backup.md)
|
||||
|
||||
# Storage & backup
|
||||
|
||||
## Storage
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](tasks.md) | [Русский](../ru/domains/tasks.md)
|
||||
|
||||
# Tasks
|
||||
|
||||
Long-running operations return a Proxmox-style **UPID**. Task rows, events,
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](ansible.md) | [Русский](../ru/examples/ansible.md)
|
||||
|
||||
# Ansible
|
||||
|
||||
Playbook uses the `uri` module against HTTP `:8006` with token auth, then
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](go.md) | [Русский](../ru/examples/go.md)
|
||||
|
||||
# Go
|
||||
|
||||
Uses the Go standard library against `http://localhost:8006` with API-token
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](java.md) | [Русский](../ru/examples/java.md)
|
||||
|
||||
# Java
|
||||
|
||||
Java 11+ `HttpClient` cookbook using API-token auth against `:8006`.
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](overview.md) | [Русский](../ru/examples/overview.md)
|
||||
|
||||
# Client examples overview
|
||||
|
||||
## Bring-up checklist
|
||||
@@ -20,7 +22,7 @@ curl -s -X POST 'http://localhost:8006/ui/api/contract/apply?major=8'
|
||||
| URL | When |
|
||||
|---|---|
|
||||
| `http://localhost:8006` | curl, Go, Java, Perl, Ansible, requests |
|
||||
| `https://localhost:8007` | proxmoxer, many Terraform/Pulumi TLS clients |
|
||||
| `http://localhost:8006` | proxmoxer, many Terraform/Pulumi TLS clients |
|
||||
|
||||
## Auth quick reference
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](perl.md) | [Русский](../ru/examples/perl.md)
|
||||
|
||||
# Perl
|
||||
|
||||
`HTTP::Tiny` + JSON cookbook with API-token auth.
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](pulumi.md) | [Русский](../ru/examples/pulumi.md)
|
||||
|
||||
# Pulumi
|
||||
|
||||
Python Pulumi program that drives the simulator over HTTPS using token auth via
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](python-proxmoxer.md) | [Русский](../ru/examples/python-proxmoxer.md)
|
||||
|
||||
# Python — proxmoxer
|
||||
|
||||
Canonical library path against the HTTPS gateway.
|
||||
@@ -11,7 +13,7 @@ python examples/python/proxmoxer_cookbook.py
|
||||
```
|
||||
|
||||
Environment overrides: `PVE_HOST` (default `localhost`), `PVE_PORT` (default
|
||||
`8007`), `PVE_USER`, `PVE_PASSWORD`, or token via `PVE_TOKEN_NAME` /
|
||||
`8006`), `PVE_USER`, `PVE_PASSWORD`, or token via `PVE_TOKEN_NAME` /
|
||||
`PVE_TOKEN_VALUE`.
|
||||
|
||||
## Notes
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](python-requests.md) | [Русский](../ru/examples/python-requests.md)
|
||||
|
||||
# Python — requests
|
||||
|
||||
Raw HTTP against `:8006` without proxmoxer.
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
**Language / Язык:** [English](terraform.md) | [Русский](../ru/examples/terraform.md)
|
||||
|
||||
# Terraform
|
||||
|
||||
Example uses a Proxmox provider pointed at the local HTTPS gateway
|
||||
(`https://localhost:8007`) with `insecure = true` for the development
|
||||
(`http://localhost:8006`) with `insecure = true` for the development
|
||||
certificate.
|
||||
|
||||
```bash
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
**Language / Язык:** [English](troubleshooting-clients.md) | [Русский](../ru/examples/troubleshooting-clients.md)
|
||||
|
||||
# Troubleshooting clients
|
||||
|
||||
| Symptom | Fix |
|
||||
|---|---|
|
||||
| TLS certificate errors | Use `:8007` with verify disabled **only** locally, or use HTTP `:8006` |
|
||||
| TLS certificate errors | Compose default is `http://localhost:8006` (no TLS). For proxmoxer use `docker compose --profile tls` and `https://localhost:8443` with verify disabled **only** locally (`curl -sk`, `verify_ssl=False`, `insecure=true`). On Kubernetes use your Ingress hostname with cert-manager TLS. |
|
||||
| CSRF failure | Send `CSRFPreventionToken` with ticket mutations; prefer token auth in scripts |
|
||||
| Node not found | `small` seed uses `pve01` |
|
||||
| 403 on power | You may be using `auditor@pve` / readonly token — switch to root or operator |
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](faq.md) | [Русский](ru/faq.md)
|
||||
|
||||
# FAQ
|
||||
|
||||
## Is this a real Proxmox hypervisor?
|
||||
@@ -14,7 +16,10 @@ See [API versions](api-versions.md) and [Compatibility](compatibility.md).
|
||||
## Can I use this in CI for Terraform / Ansible / custom clients?
|
||||
|
||||
Yes. That is a primary use case. Pin the API major, seed a profile, and point
|
||||
clients at HTTP `:8006` or HTTPS `:8007`. See [Clients](clients.md).
|
||||
clients at **HTTP `:8006`** (Compose) or your Ingress **HTTPS** hostname on
|
||||
Kubernetes. See [Clients](clients.md). For the
|
||||
Pulumi surface suite see [`pulumi-tests/`](../pulumi-tests/README.md).
|
||||
Full test map and latest recorded results: [Testing](testing.md).
|
||||
|
||||
## Why do some OpenID / LDAP / ACME / Ceph calls “succeed” without remotes?
|
||||
|
||||
@@ -40,4 +45,16 @@ Hub image. Ingress + cert-manager Let's Encrypt is supported — see
|
||||
|
||||
## Which node name does the small seed use?
|
||||
|
||||
`pve01`.
|
||||
`pve01`. Profiles `medium` and `ha-demo` use **`pve1` / `pve2` / `pve3`**.
|
||||
|
||||
## What ports does real Proxmox VE use vs this simulator?
|
||||
|
||||
Real PVE serves the Web UI and REST API on **HTTPS `:8006`** only. Related
|
||||
management ports include SPICE `:3128`, VNC `:5900–5999`, SSH `:22`, and
|
||||
Corosync UDP `:5405–5412`. Port `:8007` on real hardware is typically
|
||||
**Proxmox Backup Server**, not PVE.
|
||||
|
||||
This lab publishes plain **HTTP `:8006`** in Compose (same port number as real
|
||||
PVE). HTTPS belongs on Kubernetes Ingress. Optional proxmoxer TLS:
|
||||
`docker compose --profile tls` on `:8443`. Host `:8007` is **not** used. Details:
|
||||
[Ports and TLS](configuration.md#ports-and-tls).
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
**Language / Язык:** [English](getting-started.md) | [Русский](ru/getting-started.md)
|
||||
|
||||
# Getting started
|
||||
|
||||
Bring up a local laboratory cluster, authenticate, and exercise a first
|
||||
@@ -17,15 +19,18 @@ Python toolchain for day-to-day use.
|
||||
|---|---|
|
||||
| [Published image](#1a-published-image-docker-hub) | Fastest lab using `inecs/proxmox-api-simulator` |
|
||||
| [Helm / Kubernetes](kubernetes.md) | Cluster install with Ingress + Let's Encrypt |
|
||||
| [Development checkout](#1b-development-checkout) | Contribute / bind-mount source / HTTPS gateway on `:8007` |
|
||||
| [Development checkout](#1b-development-checkout) | Contribute / bind-mount source / HTTP API on `:8006` |
|
||||
|
||||
## 1a. Published image (Docker Hub)
|
||||
|
||||
Uses [`docker-compose.release.yml`](../docker-compose.release.yml) — PostgreSQL +
|
||||
runtime simulator from Hub. No source build required.
|
||||
|
||||
> Laboratory / CI only — rotate `TICKET_SIGNING_KEY` and the DB password before
|
||||
> any shared or networked demo. See [SECURITY.md](../SECURITY.md).
|
||||
|
||||
```bash
|
||||
# from this repository, or download docker-compose.release.yml alone
|
||||
# from this repository (compose file + docker/tls/)
|
||||
docker compose -f docker-compose.release.yml pull
|
||||
docker compose -f docker-compose.release.yml up -d
|
||||
docker compose -f docker-compose.release.yml run --rm --entrypoint python \
|
||||
@@ -47,7 +52,7 @@ make release-seed PROFILE=small
|
||||
|
||||
| Host port | Service |
|
||||
|---|---|
|
||||
| `8006` | HTTP API + Web UI |
|
||||
| `8006` | HTTP API + Web UI (same port as real PVE; real PVE uses HTTPS) |
|
||||
| `5432` | PostgreSQL (localhost only) |
|
||||
|
||||
Migrations run automatically via the `migrate` one-shot service.
|
||||
@@ -65,17 +70,22 @@ Services:
|
||||
|
||||
| Host port | Service |
|
||||
|---|---|
|
||||
| `8006` | HTTP API + Web UI |
|
||||
| `8007` | HTTPS nginx gateway → simulator |
|
||||
| `8006` | HTTP API + Web UI (same port as real PVE; real PVE uses HTTPS) |
|
||||
| `5432` | PostgreSQL (localhost only) |
|
||||
|
||||
On real Proxmox VE the REST API is **only** `https://<host>:8006/api2/json/...`.
|
||||
The lab publishes plain **HTTP** on host `:8006`; see
|
||||
[Ports and TLS](configuration.md#ports-and-tls). Optional HTTPS for proxmoxer:
|
||||
`docker compose --profile tls` → `https://localhost:8443/`. Host `:8007` is
|
||||
**not** used (on hardware it is typically PBS, not PVE API).
|
||||
|
||||
Migrations apply automatically before the simulator becomes ready.
|
||||
|
||||
## 2. Wait until ready
|
||||
|
||||
```bash
|
||||
curl http://localhost:8006/health/live
|
||||
curl http://localhost:8006/health/ready
|
||||
curl -sS http://localhost:8006/health/live
|
||||
curl -sS http://localhost:8006/health/ready
|
||||
```
|
||||
|
||||
`/health/ready` returns HTTP 503 until PostgreSQL is reachable **and** the
|
||||
@@ -94,7 +104,7 @@ local storages, and the standard development principals. See
|
||||
## 4. Check the API version
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8006/api2/json/version | jq .
|
||||
curl -sS http://localhost:8006/api2/json/version | jq .
|
||||
```
|
||||
|
||||
The cold-start contract defaults to the bundled PVE **9.2.3** snapshot in Docker
|
||||
@@ -104,7 +114,7 @@ Compose. Switch majors 6–9 from the Web UI or
|
||||
## 5. Authenticate
|
||||
|
||||
```bash
|
||||
curl -s -X POST \
|
||||
curl -sS -X POST \
|
||||
-d 'username=root@pam&password=secret' \
|
||||
http://localhost:8006/api2/json/access/ticket | jq .
|
||||
```
|
||||
@@ -120,10 +130,10 @@ Details: [Authentication](authentication.md).
|
||||
|
||||
```bash
|
||||
# replace TICKET / CSRF from the previous response
|
||||
curl -s -H "Cookie: PVEAuthCookie=$TICKET" \
|
||||
curl -sS -H "Cookie: PVEAuthCookie=$TICKET" \
|
||||
http://localhost:8006/api2/json/nodes/pve01/qemu | jq .
|
||||
|
||||
curl -s -X POST \
|
||||
curl -sS -X POST \
|
||||
-H "Cookie: PVEAuthCookie=$TICKET" \
|
||||
-H "CSRFPreventionToken: $CSRF" \
|
||||
http://localhost:8006/api2/json/nodes/pve01/qemu/100/status/start | jq .
|
||||
@@ -132,7 +142,7 @@ curl -s -X POST \
|
||||
Async operations return a UPID string. Poll until the task finishes:
|
||||
|
||||
```bash
|
||||
curl -s -H "Cookie: PVEAuthCookie=$TICKET" \
|
||||
curl -sS -H "Cookie: PVEAuthCookie=$TICKET" \
|
||||
"http://localhost:8006/api2/json/nodes/pve01/tasks/${UPID}/status" | jq .
|
||||
```
|
||||
|
||||
@@ -140,8 +150,8 @@ curl -s -H "Cookie: PVEAuthCookie=$TICKET" \
|
||||
|
||||
Visit [http://localhost:8006/](http://localhost:8006/) for the interactive
|
||||
console, contract catalog (PVE 6–9), compatibility view, runtime contract apply,
|
||||
and demo-cluster controls. See [Web UI](web-ui.md) for light/dark theme
|
||||
screenshots and the full feature list.
|
||||
and demo-cluster controls. See [Web UI](web-ui.md) for screenshots and the
|
||||
full feature list.
|
||||
|
||||
## 8. Try a client library
|
||||
|
||||
|
||||
|
After Width: | Height: | Size: 140 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
Before Width: | Height: | Size: 53 KiB After Width: | Height: | Size: 175 KiB |
|
After Width: | Height: | Size: 135 KiB |
|
After Width: | Height: | Size: 124 KiB |
|
After Width: | Height: | Size: 72 KiB |
|
After Width: | Height: | Size: 133 KiB |
|
After Width: | Height: | Size: 222 KiB |
|
Before Width: | Height: | Size: 52 KiB After Width: | Height: | Size: 171 KiB |
|
After Width: | Height: | Size: 54 KiB |
|
After Width: | Height: | Size: 86 KiB |