Prepare 0.1.0 for lab release: durable handlers, HTTP Compose, CI, and pulumi-tests.

- Harden DB-backed handlers and seed profiles; align client wire shapes for
  cluster resources, QEMU config, and node SSL fields
- Serve plain HTTP on Compose :8006; keep TLS optional (--profile tls) and
  terminate HTTPS at Kubernetes Ingress
- Add pulumi-tests (full contract surface majors 6–9 + BPG lifecycle) and
  make pulumi-tests
- Ship bilingual docs, CHANGELOG, SECURITY, CONTRIBUTING, and GitHub Actions
  (make ci + Compose/Helm validation)
This commit is contained in:
Sergey Antropoff
2026-07-18 04:18:05 +03:00
parent 777926487b
commit 48df10b17e
172 changed files with 7528 additions and 1208 deletions
+3 -1
View File
@@ -23,5 +23,7 @@ SIMULATION_SEED=42
SIMULATION_TIME_SCALE=10 SIMULATION_TIME_SCALE=10
SIMULATOR_ADMIN_ENABLED=false SIMULATOR_ADMIN_ENABLED=false
SIMULATOR_ADMIN_TOKEN=replace-with-a-long-random-secret SIMULATOR_ADMIN_TOKEN=replace-with-a-long-random-secret
PROXMOXER_HOST=tls-gateway PROXMOXER_HOST=
PROXMOXER_PORT=8443 PROXMOXER_PORT=8443
# For proxmoxer (HTTPS-only): docker compose --profile tls up -d
# then PROXMOXER_HOST=tls-gateway PROXMOXER_PORT=8443
+60
View File
@@ -0,0 +1,60 @@
name: CI
on:
push:
branches: [main]
pull_request:
workflow_dispatch:
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
validate:
name: Compose / Helm validate
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- name: Validate Compose files
run: |
docker compose -f docker-compose.yml config --quiet
docker compose -f docker-compose.release.yml config --quiet
- uses: azure/setup-helm@v4
with:
version: v3.16.4
- name: Lint and render Helm chart
run: |
make helm-lint
make helm-template >/dev/null
quality:
name: make ci
runs-on: ubuntu-latest
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Prepare lab env
run: test -f .env || cp .env.example .env
- name: Build runtime and development images
run: make install
- name: Ruff + mypy + offline pytest + API surface probe
run: make ci
- name: Upload coverage XML
if: always()
uses: actions/upload-artifact@v4
with:
name: coverage-xml
path: coverage.xml
if-no-files-found: ignore
+30
View File
@@ -0,0 +1,30 @@
# Changelog
All notable changes to this project are documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.1.0] - 2026-07-17
### Added
- Stateful Proxmox VE API simulator backed by PostgreSQL with durable UPID task
workers.
- Imported official API contracts for PVE majors **6.4-15**, **7.4-16**,
**8.4.5**, and **9.2.3** with full handler registration for declared methods.
- Interactive Web UI (catalog, runtime contract Apply, task monitor).
- Development Compose stack with plain HTTP on host `:8006` (real PVE port; real
PVE uses HTTPS). Optional TLS gateway via `--profile tls` on `:8443`.
- Published runtime image workflow (`make release`) for
[`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator).
- `docker-compose.release.yml` + Helm chart with optional Ingress / cert-manager.
- Bilingual documentation (English + Russian) and client cookbooks.
- GitHub Actions CI (`make ci`, Compose/Helm validation).
### Security
- Explicit lab-only threat model: default secrets and open `/ui` / `/admin`
helpers are for local/CI use. See [SECURITY.md](SECURITY.md).
[0.1.0]: https://github.com/sergeyantropoff/proxmox-api-simulator/releases/tag/v0.1.0
+66
View File
@@ -0,0 +1,66 @@
**Language / Язык:** [English](CONTRIBUTING.md) | [Русский](CONTRIBUTING.ru.md)
# Contributing
Thanks for helping improve the Proxmox VE API laboratory simulator.
## Prerequisites
- Docker + Docker Compose
- `make`
- No local Python toolchain required for day-to-day work (tools run in Compose)
## Local loop
```bash
cp -n .env.example .env
make install
make up
make seed PROFILE=small
curl -sS http://localhost:8006/health/ready
```
Primary HTTP endpoint (same port as real PVE): `http://localhost:8006/`
Optional HTTPS for proxmoxer: `docker compose --profile tls`
`https://localhost:8443/` (self-signed — use `curl -sk` or accept the browser
warning).
## Quality gates (must pass before a PR)
```bash
make ci # ruff format/check + mypy + offline pytest + surface probe
make ci-all # also remaining integration + proxmoxer compatibility
make helm-lint # chart lint (+ ingress example values)
make pulumi-tests # Pulumi surface (majors 69) + pulumi-proxmoxve lifecycle
```
GitHub Actions runs `make ci` plus Compose/Helm validation on every push and PR
to `main`. Run `make ci-all`, `make helm-lint`, and `make pulumi-tests` locally
before larger API or client-facing changes.
## Project rules worth remembering
1. Mutations must **persist to PostgreSQL** (tables and/or jsonb metadata).
2. Do **not** add user-facing “not supported in the simulator” style errors —
see `.cursor/rules/durable-simulator.mdc`.
3. Prefer matching Proxmox request/response shapes from the contract snapshot.
4. Keep EN and RU docs in sync when you change operator-facing behaviour.
## Docs
- Index: [docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md)
- Security / lab threat model: [SECURITY.md](SECURITY.md)
## Releases
Maintainers publish the runtime image with:
```bash
docker login
make release # pushes inecs/proxmox-api-simulator:<version> (+ :latest)
```
After a public release, paste the overview from
[docs/docker-hub-overview.md](docs/docker-hub-overview.md) into the Docker Hub
repository description if it drifted.
+63
View File
@@ -0,0 +1,63 @@
**Language / Язык:** [English](CONTRIBUTING.md) | [Русский](CONTRIBUTING.ru.md)
# Участие в разработке
Спасибо за помощь с лабораторным симулятором Proxmox VE API.
## Требования
- Docker + Docker Compose
- `make`
- Локальный Python для повседневной работы не нужен (инструменты в Compose)
## Локальный цикл
```bash
cp -n .env.example .env
make install
make up
make seed PROFILE=small
curl -sS http://localhost:8006/health/ready
```
Основной HTTPS-эндпоинт (как у реального PVE): `http://localhost:8006/`
(self-signed — `curl -sk` или принять предупреждение в браузере).
## Quality gates (обязательны перед PR)
```bash
make ci # ruff format/check + mypy + offline pytest + surface probe
make ci-all # ещё integration + proxmoxer compatibility
make helm-lint # lint chart (+ ingress example values)
make pulumi-tests # Pulumi surface (majors 69) + pulumi-proxmoxve lifecycle
```
GitHub Actions на каждый push/PR в `main` запускает `make ci` и проверку
Compose/Helm. Перед крупными API/клиентскими изменениями локально прогоняйте
`make ci-all`, `make helm-lint` и `make pulumi-tests`.
## Важные правила проекта
1. Мутации должны **persist в PostgreSQL** (таблицы и/или jsonb metadata).
2. Не добавляйте user-facing сообщения вроде «not supported in the simulator» —
см. `.cursor/rules/durable-simulator.mdc`.
3. Сохраняйте формы запросов/ответов Proxmox из снимка контракта.
4. При изменении операторского поведения синхронизируйте EN и RU docs.
## Документация
- Индекс: [docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md)
- Безопасность / модель угроз лаборатории: [SECURITY.md](SECURITY.md)
## Релизы
Maintainers публикуют runtime-образ так:
```bash
docker login
make release # пушит inecs/proxmox-api-simulator:<version> (+ :latest)
```
После публичного релиза при необходимости вставьте overview из
[docs/docker-hub-overview.md](docs/docker-hub-overview.md) в описание репозитория
Docker Hub.
+3 -1
View File
@@ -16,7 +16,9 @@ FROM python:3.13-slim AS runtime
ARG APP_VERSION=0.1.0 ARG APP_VERSION=0.1.0
LABEL org.opencontainers.image.title="proxmox-api-simulator" \ LABEL org.opencontainers.image.title="proxmox-api-simulator" \
org.opencontainers.image.version="$APP_VERSION" \ org.opencontainers.image.version="$APP_VERSION" \
org.opencontainers.image.source="https://github.com/example/proxmox-api-simulator" org.opencontainers.image.description="Stateful Proxmox VE API simulator for labs and CI" \
org.opencontainers.image.source="https://github.com/sergeyantropoff/proxmox-api-simulator" \
org.opencontainers.image.licenses="Apache-2.0"
ENV PATH="/opt/venv/bin:$PATH" \ ENV PATH="/opt/venv/bin:$PATH" \
PYTHONDONTWRITEBYTECODE=1 \ PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \ PYTHONUNBUFFERED=1 \
+23 -8
View File
@@ -13,7 +13,7 @@ PUSH_LATEST ?= 1
COMPOSE_RELEASE ?= $(COMPOSE) -f docker-compose.release.yml COMPOSE_RELEASE ?= $(COMPOSE) -f docker-compose.release.yml
HELM_CHART ?= ./helm/proxmox-api-simulator HELM_CHART ?= ./helm/proxmox-api-simulator
.PHONY: help install format lint typecheck test test-unit test-integration test-contract test-compatibility test-surface evidence coverage run dev up down restart logs docker-build docker-up docker-down docker-logs docker-restart db-up db-down db-migrate db-reset api-import api-diff seed clean ci ci-all shell release release-build release-up release-down release-seed helm-deps helm-template .PHONY: help install format lint typecheck test test-unit test-integration test-contract test-compatibility test-surface evidence coverage run dev up down restart logs docker-build docker-up docker-down docker-logs docker-restart db-up db-down db-migrate db-reset api-import api-diff seed clean ci ci-all shell release release-build release-up release-down release-seed helm-deps helm-template helm-lint pulumi-tests
help: ## Show available commands help: ## Show available commands
@awk 'BEGIN {FS = ":.*## "}; /^[a-zA-Z0-9_-]+:.*## / {printf "%-22s %s\n", $$1, $$2}' $(MAKEFILE_LIST) @awk 'BEGIN {FS = ":.*## "}; /^[a-zA-Z0-9_-]+:.*## / {printf "%-22s %s\n", $$1, $$2}' $(MAKEFILE_LIST)
@@ -45,11 +45,16 @@ test-integration: ## Run tests that require PostgreSQL
test-contract: ## Run offline API contract tests test-contract: ## Run offline API contract tests
$(COMPOSE) run --rm --no-deps $(SERVICE_DEV) pytest -m contract $(COMPOSE) run --rm --no-deps $(SERVICE_DEV) pytest -m contract
test-compatibility: ## Run proxmoxer smoke flow against the Compose stack test-compatibility: ## Run proxmoxer smoke flow (needs --profile tls; proxmoxer is HTTPS-only)
@test -f .env || cp .env.example .env @test -f .env || cp .env.example .env
$(COMPOSE) up -d --build --wait $(COMPOSE) --profile tls up -d --build --wait
$(COMPOSE) run --rm --entrypoint python $(SERVICE_SIM) -m app.simulation.seed_cli # Medium profile provides pve1/pve2/pve3 required by the proxmoxer migration smoke.
$(COMPOSE) run --rm $(SERVICE_DEV) pytest -m compatibility $(COMPOSE) run --rm -e SEED_PROFILE=medium --entrypoint python $(SERVICE_SIM) \
-m app.simulation.seed_cli
$(COMPOSE) run --rm \
-e PROXMOXER_HOST=tls-gateway \
-e PROXMOXER_PORT=8443 \
$(SERVICE_DEV) pytest -m compatibility
test-surface: ## Probe every declared method on majors 6-9 (0x501 / 0xexception) test-surface: ## Probe every declared method on majors 6-9 (0x501 / 0xexception)
@test -f .env || cp .env.example .env @test -f .env || cp .env.example .env
@@ -66,7 +71,7 @@ run: ## Run the application in the foreground
@test -f .env || cp .env.example .env @test -f .env || cp .env.example .env
$(COMPOSE) up --build $(COMPOSE) up --build
up: ## Start PostgreSQL, simulator, and TLS gateway up: ## Start PostgreSQL and simulator (plain HTTP :8006)
@test -f .env || cp .env.example .env @test -f .env || cp .env.example .env
$(COMPOSE) up -d --build --wait $(COMPOSE) up -d --build --wait
@@ -90,8 +95,8 @@ docker-build: ## Build runtime and development images
docker-up: up ## Alias for up docker-up: up ## Alias for up
docker-restart: ## Rebuild and recreate simulator and TLS gateway only docker-restart: ## Rebuild and recreate the simulator
$(COMPOSE) up -d --build --force-recreate simulator tls-gateway $(COMPOSE) up -d --build --force-recreate simulator
docker-down: down ## Alias for down docker-down: down ## Alias for down
@@ -176,8 +181,18 @@ release-seed: ## Seed the published Hub stack (PROFILE=small by default)
helm-deps: ## No-op placeholder (chart has no OCI dependencies) helm-deps: ## No-op placeholder (chart has no OCI dependencies)
@echo "Chart $(HELM_CHART) vendors PostgreSQL templates; no helm dependency update required." @echo "Chart $(HELM_CHART) vendors PostgreSQL templates; no helm dependency update required."
helm-lint: ## Lint the Helm chart (requires helm)
helm lint $(HELM_CHART)
helm lint $(HELM_CHART) -f $(HELM_CHART)/values-ingress-example.yaml \
--set certManager.email=docs@example.com \
--set secret.ticketSigningKey=docs-only-signing-key
helm-template: ## Render Helm manifests locally (requires helm) helm-template: ## Render Helm manifests locally (requires helm)
helm template pve-sim $(HELM_CHART) \ helm template pve-sim $(HELM_CHART) \
-f $(HELM_CHART)/values-ingress-example.yaml \ -f $(HELM_CHART)/values-ingress-example.yaml \
--set certManager.email=docs@example.com \ --set certManager.email=docs@example.com \
--set secret.ticketSigningKey=docs-only-signing-key --set secret.ticketSigningKey=docs-only-signing-key
pulumi-tests: ## Full Pulumi suite (surface majors 69 + lifecycle, HTML report)
$(MAKE) -C pulumi-tests up
$(MAKE) -C pulumi-tests test
+45 -13
View File
@@ -1,9 +1,20 @@
**Language / Язык:** [English](README.md) | [Русский](README.ru.md)
# proxmox-api-simulator # proxmox-api-simulator
[![CI](https://github.com/sergeyantropoff/proxmox-api-simulator/actions/workflows/ci.yml/badge.svg)](https://github.com/sergeyantropoff/proxmox-api-simulator/actions/workflows/ci.yml)
[![Docker Image](https://img.shields.io/docker/v/inecs/proxmox-api-simulator?label=docker%20hub&sort=semver)](https://hub.docker.com/r/inecs/proxmox-api-simulator)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
Stateful asynchronous [Proxmox VE](https://www.proxmox.com/) API simulator for Stateful asynchronous [Proxmox VE](https://www.proxmox.com/) API simulator for
testing API clients and infrastructure tooling without a real hypervisor testing API clients and infrastructure tooling without a real hypervisor
cluster. cluster.
> **Laboratory / CI only.** Default credentials, signing keys, and open UI/admin
> helpers are intentional lab defaults. Do **not** expose this stack to the
> public Internet without replacing secrets and adding your own controls.
> See [SECURITY.md](SECURITY.md) and [Security](docs/security.md).
The simulator is backed by PostgreSQL, driven by imported official API The simulator is backed by PostgreSQL, driven by imported official API
contracts, and exposes the same `/api2/json` and `/api2/extjs` surfaces as contracts, and exposes the same `/api2/json` and `/api2/extjs` surfaces as
Proxmox VE. Semantic handlers persist mutations; long-running work returns Proxmox VE. Semantic handlers persist mutations; long-running work returns
@@ -37,13 +48,16 @@ Image: [`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-ap
### Docker Compose ### Docker Compose
Needs a checkout that includes `docker-compose.release.yml`. Do not publish host
`:8006` beyond a trusted lab without rotating `TICKET_SIGNING_KEY` / DB password.
```bash ```bash
docker compose -f docker-compose.release.yml up -d docker compose -f docker-compose.release.yml up -d
docker compose -f docker-compose.release.yml run --rm --entrypoint python \ docker compose -f docker-compose.release.yml run --rm --entrypoint python \
simulator -m app.simulation.seed_cli simulator -m app.simulation.seed_cli
curl http://localhost:8006/health/ready curl -sS http://localhost:8006/health/ready
curl http://localhost:8006/api2/json/version curl -sS http://localhost:8006/api2/json/version
``` ```
Or: `make release-up && make release-seed PROFILE=small` Or: `make release-up && make release-seed PROFILE=small`
@@ -57,10 +71,14 @@ helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
--set certManager.email=you@example.com \ --set certManager.email=you@example.com \
--set ingress.hosts[0].host=pve-sim.example.com \ --set ingress.hosts[0].host=pve-sim.example.com \
--set ingress.tls[0].hosts[0]=pve-sim.example.com \ --set ingress.tls[0].hosts[0]=pve-sim.example.com \
--set secret.ticketSigningKey="$(openssl rand -hex 32)" --set secret.ticketSigningKey="$(openssl rand -hex 32)" \
--set postgresql.auth.password="$(openssl rand -hex 16)"
``` ```
Requires an Ingress controller and cert-manager. Details: Requires an Ingress controller and cert-manager. The chart Service speaks
**HTTP** `:8006`; TLS terminates at Ingress. Compose also serves plain HTTP on
`:8006`; optional HTTPS for proxmoxer-style clients is
`docker compose --profile tls` on host `:8443`. Details:
[Kubernetes / Helm](docs/kubernetes.md). [Kubernetes / Helm](docs/kubernetes.md).
- HTTP API and Web UI (Compose): [http://localhost:8006/](http://localhost:8006/) - HTTP API and Web UI (Compose): [http://localhost:8006/](http://localhost:8006/)
@@ -76,22 +94,24 @@ make install
make up make up
make seed PROFILE=small make seed PROFILE=small
curl http://localhost:8006/health/ready curl -sS http://localhost:8006/health/ready
curl http://localhost:8006/api2/json/version curl -sS http://localhost:8006/api2/json/version
curl -X POST -d 'username=root@pam&password=secret' \ curl -sS -X POST -d 'username=root@pam&password=secret' \
http://localhost:8006/api2/json/access/ticket http://localhost:8006/api2/json/access/ticket
``` ```
- HTTP API and Web UI: [http://localhost:8006/](http://localhost:8006/) - HTTP API and Web UI: [http://localhost:8006/](http://localhost:8006/)
- HTTPS gateway (self-signed, development only): `https://localhost:8007` (real PVE uses **HTTPS** on `:8006`; this lab uses plain HTTP on the same
— checked-in `docker/tls/server.key` is a **lab-only** localhost cert; do not port. Optional TLS for proxmoxer: `docker compose --profile tls`
reuse it outside local Compose. `https://localhost:8443/`)
- Port map and TLS notes: [Ports and TLS](docs/configuration.md#ports-and-tls).
- FastAPI schema docs: [http://localhost:8006/docs](http://localhost:8006/docs) - FastAPI schema docs: [http://localhost:8006/docs](http://localhost:8006/docs)
### Web UI ### Web UI
Interactive console with light/dark themes, endpoint catalog for PVE 69, and Interactive console with light/dark themes, endpoint catalog for PVE 69,
runtime contract hot-swap. More detail: [Web UI](docs/web-ui.md). runtime contract hot-swap, and a UPID task monitor. More detail:
[Web UI](docs/web-ui.md).
![Web UI light theme](docs/images/web-ui-light.png) ![Web UI light theme](docs/images/web-ui-light.png)
@@ -99,6 +119,10 @@ runtime contract hot-swap. More detail: [Web UI](docs/web-ui.md).
## Documentation ## Documentation
Documentation is bilingual. Use the **Language / Язык** switcher at the top of
each page, or open the Russian root [README.ru.md](README.ru.md). Index:
[docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md).
| Guide | Description | | Guide | Description |
|---|---| |---|---|
| [Getting started](docs/getting-started.md) | First successful lab session | | [Getting started](docs/getting-started.md) | First successful lab session |
@@ -120,6 +144,8 @@ runtime contract hot-swap. More detail: [Web UI](docs/web-ui.md).
| [Compatibility](docs/compatibility.md) | Evidence model and release matrix | | [Compatibility](docs/compatibility.md) | Evidence model and release matrix |
Runnable cookbooks live under [`examples/`](examples/README.md). Runnable cookbooks live under [`examples/`](examples/README.md).
Pulumi integration suite (contract surface majors 69 + lifecycle, HTML report):
[`pulumi-tests/`](pulumi-tests/README.md) (`make pulumi-tests`).
## proxmoxer (HTTPS gateway) ## proxmoxer (HTTPS gateway)
@@ -128,7 +154,7 @@ from proxmoxer import ProxmoxAPI
proxmox = ProxmoxAPI( proxmox = ProxmoxAPI(
"localhost", "localhost",
port=8007, port=8006,
user="root@pam", user="root@pam",
password="secret", password="secret",
verify_ssl=False, # local self-signed development certificate only verify_ssl=False, # local self-signed development certificate only
@@ -169,6 +195,12 @@ make release-build # build/tag only, no push
make release-up && make release-seed # run the published stack locally make release-up && make release-seed # run the published stack locally
``` ```
## Contributing / security / changelog
- [CONTRIBUTING.md](CONTRIBUTING.md) · [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md)
- [SECURITY.md](SECURITY.md)
- [CHANGELOG.md](CHANGELOG.md)
## What this is not ## What this is not
- Not a hypervisor: no KVM/LXC execution on bare metal or nested hosts. - Not a hypervisor: no KVM/LXC execution on bare metal or nested hosts.
+210
View File
@@ -0,0 +1,210 @@
**Language / Язык:** [English](README.md) | [Русский](README.ru.md)
# proxmox-api-simulator
[![CI](https://github.com/sergeyantropoff/proxmox-api-simulator/actions/workflows/ci.yml/badge.svg)](https://github.com/sergeyantropoff/proxmox-api-simulator/actions/workflows/ci.yml)
[![Docker Image](https://img.shields.io/docker/v/inecs/proxmox-api-simulator?label=docker%20hub&sort=semver)](https://hub.docker.com/r/inecs/proxmox-api-simulator)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
Stateful-асинхронный симулятор API [Proxmox VE](https://www.proxmox.com/) для
тестирования API-клиентов и инфраструктурных инструментов без реального
гипервизорного кластера.
> **Только лаборатория / CI.** Учётные данные, signing keys и открытые UI/admin
> helpers по умолчанию — намеренные лабораторные значения. **Не** выставляйте
> стек в публичный Интернет без замены секретов и своих сетевых ограничений.
> См. [SECURITY.md](SECURITY.md) и [Безопасность](docs/ru/security.md).
Симулятор работает на PostgreSQL, опирается на импортированные официальные
контракты API и предоставляет те же поверхности `/api2/json` и `/api2/extjs`,
что и Proxmox VE. Семантические обработчики сохраняют мутации; длительные
операции возвращают устойчивые UPID, которые выполняют воркеры с арендой задач.
## Проверенное покрытие API
Реестр обработчиков и верифицированные ledger поверхности — **100%** для каждого
включённого major:
| Контракт | Declared | Implemented | Verified |
|---|---:|---:|---:|
| PVE 6.4-15 | 504 | 504 | 504 |
| PVE 7.4-16 | 540 | 540 | 540 |
| PVE 8.4.5 | 605 | 605 | 605 |
| PVE 9.2.3 | 675 | 675 | 675 |
Переключение активного контракта в runtime — из Web UI (**Apply as runtime**) или
`POST /ui/api/contract/apply?major=N` — каждый Apply загружает
`evidence/pve-{version}.json`, чтобы observed/verified следовали выбранному
major. После импорта нового контракта перегенерируйте ledger: `make evidence`.
Живые отчёты: `/admin/compatibility` (также `.md` / `.html`). См.
[Совместимость](docs/ru/compatibility.md) и [Версии API](docs/ru/api-versions.md).
> Это измеримое покрытие контракта и обработчиков лабораторного симулятора —
> не утверждение, что каждый краевой случай Proxmox или удалённая интеграция
> ведёт себя идентично продакшен-железу.
## Быстрый старт (опубликованный образ)
Образ: [`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator)
### Docker Compose
Нужен checkout с `docker-compose.release.yml`. Не публикуйте хост `:8006` за
пределы доверенной лаборатории без ротации `TICKET_SIGNING_KEY` / пароля БД.
```bash
docker compose -f docker-compose.release.yml up -d
docker compose -f docker-compose.release.yml run --rm --entrypoint python \
simulator -m app.simulation.seed_cli
curl -sS http://localhost:8006/health/ready
curl -sS http://localhost:8006/api2/json/version
```
Или: `make release-up && make release-seed PROFILE=small`
### Helm (Kubernetes + Ingress + Let's Encrypt)
```bash
helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
-n proxmox-sim --create-namespace \
-f ./helm/proxmox-api-simulator/values-ingress-example.yaml \
--set certManager.email=you@example.com \
--set ingress.hosts[0].host=pve-sim.example.com \
--set ingress.tls[0].hosts[0]=pve-sim.example.com \
--set secret.ticketSigningKey="$(openssl rand -hex 32)" \
--set postgresql.auth.password="$(openssl rand -hex 16)"
```
Нужны Ingress-контроллер и cert-manager. Service чарта говорит по **HTTP**
`:8006`; TLS — на Ingress. Compose тоже отдаёт plain HTTP на `:8006`; опциональный
HTTPS для proxmoxer — `docker compose --profile tls` на хосте `:8443`.
Подробности: [Kubernetes / Helm](docs/ru/kubernetes.md).
- HTTP API и Web UI (Compose): [http://localhost:8006/](http://localhost:8006/)
- Схема FastAPI: [http://localhost:8006/docs](http://localhost:8006/docs)
- Админ по умолчанию после seed: `root@pam` / `secret`
## Быстрый старт (разработка из репозитория)
Сборка и запуск development-стека с bind-mount из этого репозитория:
```bash
make install
make up
make seed PROFILE=small
curl -sS http://localhost:8006/health/ready
curl -sS http://localhost:8006/api2/json/version
curl -sS -X POST -d 'username=root@pam&password=secret' \
http://localhost:8006/api2/json/access/ticket
```
- HTTP API и Web UI: [http://localhost:8006/](http://localhost:8006/)
(реальный PVE — **HTTPS** на `:8006`; лаборатория — plain HTTP на том же
порту. Опциональный TLS для proxmoxer: `docker compose --profile tls`
`https://localhost:8443/`)
- Карта портов и TLS: [Порты и TLS](docs/ru/configuration.md#порты-и-tls).
- Схема FastAPI: [http://localhost:8006/docs](http://localhost:8006/docs)
### Web UI
Интерактивная консоль со светлой/тёмной темой, каталогом эндпоинтов PVE 6–9,
горячей сменой runtime-контракта и монитором задач UPID. Подробнее:
[Web UI](docs/ru/web-ui.md).
![Web UI, светлая тема](docs/images/web-ui-light.png)
![Web UI, тёмная тема](docs/images/web-ui-dark.png)
## Документация
Документация двуязычная. Переключатель **Language / Язык** — в шапке каждой
страницы; английский корень — [README.md](README.md). Индекс:
[docs/README.md](docs/README.md) · [docs/ru/README.md](docs/ru/README.md).
| Руководство | Описание |
|---|---|
| [Начало работы](docs/ru/getting-started.md) | Первая успешная лабораторная сессия |
| [Конфигурация](docs/ru/configuration.md) | Переменные окружения и Compose |
| [Аутентификация](docs/ru/authentication.md) | Тикеты, CSRF, API-токены, ACL |
| [Версии API](docs/ru/api-versions.md) | Контракты 69 и hot-swap |
| [Клиенты и примеры](docs/ru/clients.md) | Python, Go, Java, Perl, Ansible, Terraform, Pulumi |
| [Профили seed](docs/ru/seed-profiles.md) | Детерминированные фикстуры кластера |
| [Поверхность API](docs/ru/api-surface.md) | Маршрутизация, обработчики, fallback |
| [Домены](docs/ru/domains/README.md) | QEMU, LXC, storage, HA, SDN, … |
| [Web UI](docs/ru/web-ui.md) | Интерактивная консоль и каталоги |
| [Эксплуатация](docs/ru/operations.md) | Миграции, reseed, обновления |
| [Kubernetes / Helm](docs/ru/kubernetes.md) | Образ Hub + Ingress + Let's Encrypt |
| [Безопасность](docs/ru/security.md) | Модель угроз лаборатории и учётные данные |
| [Наблюдаемость](docs/ru/observability.md) | Health-эндпоинты и логирование |
| [Устранение неполадок](docs/ru/troubleshooting.md) | Типичные сбои |
| [FAQ](docs/ru/faq.md) | Краткие ответы |
| [Архитектура](docs/ru/architecture.md) | Границы компонентов |
| [Совместимость](docs/ru/compatibility.md) | Модель evidence и матрица релиза |
Индекс гайдов: [`docs/ru/README.md`](docs/ru/README.md).
Запускаемые cookbook: [`examples/`](examples/README.ru.md).
Интеграционный набор Pulumi (surface majors 69 + lifecycle, HTML-отчёт):
[`pulumi-tests/`](pulumi-tests/README.ru.md) (`make pulumi-tests`).
## proxmoxer (HTTPS-шлюз)
```python
from proxmoxer import ProxmoxAPI
proxmox = ProxmoxAPI(
"localhost",
port=8006,
user="root@pam",
password="secret",
verify_ssl=False, # только локальный self-signed сертификат разработки
)
print(proxmox.version.get())
print(proxmox.nodes("pve01").qemu.get())
```
Пример API-токена: пользователь `root@pam`, `token_name="automation"`,
`token_value="automation-secret"`. Запросам с токеном CSRF не нужен; мутациям
по тикету — нужен.
## Частые цели Make
```bash
make up / make down / make logs / make dev
make test # unit + contract (включая verified surface)
make test-integration # с PostgreSQL
make test-surface # все глаголы × majors 6-9 (0x501 / 0xexception)
make test-compatibility # proxmoxer против Compose
make evidence # перегенерация evidence/pve-*.json
make seed PROFILE=small
make db-migrate
make shell
make ci # ruff + mypy + offline pytest + surface probe
make release # сборка + push runtime-образа в Docker Hub
make release-up # pull/start docker-compose.release.yml
make release-seed PROFILE=small
```
Релиз в Docker Hub (нужен `docker login` владельца Hub; см.
[Эксплуатация](docs/ru/operations.md)):
```bash
make release # inecs/proxmox-api-simulator:<версия pyproject> + :latest
make release VERSION=0.2.0 # переопределить тег
make release-build # только build/tag, без push
make release-up && make release-seed # запустить опубликованный стек локально
```
## Участие / безопасность / changelog
- [CONTRIBUTING.md](CONTRIBUTING.md) · [CONTRIBUTING.ru.md](CONTRIBUTING.ru.md)
- [SECURITY.md](SECURITY.md)
- [CHANGELOG.md](CHANGELOG.md)
## Чем это не является
- Не гипервизор: нет выполнения KVM/LXC на железе или nested-хостах.
- Не drop-in замена мультиарендного продакшен-Proxmox.
- Удалённые IdP / LDAP / live Ceph / live ACME эндпоинты симулируются локально;
к реальным внешним системам они не обращаются.
+52
View File
@@ -0,0 +1,52 @@
**Language / Язык:** [English](SECURITY.md) | [Русский](docs/ru/security.md)
# Security policy
## Supported versions
| Version | Supported |
|---|---|
| `0.1.x` (latest) | Yes — security reports accepted |
| older / untagged | Best-effort only |
## Threat model (read this first)
This repository is a **local / CI laboratory simulator**, not a hardened
multi-tenant public Proxmox deployment.
Default Compose and Helm values intentionally ship convenient lab secrets,
seeded passwords, open Web UI helper routes (`/ui/api/*`), and compatibility
endpoints (`/admin/compatibility*`). Treat network reachability as the trust
boundary.
**Do not** expose host port `8006` (or a public Ingress) to untrusted networks
without replacing every default secret and adding controls you own.
Full lab notes: [docs/security.md](docs/security.md) ·
[docs/ru/security.md](docs/ru/security.md).
## Reporting a vulnerability
Please **do not** open a public GitHub issue for sensitive reports.
Email the maintainer privately (account that owns the GitHub repository /
Docker Hub `inecs` namespace), or use GitHub
[ privately reported vulnerabilities](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability)
if enabled on the repository.
Include:
- Affected version / image tag (`inecs/proxmox-api-simulator:…`)
- Reproduction steps against a **local** lab (not third-party instances)
- Impact assessment (auth bypass, secret leak, RCE, etc.)
You should receive an acknowledgement within a few business days.
## Lab secrets that must be rotated outside toy labs
| Secret | Where |
|---|---|
| `TICKET_SIGNING_KEY` | Compose / Helm |
| PostgreSQL password | Compose / Helm |
| Seeded `root@pam` / API tokens | After `seed` |
| `docker/tls/server.key` | Checked-in self-signed material — never reuse outside local Compose |
+66 -2
View File
@@ -17,7 +17,7 @@ from app.api.errors import ApiError, ContractValidationError
from app.api.openapi import contract_openapi_tags from app.api.openapi import contract_openapi_tags
from app.config import Settings from app.config import Settings
from app.contracts.examples import schema_example from app.contracts.examples import schema_example
from app.contracts.model import Method, Schema, Snapshot from app.contracts.model import Method, Parameter, Schema, Snapshot
from app.db.pool import AsyncpgDatabase from app.db.pool import AsyncpgDatabase
from app.security.acl import AclEntry, CapabilityRequirement, authorize, requirement_from_contract from app.security.acl import AclEntry, CapabilityRequirement, authorize, requirement_from_contract
from app.security.auth import parse_api_token, verify_csrf, verify_secret, verify_ticket from app.security.auth import parse_api_token, verify_csrf, verify_secret, verify_ticket
@@ -137,6 +137,64 @@ def register_legacy_handler_routes(
return seen return seen
_PATH_PARAM_RE = re.compile(r"\{([^{}]+)\}")
def _unbound_method(path: str, verb: str) -> Method:
"""Minimal contract method so handler-only paths can be mounted."""
parameters = tuple(
Parameter(name=name, definition=Schema(type="string", optional=True))
for name in _PATH_PARAM_RE.findall(path)
)
return Method(
verb=verb.upper(),
name=f"unbound-{verb.lower()}-{path}",
description="Handler-backed route not present in the active contract snapshot",
parameters=parameters,
returns=Schema(),
permissions=None,
checksum="0" * 64,
)
def register_unbound_handler_routes(
app: FastAPI,
handlers: HandlerRegistry,
fallback: FallbackMode = "error",
*,
existing: set[tuple[str, str, str]] | None = None,
) -> set[tuple[str, str, str]]:
"""Mount semantic handlers that no cached contract currently declares.
Keeps DB-backed handlers reachable (no FastAPI 404) when Proxmox schema
omits a path that the simulator still implements.
"""
seen = existing if existing is not None else set()
for path, verb in sorted(handlers.keys()):
for renderer in ("json", "extjs"):
route = f"/api2/{renderer}{path}"
key = (route, verb, renderer)
if key in seen:
continue
method = _unbound_method(path, verb)
seen.add(key)
app.add_api_route(
route,
_endpoint(path, method, renderer, handlers, fallback),
methods=[verb],
name=f"unbound:{renderer}:{verb}:{path}",
tags=cast(list[str | Enum], contract_openapi_tags(path, renderer)),
openapi_extra={
"x-proxmox-method-checksum": method.checksum,
"x-proxmox-implementation": "implemented",
"x-proxmox-unbound-handler": True,
},
)
return seen
def _endpoint( def _endpoint(
semantic_path: str, semantic_path: str,
method: Method, method: Method,
@@ -302,8 +360,14 @@ async def _parse_inputs(request: Request, method: Method) -> dict[str, Any]:
continue continue
if name not in supplied: if name not in supplied:
if definition.optional: if definition.optional:
# Proxmox schema ``default`` is often UI documentation text (e.g.
# bwlimit default = "restore limit from datacenter…"), not a value
# to materialize. Only inject defaults that coerce to the type.
if definition.default is not None: if definition.default is not None:
parsed[name] = definition.default try:
parsed[name] = _coerce(definition.default, definition)
except (TypeError, ValueError):
pass
continue continue
errors[name] = "property is missing and it is not optional" errors[name] = "property is missing and it is not optional"
continue continue
+3 -1
View File
@@ -14,6 +14,7 @@ from app.api.registry import (
HandlerRegistry, HandlerRegistry,
register_contract_routes, register_contract_routes,
register_legacy_handler_routes, register_legacy_handler_routes,
register_unbound_handler_routes,
) )
from app.compatibility import ( from app.compatibility import (
CompatibilityDimension, CompatibilityDimension,
@@ -106,7 +107,7 @@ def apply_runtime_contract(
clear_contract_routes(app) clear_contract_routes(app)
registered = register_contract_routes(app, snapshot, handlers, fallback) registered = register_contract_routes(app, snapshot, handlers, fallback)
register_legacy_handler_routes( registered = register_legacy_handler_routes(
app, app,
handlers, handlers,
store_root, store_root,
@@ -114,6 +115,7 @@ def apply_runtime_contract(
primary_version=snapshot.source_version, primary_version=snapshot.source_version,
existing=registered, existing=registered,
) )
register_unbound_handler_routes(app, handlers, fallback, existing=registered)
report = build_compatibility_for_snapshot( report = build_compatibility_for_snapshot(
snapshot, snapshot,
handlers, handlers,
+45 -47
View File
@@ -10,41 +10,34 @@ from app.api.errors import ApiError
from app.api.registry import HandlerRegistry from app.api.registry import HandlerRegistry
from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs, values from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs, values
_DEFAULT_DIRECTORIES = [
{
"name": "Let's Encrypt V2",
"url": "https://acme-v02.api.letsencrypt.org/directory",
},
{
"name": "Let's Encrypt V2 Staging",
"url": "https://acme-staging-v02.api.letsencrypt.org/directory",
},
]
_CHALLENGE_SCHEMA = [
{
"id": "dns",
"name": "DNS plugin",
"type": "dns",
"fields": [{"name": "api", "type": "string"}],
}
]
def _acme(metadata: dict[str, Any]) -> dict[str, Any]: def _acme(metadata: dict[str, Any]) -> dict[str, Any]:
current = metadata.setdefault( current = metadata.get("acme")
"acme",
{"accounts": {}, "plugins": {}, "meta": {}},
)
if not isinstance(current, dict): if not isinstance(current, dict):
current = {"accounts": {}, "plugins": {}, "meta": {}} current = {}
metadata["acme"] = current metadata["acme"] = current
current.setdefault("accounts", {}) if not isinstance(current.get("accounts"), dict):
current.setdefault("plugins", {}) current["accounts"] = {}
current.setdefault("meta", {}) if not isinstance(current.get("plugins"), dict):
current["plugins"] = {}
if not isinstance(current.get("meta"), dict):
current["meta"] = {}
if not isinstance(current.get("directories"), list):
current["directories"] = []
if not isinstance(current.get("challenge_schema"), list):
current["challenge_schema"] = []
return current return current
def _default_directory(acme: dict[str, Any]) -> str:
directories = acme.get("directories")
if isinstance(directories, list):
for item in directories:
if isinstance(item, dict) and item.get("url"):
return str(item["url"])
return ""
def register_acme_handlers(registry: HandlerRegistry) -> None: def register_acme_handlers(registry: HandlerRegistry) -> None:
async def index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]: async def index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
return subdirs( return subdirs(
@@ -72,16 +65,17 @@ def register_acme_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
name = str(payload.get("name") or "default") name = str(payload.get("name") or "default")
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
accounts = _acme(metadata)["accounts"] acme = _acme(metadata)
accounts = acme["accounts"]
if name in accounts: if name in accounts:
raise ApiError(400, f"ACME account '{name}' already exists") raise ApiError(400, f"ACME account '{name}' already exists")
directory = str(payload.get("directory") or _default_directory(acme))
accounts[name] = { accounts[name] = {
"name": name, "name": name,
"contact": payload.get("contact"), "contact": payload.get("contact"),
"directory": payload.get("directory") or _DEFAULT_DIRECTORIES[0]["url"], "directory": directory,
"tos_url": payload.get("tos_url"), "tos_url": payload.get("tos_url"),
"eab-kid": payload.get("eab-kid"), "eab-kid": payload.get("eab-kid"),
# eab-hmac-key stored but never returned
"eab-hmac-key": payload.get("eab-hmac-key"), "eab-hmac-key": payload.get("eab-hmac-key"),
"location": f"https://acme.example.local/acct/{name}", "location": f"https://acme.example.local/acct/{name}",
} }
@@ -181,29 +175,33 @@ def register_acme_handlers(registry: HandlerRegistry) -> None:
del plugins[plugin_id] del plugins[plugin_id]
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def directories(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def directories(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
return list(_DEFAULT_DIRECTORIES) metadata = await cluster_metadata(request)
directories = _acme(metadata).get("directories")
if isinstance(directories, list):
return [dict(item) for item in directories if isinstance(item, dict)]
return []
async def challenge_schema(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def challenge_schema(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
return list(_CHALLENGE_SCHEMA) metadata = await cluster_metadata(request)
schema = _acme(metadata).get("challenge_schema")
if isinstance(schema, list):
return [dict(item) for item in schema if isinstance(item, dict)]
return []
async def meta(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def meta(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
directory = str(values(inputs).get("directory") or _DEFAULT_DIRECTORIES[0]["url"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
meta_store = _acme(metadata).setdefault("meta", {}) acme = _acme(metadata)
payload = meta_store.setdefault( directory = str(values(inputs).get("directory") or _default_directory(acme))
directory, meta_raw = acme.get("meta")
{ meta_store: dict[str, Any] = dict(meta_raw) if isinstance(meta_raw, dict) else {}
"termsOfService": f"{directory.rstrip('/')}/tos", payload = meta_store.get(directory)
"caaIdentities": ["letsencrypt.org"], if not isinstance(payload, dict):
}, return {}
)
await save_cluster_metadata(request, metadata)
return dict(payload) return dict(payload)
async def tos(request: Request, inputs: dict[str, Any]) -> str: async def tos(request: Request, inputs: dict[str, Any]) -> str:
directory = str(values(inputs).get("directory") or _DEFAULT_DIRECTORIES[0]["url"]) result = await meta(request, inputs)
result = await meta(request, {"values": {"directory": directory}, "provided": frozenset()})
return str(result.get("termsOfService") or "") return str(result.get("termsOfService") or "")
registry.register("/cluster/acme", "GET", index) registry.register("/cluster/acme", "GET", index)
+19 -14
View File
@@ -154,26 +154,31 @@ def register_backup_handlers(registry: HandlerRegistry) -> None:
vmid = row["vmid"] vmid = row["vmid"]
return [f"qemu/{vmid}"] if vmid is not None else [str(row["volume_id"])] return [f"qemu/{vmid}"] if vmid is not None else [str(row["volume_id"])]
async def vzdump_defaults(_request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def vzdump_defaults(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
await require_node(_request, str(values(inputs)["node"])) from app.handlers.nodes import load_node_ops
return {
"all": 0, node = str(values(inputs)["node"])
"bwlimit": 0, await require_node(request, node)
"compress": "zstd", ops = await load_node_ops(request, node)
"dumpdir": "backup", vzdump = ops.get("vzdump")
"mode": "snapshot", defaults = vzdump.get("defaults") if isinstance(vzdump, dict) else None
"remove": 0, return dict(defaults) if isinstance(defaults, dict) else {}
"storage": "nfs-backup",
"mailto": "",
"notes-template": "{{guestname}}",
}
async def vzdump_extractconfig(request: Request, inputs: dict[str, Any]) -> str: async def vzdump_extractconfig(request: Request, inputs: dict[str, Any]) -> str:
from app.handlers.nodes import load_node_ops
payload = values(inputs) payload = values(inputs)
node = str(payload["node"])
volid = str(payload.get("volume") or payload.get("volid") or "") volid = str(payload.get("volume") or payload.get("volid") or "")
if not volid: if not volid:
raise ApiError(400, "volume parameter required") raise ApiError(400, "volume parameter required")
return f"# simulated vzdump config extracted from {volid}\name: demo\nmemory: 2048\n" await require_node(request, node)
ops = await load_node_ops(request, node)
vzdump = ops.get("vzdump")
configs = vzdump.get("extractconfig") if isinstance(vzdump, dict) else None
if not isinstance(configs, dict):
return ""
return str(configs.get(volid) or "")
async def vzdump_create(request: Request, inputs: dict[str, Any]) -> str: async def vzdump_create(request: Request, inputs: dict[str, Any]) -> str:
payload = values(inputs) payload = values(inputs)
+64 -89
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import json import json
import secrets import secrets
from typing import Any, cast from typing import Any
from fastapi import Request from fastapi import Request
@@ -21,37 +21,6 @@ from app.handlers.common import (
) )
from app.simulation.seed import CLUSTER_ID from app.simulation.seed import CLUSTER_ID
DEFAULT_CLUSTER_CEPH = {
"initialized": True,
"config": {
"network": "10.10.10.0/24",
"cluster-network": "10.10.10.0/24",
"size": 3,
"min_size": 2,
"pg_bits": 7,
},
"cfg_db": [
{"section": "global", "name": "auth_client_required", "value": "cephx"},
{"section": "global", "name": "fsid", "value": "pve-simulator-fsid"},
],
"cfg_raw": "[global]\nfsid = pve-simulator-fsid\nauth_client_required = cephx\n",
"cfg_values": {},
"pools": {
"rbd": {
"pool": "rbd",
"size": 3,
"min_size": 2,
"pg_num": 128,
"application": "rbd",
"crush_rule": "replicated_rule",
}
},
"fs": {},
"rules": [{"name": "replicated_rule", "id": 0}],
"crush": "device 0 osd.0 class hdd\n",
"running": True,
}
async def _load_cluster_ceph(request: Request) -> dict[str, Any]: async def _load_cluster_ceph(request: Request) -> dict[str, Any]:
row = await database(request).pool.fetchrow( row = await database(request).pool.fetchrow(
@@ -60,14 +29,9 @@ async def _load_cluster_ceph(request: Request) -> dict[str, Any]:
) )
metadata = state(row["metadata"]) if row is not None else {} metadata = state(row["metadata"]) if row is not None else {}
ceph = metadata.get("ceph") ceph = metadata.get("ceph")
if not isinstance(ceph, dict) or not ceph: if not isinstance(ceph, dict):
return dict(DEFAULT_CLUSTER_CEPH) return {}
merged = dict(DEFAULT_CLUSTER_CEPH) return dict(ceph)
merged.update(ceph)
for key in ("config", "pools", "fs", "cfg_values"):
if not isinstance(merged.get(key), dict):
merged[key] = dict(cast(dict[str, Any], DEFAULT_CLUSTER_CEPH[key]))
return merged
async def _save_cluster_ceph(request: Request, ceph: dict[str, Any]) -> None: async def _save_cluster_ceph(request: Request, ceph: dict[str, Any]) -> None:
@@ -82,34 +46,27 @@ async def _save_cluster_ceph(request: Request, ceph: dict[str, Any]) -> None:
async def _load_node_ceph(request: Request, node: str) -> dict[str, Any]: async def _load_node_ceph(request: Request, node: str) -> dict[str, Any]:
metadata = await node_metadata(request, node) metadata = await node_metadata(request, node)
ops = metadata.setdefault("ops", {}) ops = metadata.get("ops")
ceph = ops.setdefault( if not isinstance(ops, dict):
"ceph", return {"mds": {}, "mgr": {}, "mon": {}, "log": []}
{ ceph = ops.get("ceph")
"mds": {},
"mgr": {},
"mon": {f"{node}": {"name": node, "addr": f"{node}.local:6789", "rank": 0}},
"log": [{"t": 1_700_000_000, "n": 0, "line": "ceph simulator ready"}],
},
)
if not isinstance(ceph, dict): if not isinstance(ceph, dict):
ceph = { return {"mds": {}, "mgr": {}, "mon": {}, "log": []}
"mds": {}, return {
"mgr": {}, "mds": ceph.get("mds") if isinstance(ceph.get("mds"), dict) else {},
"mon": {}, "mgr": ceph.get("mgr") if isinstance(ceph.get("mgr"), dict) else {},
"log": [], "mon": ceph.get("mon") if isinstance(ceph.get("mon"), dict) else {},
"log": list(ceph.get("log") or []) if isinstance(ceph.get("log"), list) else [],
**{key: value for key, value in ceph.items() if key not in {"mds", "mgr", "mon", "log"}},
} }
ops["ceph"] = ceph
ceph.setdefault("mds", {})
ceph.setdefault("mgr", {})
ceph.setdefault("mon", {})
ceph.setdefault("log", [])
return ceph
async def _save_node_ceph(request: Request, node: str, ceph: dict[str, Any]) -> None: async def _save_node_ceph(request: Request, node: str, ceph: dict[str, Any]) -> None:
metadata = await node_metadata(request, node) metadata = await node_metadata(request, node)
ops = metadata.setdefault("ops", {}) ops = metadata.get("ops")
if not isinstance(ops, dict):
ops = {}
metadata["ops"] = ops
ops["ceph"] = ceph ops["ceph"] = ceph
await save_node_metadata(request, node, metadata) await save_node_metadata(request, node, metadata)
@@ -175,10 +132,12 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
ceph = await _load_cluster_ceph(request) ceph = await _load_cluster_ceph(request)
keys = [item.strip() for item in str(payload.get("config-keys") or "").split(",") if item] keys = [item.strip() for item in str(payload.get("config-keys") or "").split(",") if item]
stored = ceph.setdefault("cfg_values", {}) stored = ceph.get("cfg_values")
result = {key: stored.get(key, "") for key in keys} if keys else dict(stored) if not isinstance(stored, dict):
await _save_cluster_ceph(request, ceph) stored = {}
return result if keys:
return {key: stored.get(key, "") for key in keys}
return dict(stored)
async def crush(request: Request, inputs: dict[str, Any]) -> str: async def crush(request: Request, inputs: dict[str, Any]) -> str:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
@@ -202,12 +161,13 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
async def cmd_safety(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def cmd_safety(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
return { ceph = await _load_cluster_ceph(request)
"safe": 1, safety = ceph.get("cmd_safety")
"action": payload.get("action"), result = dict(safety) if isinstance(safety, dict) else {}
"service": payload.get("service"), for key in ("action", "service", "id"):
"id": payload.get("id"), if key in payload:
} result[key] = payload[key]
return result
async def init(request: Request, inputs: dict[str, Any]) -> None: async def init(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
@@ -368,13 +328,7 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
pool = (ceph.get("pools") or {}).get(name) pool = (ceph.get("pools") or {}).get(name)
if not isinstance(pool, dict): if not isinstance(pool, dict):
raise ApiError(404, "pool does not exist") raise ApiError(404, "pool does not exist")
return { return dict(pool)
**pool,
"pg_num": pool.get("pg_num", 128),
"bytes_used": 0,
"percent_used": 0.0,
"healthy": True,
}
async def fs_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def fs_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
@@ -682,13 +636,16 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
row = await _osd_row(request, node, osdid) row = await _osd_row(request, node, osdid)
current = state(row["state"]) current = state(row["state"])
metadata = current.get("metadata")
if isinstance(metadata, dict):
return dict(metadata)
return { return {
"osd": { "osd": {
"id": int(osdid) if osdid.isdigit() else osdid, "id": int(osdid) if osdid.isdigit() else osdid,
"uuid": current.get("uuid") or f"osd-uuid-{osdid}", "uuid": current.get("uuid", ""),
"device_class": current.get("device_class", "hdd"), "device_class": current.get("device_class", ""),
}, },
"devices": [{"dev": current.get("dev") or f"/dev/sd{osdid}"}], "devices": [{"dev": current.get("dev", "")}] if current.get("dev") else [],
} }
async def cluster_ceph_status(_request: Request, _inputs: dict[str, Any]) -> dict[str, Any]: async def cluster_ceph_status(_request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
@@ -698,17 +655,35 @@ def register_ceph_handlers(registry: HandlerRegistry) -> None:
) )
total = int(row["capacity_bytes"] or 0) if row is not None else 0 total = int(row["capacity_bytes"] or 0) if row is not None else 0
used = int(row["used_bytes"] or 0) if row is not None else 0 used = int(row["used_bytes"] or 0) if row is not None else 0
osd_count = await database(_request).pool.fetchval( osd_rows = await database(_request).pool.fetch(
"SELECT count(*)::int FROM resources WHERE kind='ceph-osd'" "SELECT state FROM resources WHERE kind='ceph-osd'"
) )
num_osds = len(osd_rows)
num_up = 0
num_in = 0
for osd_row in osd_rows:
osd_state = state(osd_row["state"])
if str(osd_state.get("status") or "") == "up":
num_up += 1
if osd_state.get("in") in (True, 1, "1"):
num_in += 1
ceph = await _load_cluster_ceph(_request) ceph = await _load_cluster_ceph(_request)
version = ceph.get("version")
version_str = (
str(version.get("str"))
if isinstance(version, dict) and version.get("str") is not None
else str(version or "")
)
health = ceph.get("health")
if not isinstance(health, dict):
health = {"status": "HEALTH_OK" if ceph.get("running") else "HEALTH_WARN"}
return { return {
"version": "17.2.7", "version": version_str,
"health": {"status": "HEALTH_OK" if ceph.get("running", True) else "HEALTH_WARN"}, "health": health,
"osdmap": { "osdmap": {
"num_osds": osd_count, "num_osds": num_osds,
"num_up_osds": osd_count - 1, "num_up_osds": num_up,
"num_in_osds": osd_count - 1, "num_in_osds": num_in,
}, },
"pgmap": {"bytes_used": used, "bytes_total": total}, "pgmap": {"bytes_used": used, "bytes_total": total},
"fsmap": {"filesystems": list((ceph.get("fs") or {}).keys())}, "fsmap": {"filesystems": list((ceph.get("fs") or {}).keys())},
+17 -16
View File
@@ -46,24 +46,31 @@ def register_cluster_handlers(registry: HandlerRegistry) -> None:
"tasks", "tasks",
) )
async def cluster_status(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def cluster_status(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
rows = await database(_request).pool.fetch( from app.handlers.nodes import load_node_ops
rows = await database(request).pool.fetch(
"""SELECT id, name, status FROM nodes ORDER BY name""" """SELECT id, name, status FROM nodes ORDER BY name"""
) )
metadata = await cluster_metadata(request)
quorate = metadata.get("quorate")
result: list[dict[str, Any]] = [] result: list[dict[str, Any]] = []
for index, row in enumerate(rows): for index, row in enumerate(rows):
online = str(row["status"]) == "online" online = str(row["status"]) == "online"
ops = await load_node_ops(request, str(row["name"]))
cluster_node = ops.get("cluster_status")
entry = dict(cluster_node) if isinstance(cluster_node, dict) else {}
result.append( result.append(
{ {
"id": str(row["id"]), "id": str(row["id"]),
"name": str(row["name"]), "name": str(row["name"]),
"nodeid": index, "nodeid": entry.get("nodeid", index),
"online": 1 if online else 0, "online": 1 if online else 0,
"local": 1 if index == 0 else 0, "local": entry.get("local", 1 if index == 0 else 0),
"ip": f"10.32.{index // 254 + 1}.{index % 254 + 10}", "ip": entry.get("ip") or ops.get("ip") or "",
"level": "c", "level": entry.get("level", "c"),
"type": "node", "type": entry.get("type", "node"),
"quorate": 1, "quorate": entry.get("quorate", quorate if quorate is not None else 1),
} }
) )
return result return result
@@ -95,14 +102,8 @@ def register_cluster_handlers(registry: HandlerRegistry) -> None:
metadata = state(row["metadata"]) if row is not None else {} metadata = state(row["metadata"]) if row is not None else {}
options = metadata.get("options", {}) options = metadata.get("options", {})
if not isinstance(options, dict): if not isinstance(options, dict):
options = {} return {}
return { return dict(options)
"keyboard": options.get("keyboard", "en-us"),
"email_from": options.get("email_from", "root@localhost"),
"http_proxy": options.get("http_proxy", ""),
"description": options.get("description", "Proxmox API emulator cluster"),
**options,
}
async def cluster_options_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def cluster_options_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
current = await cluster_options_get(request, inputs) current = await cluster_options_get(request, inputs)
+36 -39
View File
@@ -19,30 +19,8 @@ from app.handlers.common import (
def _config(metadata: dict[str, Any]) -> dict[str, Any]: def _config(metadata: dict[str, Any]) -> dict[str, Any]:
current = metadata.setdefault( current = metadata.get("cluster_config")
"cluster_config", return current if isinstance(current, dict) else {}
{
"clustername": "pve-simulator",
"votes": 1,
"links": {},
"join_info": {},
"totem": {"version": 2, "secauth": "on", "cluster_name": "pve-simulator"},
"qdevice": {"status": "disabled"},
"apiversion": 1,
},
)
if not isinstance(current, dict):
current = {
"clustername": "pve-simulator",
"votes": 1,
"links": {},
"join_info": {},
"totem": {"version": 2, "secauth": "on", "cluster_name": "pve-simulator"},
"qdevice": {"status": "disabled"},
"apiversion": 1,
}
metadata["cluster_config"] = current
return current
def register_cluster_config_handlers(registry: HandlerRegistry) -> None: def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
@@ -52,10 +30,12 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
async def create(request: Request, inputs: dict[str, Any]) -> None: async def create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
config = _config(metadata) config = dict(_config(metadata))
if payload.get("clustername"): if payload.get("clustername"):
config["clustername"] = str(payload["clustername"]) config["clustername"] = str(payload["clustername"])
config.setdefault("totem", {})["cluster_name"] = str(payload["clustername"]) totem = dict(config.get("totem") or {})
totem["cluster_name"] = str(payload["clustername"])
config["totem"] = totem
if "votes" in payload: if "votes" in payload:
config["votes"] = payload["votes"] config["votes"] = payload["votes"]
if "nodeid" in payload: if "nodeid" in payload:
@@ -64,6 +44,7 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
if links: if links:
config["links"] = links config["links"] = links
config["token"] = secrets.token_hex(16) config["token"] = secrets.token_hex(16)
metadata["cluster_config"] = config
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
await database(request).pool.execute( await database(request).pool.execute(
"""UPDATE clusters """UPDATE clusters
@@ -97,11 +78,11 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
async def join_post(request: Request, inputs: dict[str, Any]) -> None: async def join_post(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
config = _config(metadata) config = dict(_config(metadata))
hostname = str(payload.get("hostname") or payload.get("node") or "") hostname = str(payload.get("hostname") or payload.get("node") or "")
if not hostname: if not hostname:
raise ApiError(400, "parameter verification failed - 'hostname' missing") raise ApiError(400, "parameter verification failed - 'hostname' missing")
joins = config.setdefault("join_info", {}) joins = dict(config.get("join_info") or {})
joins[hostname] = { joins[hostname] = {
"hostname": hostname, "hostname": hostname,
"fingerprint": payload.get("fingerprint"), "fingerprint": payload.get("fingerprint"),
@@ -112,6 +93,8 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
# password accepted but not stored in clear form # password accepted but not stored in clear form
if payload.get("password"): if payload.get("password"):
joins[hostname]["password_set"] = True joins[hostname]["password_set"] = True
config["join_info"] = joins
metadata["cluster_config"] = config
exists = await database(request).pool.fetchval( exists = await database(request).pool.fetchval(
"SELECT EXISTS(SELECT 1 FROM nodes WHERE name=$1)", "SELECT EXISTS(SELECT 1 FROM nodes WHERE name=$1)",
hostname, hostname,
@@ -128,14 +111,19 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
rows = await database(request).pool.fetch("SELECT name, status FROM nodes ORDER BY name") rows = await database(request).pool.fetch("SELECT name, status FROM nodes ORDER BY name")
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
config = _config(metadata) config = _config(metadata)
added_raw = config.get("added_nodes")
added: dict[str, Any] = dict(added_raw) if isinstance(added_raw, dict) else {}
result = [] result = []
for index, row in enumerate(rows, start=1): for index, row in enumerate(rows, start=1):
name = str(row["name"])
entry_raw = added.get(name)
entry: dict[str, Any] = dict(entry_raw) if isinstance(entry_raw, dict) else {}
result.append( result.append(
{ {
"node": str(row["name"]), "node": name,
"nodeid": index, "nodeid": entry.get("nodeid", index),
"ring0_addr": f"{row['name']}.local", "ring0_addr": entry.get("ring0_addr", ""),
"quorum_votes": config.get("votes", 1), "quorum_votes": entry.get("quorum_votes", config.get("votes", 0)),
} }
) )
return result return result
@@ -144,8 +132,8 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
node = str(payload["node"]) node = str(payload["node"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
config = _config(metadata) config = dict(_config(metadata))
added = config.setdefault("added_nodes", {}) added = dict(config.get("added_nodes") or {})
added[node] = { added[node] = {
"node": node, "node": node,
"nodeid": payload.get("nodeid"), "nodeid": payload.get("nodeid"),
@@ -153,10 +141,14 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
"votes": payload.get("votes", 1), "votes": payload.get("votes", 1),
"apiversion": payload.get("apiversion"), "apiversion": payload.get("apiversion"),
"force": payload.get("force"), "force": payload.get("force"),
"ring0_addr": payload.get("ring0_addr") or f"{node}.local",
"quorum_votes": payload.get("quorum_votes", payload.get("votes", 1)),
} }
links = {key: value for key, value in payload.items() if key.startswith("link")} links = {key: value for key, value in payload.items() if key.startswith("link")}
if links: if links:
added[node]["links"] = links added[node]["links"] = links
config["added_nodes"] = added
metadata["cluster_config"] = config
exists = await database(request).pool.fetchval( exists = await database(request).pool.fetchval(
"SELECT EXISTS(SELECT 1 FROM nodes WHERE name=$1)", "SELECT EXISTS(SELECT 1 FROM nodes WHERE name=$1)",
node, node,
@@ -172,11 +164,14 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
async def nodes_delete(request: Request, inputs: dict[str, Any]) -> None: async def nodes_delete(request: Request, inputs: dict[str, Any]) -> None:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
config = _config(metadata) config = dict(_config(metadata))
added = config.setdefault("added_nodes", {}) added = dict(config.get("added_nodes") or {})
added.pop(node, None) added.pop(node, None)
joins = config.setdefault("join_info", {}) joins = dict(config.get("join_info") or {})
joins.pop(node, None) joins.pop(node, None)
config["added_nodes"] = added
config["join_info"] = joins
metadata["cluster_config"] = config
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
# Keep node row; mark offline to avoid cascading guest deletes. # Keep node row; mark offline to avoid cascading guest deletes.
await database(request).pool.execute( await database(request).pool.execute(
@@ -186,11 +181,13 @@ def register_cluster_config_handlers(registry: HandlerRegistry) -> None:
async def qdevice(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]: async def qdevice(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
return dict(_config(metadata).get("qdevice") or {"status": "disabled"}) qdevice = _config(metadata).get("qdevice")
return dict(qdevice) if isinstance(qdevice, dict) else {}
async def totem(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]: async def totem(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
return dict(_config(metadata).get("totem") or {}) totem = _config(metadata).get("totem")
return dict(totem) if isinstance(totem, dict) else {}
registry.register("/cluster/config", "GET", index) registry.register("/cluster/config", "GET", index)
registry.register("/cluster/config", "POST", create) registry.register("/cluster/config", "POST", create)
+60 -96
View File
@@ -2,7 +2,6 @@
from __future__ import annotations from __future__ import annotations
import copy
import secrets import secrets
import time import time
from typing import Any from typing import Any
@@ -22,56 +21,20 @@ from app.handlers.common import (
from app.tasks.repository import TaskRepository from app.tasks.repository import TaskRepository
from app.tasks.upid import Upid from app.tasks.upid import Upid
DEFAULT_CEPH_FLAGS: dict[str, int] = {
"nobackfill": 0,
"nodeep-scrub": 0,
"nodown": 0,
"noin": 0,
"noout": 0,
"norebalance": 0,
"norecover": 0,
"noscrub": 0,
"notieragent": 0,
"pause": 0,
}
DEFAULT_CPU_FLAGS: list[dict[str, Any]] = [
{"name": "aes", "introduces": "Westmere"},
{"name": "avx", "introduces": "SandyBridge"},
{"name": "avx2", "introduces": "Haswell"},
]
def _jobs(metadata: dict[str, Any]) -> dict[str, Any]: def _jobs(metadata: dict[str, Any]) -> dict[str, Any]:
jobs = metadata.setdefault("jobs", {}) jobs = metadata.get("jobs")
if not isinstance(jobs, dict): return jobs if isinstance(jobs, dict) else {}
jobs = {}
metadata["jobs"] = jobs
sync = jobs.setdefault("realm_sync", {})
if not isinstance(sync, dict):
sync = {}
jobs["realm_sync"] = sync
return jobs
def _metrics(metadata: dict[str, Any]) -> dict[str, Any]: def _metrics(metadata: dict[str, Any]) -> dict[str, Any]:
metrics = metadata.setdefault("metrics", {}) metrics = metadata.get("metrics")
if not isinstance(metrics, dict): return metrics if isinstance(metrics, dict) else {}
metrics = {}
metadata["metrics"] = metrics
servers = metrics.setdefault("servers", {})
if not isinstance(servers, dict):
servers = {}
metrics["servers"] = servers
return metrics
def _cpu_models(metadata: dict[str, Any]) -> dict[str, Any]: def _cpu_models(metadata: dict[str, Any]) -> dict[str, Any]:
models = metadata.get("qemu_cpu_models") models = metadata.get("qemu_cpu_models")
if not isinstance(models, dict): return models if isinstance(models, dict) else {}
models = {}
metadata["qemu_cpu_models"] = models
return models
def _ha_rules_store(metadata: dict[str, Any]) -> list[dict[str, Any]]: def _ha_rules_store(metadata: dict[str, Any]) -> list[dict[str, Any]]:
@@ -84,12 +47,7 @@ def _ha_rules_store(metadata: dict[str, Any]) -> list[dict[str, Any]]:
] ]
if isinstance(rules, list): if isinstance(rules, list):
return [dict(item) for item in rules if isinstance(item, dict)] return [dict(item) for item in rules if isinstance(item, dict)]
defaults = [ return []
{"rule": "node-fencing", "type": "node", "action": "restart"},
{"rule": "service-ha", "type": "resource", "action": "failover"},
]
metadata["ha_rules"] = defaults
return list(defaults)
def _save_ha_rules(metadata: dict[str, Any], rules: list[dict[str, Any]]) -> None: def _save_ha_rules(metadata: dict[str, Any], rules: list[dict[str, Any]]) -> None:
@@ -105,18 +63,7 @@ def _replication_jobs(metadata: dict[str, Any]) -> list[dict[str, Any]]:
def _ceph(metadata: dict[str, Any]) -> dict[str, Any]: def _ceph(metadata: dict[str, Any]) -> dict[str, Any]:
ceph = metadata.get("ceph") ceph = metadata.get("ceph")
if not isinstance(ceph, dict): return ceph if isinstance(ceph, dict) else {}
ceph = {}
flags = ceph.get("flags")
if not isinstance(flags, dict):
flags = copy.deepcopy(DEFAULT_CEPH_FLAGS)
else:
merged = copy.deepcopy(DEFAULT_CEPH_FLAGS)
merged.update({str(key): int(value) for key, value in flags.items()})
flags = merged
ceph["flags"] = flags
metadata["ceph"] = ceph
return ceph
async def _cluster_task(request: Request, *, task_type: str, worker: str) -> str: async def _cluster_task(request: Request, *, task_type: str, worker: str) -> str:
@@ -174,8 +121,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
job_id = str(payload["id"]) job_id = str(payload["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
jobs = _jobs(metadata) jobs = dict(_jobs(metadata))
sync = jobs.setdefault("realm_sync", {}) sync = dict(jobs.get("realm_sync") or {})
if job_id in sync: if job_id in sync:
raise ApiError(409, "realm-sync job already exists") raise ApiError(409, "realm-sync job already exists")
entry = { entry = {
@@ -185,6 +132,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
entry.setdefault("enabled", 1) entry.setdefault("enabled", 1)
entry.setdefault("realm", str(payload.get("realm") or "pam")) entry.setdefault("realm", str(payload.get("realm") or "pam"))
sync[job_id] = entry sync[job_id] = entry
jobs["realm_sync"] = sync
metadata["jobs"] = jobs metadata["jobs"] = jobs
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
return {"id": job_id, **entry} return {"id": job_id, **entry}
@@ -193,8 +141,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
job_id = str(payload["id"]) job_id = str(payload["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
jobs = _jobs(metadata) jobs = dict(_jobs(metadata))
sync = jobs.setdefault("realm_sync", {}) sync = dict(jobs.get("realm_sync") or {})
if job_id not in sync: if job_id not in sync:
raise ApiError(404, "realm-sync job does not exist") raise ApiError(404, "realm-sync job does not exist")
updated = { updated = {
@@ -206,6 +154,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
}, },
} }
sync[job_id] = updated sync[job_id] = updated
jobs["realm_sync"] = sync
metadata["jobs"] = jobs metadata["jobs"] = jobs
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
return {"id": job_id, **updated} return {"id": job_id, **updated}
@@ -213,23 +162,26 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
async def realm_sync_delete(request: Request, inputs: dict[str, Any]) -> None: async def realm_sync_delete(request: Request, inputs: dict[str, Any]) -> None:
job_id = str(values(inputs)["id"]) job_id = str(values(inputs)["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
jobs = _jobs(metadata) jobs = dict(_jobs(metadata))
sync = jobs.setdefault("realm_sync", {}) sync = dict(jobs.get("realm_sync") or {})
if job_id not in sync: if job_id not in sync:
raise ApiError(404, "realm-sync job does not exist") raise ApiError(404, "realm-sync job does not exist")
del sync[job_id] del sync[job_id]
jobs["realm_sync"] = sync
metadata["jobs"] = jobs metadata["jobs"] = jobs
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def schedule_analyze(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def schedule_analyze(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
schedule = str(values(inputs).get("schedule") or "*/15") schedule = str(values(inputs).get("schedule") or "*/15")
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
jobs = _jobs(metadata) jobs = dict(_jobs(metadata))
results = jobs.get("schedule_analyze_results")
if not isinstance(results, list):
results = []
jobs["last_schedule_analyze"] = {"schedule": schedule, "at": int(time.time())} jobs["last_schedule_analyze"] = {"schedule": schedule, "at": int(time.time())}
metadata["jobs"] = jobs metadata["jobs"] = jobs
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
now = int(time.time()) return [dict(item) for item in results if isinstance(item, dict)]
return [{"timestamp": now + offset * 900, "utc": True} for offset in range(4)]
async def metrics_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]: async def metrics_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
return subdirs("export", "server") return subdirs("export", "server")
@@ -238,8 +190,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
metrics = _metrics(metadata) metrics = _metrics(metadata)
return { return {
"data": metrics.get("export_data") "data": str(metrics.get("export_data") or ""),
or '# HELP pve_up Node is up\npve_up{node="pve01"} 1\n',
"timestamp": int(time.time()), "timestamp": int(time.time()),
} }
@@ -248,9 +199,12 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
metrics = _metrics(metadata) metrics = _metrics(metadata)
servers = metrics.get("servers")
if not isinstance(servers, dict):
return []
return [ return [
{"id": server_id, **dict(payload)} {"id": server_id, **dict(payload)}
for server_id, payload in sorted(metrics.get("servers", {}).items()) for server_id, payload in sorted(servers.items())
if isinstance(payload, dict) if isinstance(payload, dict)
] ]
@@ -267,8 +221,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
server_id = str(payload["id"]) server_id = str(payload["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
metrics = _metrics(metadata) metrics = dict(_metrics(metadata))
servers = metrics.setdefault("servers", {}) servers = dict(metrics.get("servers") or {})
if server_id in servers: if server_id in servers:
raise ApiError(409, "metrics server already exists") raise ApiError(409, "metrics server already exists")
entry = { entry = {
@@ -279,6 +233,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
entry.setdefault("port", 8086) entry.setdefault("port", 8086)
entry.setdefault("enable", 1) entry.setdefault("enable", 1)
servers[server_id] = entry servers[server_id] = entry
metrics["servers"] = servers
metadata["metrics"] = metrics metadata["metrics"] = metrics
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
return {"id": server_id, **entry} return {"id": server_id, **entry}
@@ -287,8 +242,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
server_id = str(payload["id"]) server_id = str(payload["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
metrics = _metrics(metadata) metrics = dict(_metrics(metadata))
servers = metrics.setdefault("servers", {}) servers = dict(metrics.get("servers") or {})
if server_id not in servers: if server_id not in servers:
raise ApiError(404, "metrics server does not exist") raise ApiError(404, "metrics server does not exist")
updated = { updated = {
@@ -300,6 +255,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
}, },
} }
servers[server_id] = updated servers[server_id] = updated
metrics["servers"] = servers
metadata["metrics"] = metrics metadata["metrics"] = metrics
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
return {"id": server_id, **updated} return {"id": server_id, **updated}
@@ -307,19 +263,24 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
async def metrics_server_delete(request: Request, inputs: dict[str, Any]) -> None: async def metrics_server_delete(request: Request, inputs: dict[str, Any]) -> None:
server_id = str(values(inputs)["id"]) server_id = str(values(inputs)["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
metrics = _metrics(metadata) metrics = dict(_metrics(metadata))
servers = metrics.setdefault("servers", {}) servers = dict(metrics.get("servers") or {})
if server_id not in servers: if server_id not in servers:
raise ApiError(404, "metrics server does not exist") raise ApiError(404, "metrics server does not exist")
del servers[server_id] del servers[server_id]
metrics["servers"] = servers
metadata["metrics"] = metrics metadata["metrics"] = metrics
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def qemu_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]: async def qemu_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
return subdirs("cpu-flags", "custom-cpu-models") return subdirs("cpu-flags", "custom-cpu-models")
async def qemu_cpu_flags(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def qemu_cpu_flags(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
return list(DEFAULT_CPU_FLAGS) metadata = await cluster_metadata(request)
flags = metadata.get("qemu_cpu_flags")
if isinstance(flags, list):
return [dict(item) for item in flags if isinstance(item, dict)]
return []
async def cpu_models_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def cpu_models_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
@@ -440,14 +401,16 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
async def ceph_flags_get(request: Request, _inputs: dict[str, Any]) -> dict[str, int]: async def ceph_flags_get(request: Request, _inputs: dict[str, Any]) -> dict[str, int]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ceph = _ceph(metadata) ceph = _ceph(metadata)
await save_cluster_metadata(request, metadata) flags = ceph.get("flags")
return {str(key): int(value) for key, value in ceph["flags"].items()} if not isinstance(flags, dict):
return {}
return {str(key): int(value) for key, value in flags.items()}
async def ceph_flags_put(request: Request, inputs: dict[str, Any]) -> dict[str, int]: async def ceph_flags_put(request: Request, inputs: dict[str, Any]) -> dict[str, int]:
payload = values(inputs) payload = values(inputs)
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ceph = _ceph(metadata) ceph = dict(_ceph(metadata))
flags = dict(ceph["flags"]) flags = dict(ceph.get("flags") or {})
for key, value in payload.items(): for key, value in payload.items():
if key in {"delete", "digest"}: if key in {"delete", "digest"}:
continue continue
@@ -461,8 +424,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
flag = str(values(inputs)["flag"]) flag = str(values(inputs)["flag"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ceph = _ceph(metadata) ceph = _ceph(metadata)
flags = ceph["flags"] flags = ceph.get("flags")
if flag not in flags: if not isinstance(flags, dict) or flag not in flags:
raise ApiError(404, "ceph flag does not exist") raise ApiError(404, "ceph flag does not exist")
return {flag: int(flags[flag])} return {flag: int(flags[flag])}
@@ -470,8 +433,8 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
flag = str(payload["flag"]) flag = str(payload["flag"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ceph = _ceph(metadata) ceph = dict(_ceph(metadata))
flags = dict(ceph["flags"]) flags = dict(ceph.get("flags") or {})
if "value" in payload: if "value" in payload:
flags[flag] = int(payload["value"]) flags[flag] = int(payload["value"])
elif flag in payload: elif flag in payload:
@@ -486,14 +449,15 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
async def ceph_metadata(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]: async def ceph_metadata(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ceph = _ceph(metadata) ceph = _ceph(metadata)
await save_cluster_metadata(request, metadata) config_raw = ceph.get("config")
config: dict[str, Any] = dict(config_raw) if isinstance(config_raw, dict) else {}
version = ceph.get("version")
flags = ceph.get("flags")
return { return {
"version": ceph.get("version") or {"str": "18.2.2", "parts": [18, 2, 2]}, "version": dict(version) if isinstance(version, dict) else {},
"fsid": ceph.get("config", {}).get("fsid") "fsid": str(config.get("fsid") or ""),
if isinstance(ceph.get("config"), dict) "initialized": int(bool(ceph.get("initialized"))),
else "pve-simulator-fsid", "flags": dict(flags) if isinstance(flags, dict) else {},
"initialized": int(bool(ceph.get("initialized", True))),
"flags": ceph.get("flags", {}),
} }
async def ha_rule_create(request: Request, inputs: dict[str, Any]) -> None: async def ha_rule_create(request: Request, inputs: dict[str, Any]) -> None:
@@ -588,7 +552,7 @@ def register_cluster_extra_handlers(registry: HandlerRegistry) -> None:
log = job.get("log") log = job.get("log")
if isinstance(log, list): if isinstance(log, list):
return [dict(item) for item in log if isinstance(item, dict)] return [dict(item) for item in log if isinstance(item, dict)]
return [{"t": int(time.time()), "n": 0, "msg": f"replication idle for {job.get('id')}"}] return []
async def node_replication_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_replication_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
job = await node_replication_get(request, inputs) job = await node_replication_get(request, inputs)
+189 -13
View File
@@ -68,42 +68,218 @@ def build_core_handlers(settings: Settings) -> HandlerRegistry:
} }
async def nodes(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def nodes(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
from app.handlers.nodes import load_node_ops
rows = await _database(request).pool.fetch( rows = await _database(request).pool.fetch(
"SELECT name AS node, status FROM nodes ORDER BY name" "SELECT name AS node, status FROM nodes ORDER BY name"
) )
return [{"node": str(row["node"]), "status": str(row["status"])} for row in rows] result: list[dict[str, Any]] = []
for row in rows:
name = str(row["node"])
ops = await load_node_ops(request, name)
status_payload = ops.get("status")
status_dict = dict(status_payload) if isinstance(status_payload, dict) else {}
fingerprint = status_dict.get("ssl_fingerprint") or status_dict.get("fingerprint")
if fingerprint in (None, "", 0) or isinstance(fingerprint, dict | list):
fingerprint = ":".join(["00"] * 32)
def _as_float(value: object, default: float) -> float:
if isinstance(value, bool) or value is None or isinstance(value, dict | list):
return default
try:
return float(str(value))
except (TypeError, ValueError):
return default
def _as_int(value: object, default: int) -> int:
if isinstance(value, bool) or value is None or isinstance(value, dict | list):
return default
try:
return int(float(str(value)))
except (TypeError, ValueError):
return default
item: dict[str, Any] = {
"node": name,
"status": str(row["status"]),
"type": "node",
"ssl_fingerprint": str(fingerprint),
"cpu": _as_float(status_dict.get("cpu"), 0.0),
"maxcpu": _as_int(status_dict.get("maxcpu"), 4),
"mem": _as_int(status_dict.get("mem"), _as_int(status_dict.get("memory"), 0)),
"maxmem": _as_int(status_dict.get("maxmem"), 8 * 1024**3),
"uptime": _as_int(status_dict.get("uptime"), 0),
"level": str(status_dict.get("level") or ""),
}
result.append(item)
return result
async def node_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
from app.handlers.nodes import load_node_ops
node = str(cast(dict[str, Any], inputs["values"])["node"]) node = str(cast(dict[str, Any], inputs["values"])["node"])
row = await _database(request).pool.fetchrow( row = await _database(request).pool.fetchrow(
"SELECT name, status FROM nodes WHERE name=$1", node "SELECT name, status FROM nodes WHERE name=$1", node
) )
if row is None: if row is None:
raise ApiError(404, "node does not exist") raise ApiError(404, "node does not exist")
ops = await load_node_ops(request, node)
status = ops.get("status")
payload = dict(status) if isinstance(status, dict) else {}
return { return {
"status": str(row["status"]), "status": str(row["status"]),
"node": str(row["name"]), "node": str(row["name"]),
"uptime": 0, **payload,
"cpu": 0.0,
"memory": {"used": 0, "total": 0},
} }
async def resources(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def resources(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
"""Cluster-wide inventory in Proxmox ``/cluster/resources`` shape.
Do not dump guest ``state`` / config blobs: QEMU ``cpu`` is a model
string (e.g. ``qemu64``), while this endpoint's ``cpu`` is utilization
(float). Bridged clients (bpg / pulumi-proxmoxve) decode strictly.
"""
def _as_dict(raw: object) -> dict[str, Any]:
if isinstance(raw, str):
loaded = json.loads(raw)
return dict(loaded) if isinstance(loaded, dict) else {}
return dict(raw) if isinstance(raw, dict) else {}
def _num(value: object, default: float | int) -> float | int:
if isinstance(value, bool) or value is None or isinstance(value, dict | list):
return default
try:
if isinstance(default, float):
return float(str(value))
return int(float(str(value)))
except (TypeError, ValueError):
return default
def _memory_bytes(state: dict[str, Any]) -> int:
raw = state.get("maxmem", state.get("memory"))
if raw in (None, ""):
return 0
value = _num(raw, 0)
# QEMU config stores memory in MiB; cluster resources use bytes.
if isinstance(raw, str) or (isinstance(value, int) and 0 < value < 10_000_000):
return int(value) * 1024 * 1024
return int(value)
def _maxcpu(state: dict[str, Any]) -> int:
cores = int(_num(state.get("cores", state.get("cpus", 1)), 1))
sockets = int(_num(state.get("sockets", 1), 1))
return max(cores * sockets, 1)
def _cpu_util(state: dict[str, Any], *, running: bool) -> float:
if not running:
return 0.0
samples = state.get("rrddata")
if isinstance(samples, list) and samples:
last = samples[-1]
if isinstance(last, dict):
return float(_num(last.get("cpu"), 0.0))
return (
float(_num(state.get("cpu"), 0.0)) if not isinstance(state.get("cpu"), str) else 0.0
)
type_filter = cast(dict[str, Any], inputs.get("values") or {}).get("type")
result: list[dict[str, Any]] = []
if type_filter in (None, "node"):
node_rows = await _database(request).pool.fetch(
"SELECT name AS node, status FROM nodes ORDER BY name"
)
from app.handlers.nodes import load_node_ops
for row in node_rows:
name = str(row["node"])
ops = await load_node_ops(request, name)
status_payload = ops.get("status")
status_dict = dict(status_payload) if isinstance(status_payload, dict) else {}
result.append(
{
"type": "node",
"id": f"node/{name}",
"node": name,
"status": str(row["status"]),
"cpu": float(_num(status_dict.get("cpu"), 0.0)),
"maxcpu": int(_num(status_dict.get("maxcpu"), 4)),
"mem": int(
_num(status_dict.get("mem"), _num(status_dict.get("memory"), 0))
),
"maxmem": int(_num(status_dict.get("maxmem"), 8 * 1024**3)),
"uptime": int(_num(status_dict.get("uptime"), 0)),
"level": str(status_dict.get("level") or ""),
}
)
kind_filter: tuple[str, ...] | None
if type_filter == "vm":
kind_filter = ("qemu", "lxc")
elif type_filter == "storage":
kind_filter = ("storage",)
elif type_filter in (None,):
kind_filter = ("qemu", "lxc", "storage")
elif type_filter in {"qemu", "lxc", "storage", "pool", "sdn"}:
kind_filter = (str(type_filter),)
else:
kind_filter = ()
if kind_filter:
rows = await _database(request).pool.fetch( rows = await _database(request).pool.fetch(
"""SELECT r.kind AS type, r.external_id, r.state, n.name AS node """SELECT r.kind AS type, r.external_id, r.state, n.name AS node
FROM resources r JOIN nodes n ON n.id=r.node_id FROM resources r JOIN nodes n ON n.id=r.node_id
ORDER BY r.kind, r.external_id""" WHERE r.kind = ANY($1::text[])
ORDER BY r.kind, r.external_id""",
list(kind_filter),
) )
result: list[dict[str, Any]] = []
for row in rows: for row in rows:
raw_state = row["state"] kind = str(row["type"])
state = json.loads(raw_state) if isinstance(raw_state, str) else dict(raw_state) external_id = str(row["external_id"])
node = str(row["node"])
state = _as_dict(row["state"])
if kind in {"qemu", "lxc"}:
status = str(state.get("status") or "stopped")
running = status in {"running", "paused"}
vmid = int(external_id)
item: dict[str, Any] = {
"type": kind,
"id": f"{kind}/{external_id}",
"node": node,
"vmid": vmid,
"name": str(state.get("name") or f"{kind}-{external_id}"),
"status": status,
"template": 1 if state.get("template") in {True, "1"} else 0,
"cpu": _cpu_util(state, running=running),
"maxcpu": _maxcpu(state),
"mem": int(_num(state.get("mem"), 0)) if running else 0,
"maxmem": _memory_bytes(state),
"disk": int(_num(state.get("disk"), 0)),
"maxdisk": int(_num(state.get("maxdisk"), 0)),
"uptime": int(_num(state.get("uptime"), 0)) if running else 0,
}
result.append(item)
elif kind == "storage":
content = state.get("content")
if isinstance(content, list):
content_text = ",".join(str(item) for item in content)
else:
content_text = str(content or "")
result.append( result.append(
{ {
"type": str(row["type"]), "type": "storage",
"id": f"{row['type']}/{row['external_id']}", "id": f"storage/{node}/{external_id}",
"node": str(row["node"]), "node": node,
**state, "storage": external_id,
"status": str(state.get("status") or "available"),
"content": content_text,
"disk": int(_num(state.get("disk"), 0)),
"maxdisk": int(_num(state.get("maxdisk"), 1 * 1024**3)),
"shared": int(_num(state.get("shared"), 0)),
"plugintype": str(
state.get("plugintype") or state.get("type") or "dir"
),
} }
) )
return result return result
+67 -34
View File
@@ -13,20 +13,6 @@ from app.api.registry import HandlerRegistry
from app.handlers.common import database, require_node, state, subdirs, values from app.handlers.common import database, require_node, state, subdirs, values
from app.simulation.seed import CLUSTER_ID from app.simulation.seed import CLUSTER_ID
DEFAULT_OPTIONS = {
"enable": 1,
"policy_in": "DROP",
"policy_out": "ACCEPT",
"log_level_in": "nolog",
"log_level_out": "nolog",
}
DEFAULT_MACROS = [
{"macro": "SSH", "descr": "Secure Shell"},
{"macro": "HTTPS", "descr": "Secure web server"},
{"macro": "HTTP", "descr": "Web server"},
]
ScopeFn = Callable[[dict[str, Any]], str] ScopeFn = Callable[[dict[str, Any]], str]
@@ -50,19 +36,36 @@ async def _save_firewall(request: Request, firewall: dict[str, Any]) -> None:
) )
def _scope_data(firewall: dict[str, Any], scope: str) -> dict[str, Any]: def _empty_scope() -> dict[str, Any]:
scopes = firewall.setdefault("scopes", {}) return {
if scope not in scopes or not isinstance(scopes[scope], dict): "options": {},
scopes[scope] = {
"options": dict(DEFAULT_OPTIONS),
"rules": [], "rules": [],
"aliases": {}, "aliases": {},
"ipset": {}, "ipset": {},
"groups": {}, "groups": {},
"log": [], "log": [],
} }
section = scopes[scope]
section.setdefault("options", dict(DEFAULT_OPTIONS))
def _get_scope(firewall: dict[str, Any], scope: str) -> dict[str, Any] | None:
scopes = firewall.get("scopes")
if not isinstance(scopes, dict):
return None
section = scopes.get(scope)
return cast(dict[str, Any], section) if isinstance(section, dict) else None
def _ensure_scope(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
"""Create an empty durable scope on mutation; never injects catalog defaults."""
scopes = firewall.setdefault("scopes", {})
if not isinstance(scopes, dict):
scopes = {}
firewall["scopes"] = scopes
section = scopes.get(scope)
if not isinstance(section, dict):
section = _empty_scope()
scopes[scope] = section
section.setdefault("options", {})
section.setdefault("rules", []) section.setdefault("rules", [])
section.setdefault("aliases", {}) section.setdefault("aliases", {})
section.setdefault("ipset", {}) section.setdefault("ipset", {})
@@ -71,6 +74,11 @@ def _scope_data(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
return cast(dict[str, Any], section) return cast(dict[str, Any], section)
def _scope_data(firewall: dict[str, Any], scope: str) -> dict[str, Any]:
"""Mutation helper — ensure scope exists without template defaults."""
return _ensure_scope(firewall, scope)
def register_firewall_handlers(registry: HandlerRegistry) -> None: def register_firewall_handlers(registry: HandlerRegistry) -> None:
def register_scope( def register_scope(
base: str, base: str,
@@ -98,13 +106,17 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
async def options_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def options_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
return dict(_scope_data(firewall, scope_fn(payload)).get("options", DEFAULT_OPTIONS)) section = _get_scope(firewall, scope_fn(payload))
if section is None:
return {}
options = section.get("options", {})
return dict(options) if isinstance(options, dict) else {}
async def options_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def options_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
section = _scope_data(firewall, scope_fn(payload)) section = _ensure_scope(firewall, scope_fn(payload))
current = dict(section.get("options", DEFAULT_OPTIONS)) current = dict(section.get("options") or {})
for key, value in payload.items(): for key, value in payload.items():
if key in {"node", "vmid", "delete", "digest"}: if key in {"node", "vmid", "delete", "digest"}:
continue continue
@@ -116,7 +128,10 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
async def rules_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def rules_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
rules = _scope_data(firewall, scope_fn(payload)).get("rules", []) section = _get_scope(firewall, scope_fn(payload))
if section is None:
return []
rules = section.get("rules", [])
return list(rules) if isinstance(rules, list) else [] return list(rules) if isinstance(rules, list) else []
async def rules_create(request: Request, inputs: dict[str, Any]) -> None: async def rules_create(request: Request, inputs: dict[str, Any]) -> None:
@@ -169,7 +184,8 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
async def aliases_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def aliases_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
aliases = _scope_data(firewall, scope_fn(payload)).get("aliases", {}) section = _get_scope(firewall, scope_fn(payload))
aliases = section.get("aliases", {}) if section else {}
if not isinstance(aliases, dict): if not isinstance(aliases, dict):
return [] return []
return [ return [
@@ -236,7 +252,8 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
async def ipset_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def ipset_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
ipsets = _scope_data(firewall, scope_fn(payload)).get("ipset", {}) section = _get_scope(firewall, scope_fn(payload))
ipsets = section.get("ipset", {}) if section else {}
if not isinstance(ipsets, dict): if not isinstance(ipsets, dict):
return [] return []
return [ return [
@@ -352,27 +369,43 @@ def register_firewall_handlers(registry: HandlerRegistry) -> None:
async def refs_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def refs_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
section = _scope_data(firewall, scope_fn(payload)) section = _get_scope(firewall, scope_fn(payload))
if section is None:
return []
refs: list[dict[str, Any]] = [] refs: list[dict[str, Any]] = []
for name in section.get("aliases", {}): for name in section.get("aliases", {}) or {}:
refs.append({"type": "alias", "name": name}) refs.append({"type": "alias", "name": name})
for name in section.get("ipset", {}): for name in section.get("ipset", {}) or {}:
refs.append({"type": "ipset", "name": name}) refs.append({"type": "ipset", "name": name})
return refs return refs
async def log_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def log_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
log = _scope_data(firewall, scope_fn(payload)).get("log", []) section = _get_scope(firewall, scope_fn(payload))
if section is None:
return []
log = section.get("log", [])
return list(log) if isinstance(log, list) else [] return list(log) if isinstance(log, list) else []
async def macros_list(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def macros_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
return list(DEFAULT_MACROS) from app.handlers.common import cluster_metadata
metadata = await cluster_metadata(request)
macros = metadata.get("firewall_macros")
if isinstance(macros, list):
return [dict(item) for item in macros if isinstance(item, dict)]
firewall = await _load_firewall(request)
nested = firewall.get("macros")
if isinstance(nested, list):
return [dict(item) for item in nested if isinstance(item, dict)]
return []
async def groups_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def groups_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = await _ready(request, inputs) payload = await _ready(request, inputs)
firewall = await _load_firewall(request) firewall = await _load_firewall(request)
groups = _scope_data(firewall, scope_fn(payload)).get("groups", {}) section = _get_scope(firewall, scope_fn(payload))
groups = section.get("groups", {}) if section else {}
if not isinstance(groups, dict): if not isinstance(groups, dict):
return [] return []
return [ return [
+19 -38
View File
@@ -142,7 +142,7 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
async def ha_groups(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def ha_groups(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
configured = _ha_groups(metadata) configured = _ha_groups(metadata)
result = [ return [
{ {
"group": group_id, "group": group_id,
"nodes": str(payload.get("nodes", "")), "nodes": str(payload.get("nodes", "")),
@@ -153,26 +153,6 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
} }
for group_id, payload in sorted(configured.items()) for group_id, payload in sorted(configured.items())
] ]
if result:
return result
rows = await database(request).pool.fetch(
"""SELECT DISTINCT state->>'group' AS group_id
FROM resources WHERE kind='ha' AND state ? 'group'
ORDER BY 1"""
)
node_names = await database(request).pool.fetch("SELECT name FROM nodes ORDER BY name")
nodes = ",".join(str(row["name"]) for row in node_names) or "pve01"
return [
{
"group": str(row["group_id"]),
"nodes": nodes,
"nofailback": 0,
"restricted": 0,
"type": "group",
}
for row in rows
if row["group_id"]
]
async def ha_group_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def ha_group_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
group = str(values(inputs)["group"]) group = str(values(inputs)["group"])
@@ -238,38 +218,39 @@ def register_ha_handlers(registry: HandlerRegistry) -> None:
"SELECT name FROM nodes WHERE status='online' ORDER BY name LIMIT 1" "SELECT name FROM nodes WHERE status='online' ORDER BY name LIMIT 1"
) )
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ha = metadata.get("ha", {}) if isinstance(metadata.get("ha"), dict) else {} ha_raw = metadata.get("ha")
ha: dict[str, Any] = dict(ha_raw) if isinstance(ha_raw, dict) else {}
status_raw = ha.get("status_current")
status: dict[str, Any] = dict(status_raw) if isinstance(status_raw, dict) else {}
armed = bool(ha.get("armed")) if "armed" in ha else False
return { return {
"quorate": 1, "quorate": status.get("quorate", metadata.get("quorate", 0)),
"mode": "active" if ha.get("armed", True) else "disabled", "mode": status.get("mode", "active" if armed else "disabled"),
"master_node": str(master or "pve01"), "master_node": status.get("master_node", str(master or "")),
"ha_started": int(row["started"] or 0), "ha_started": int(row["started"] or 0),
"ha_total": int(row["total"] or 0), "ha_total": int(row["total"] or 0),
"armed": 1 if ha.get("armed", True) else 0, "armed": 1 if armed else 0,
} }
async def ha_manager_status(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]: async def ha_manager_status(request: Request, _inputs: dict[str, Any]) -> dict[str, Any]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
ha = metadata.get("ha", {}) if isinstance(metadata.get("ha"), dict) else {} ha_raw = metadata.get("ha")
armed = bool(ha.get("armed", True)) ha: dict[str, Any] = dict(ha_raw) if isinstance(ha_raw, dict) else {}
manager_raw = ha.get("manager_status")
manager: dict[str, Any] = dict(manager_raw) if isinstance(manager_raw, dict) else {}
armed = bool(ha.get("armed")) if "armed" in ha else False
return { return {
"manager_status": "active" if armed else "disabled", "manager_status": manager.get("manager_status", "active" if armed else "disabled"),
"quorum": "OK", "quorum": manager.get("quorum", ""),
"armed": 1 if armed else 0, "armed": 1 if armed else 0,
} }
async def ha_rules(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def ha_rules(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
rules = metadata.get("ha_rules") rules = metadata.get("ha_rules")
if isinstance(rules, list) and rules: if isinstance(rules, list):
return [dict(item) for item in rules if isinstance(item, dict)] return [dict(item) for item in rules if isinstance(item, dict)]
defaults = [ return []
{"rule": "node-fencing", "type": "node", "action": "restart"},
{"rule": "service-ha", "type": "resource", "action": "failover"},
]
metadata["ha_rules"] = defaults
await save_cluster_metadata(request, metadata)
return list(defaults)
async def ha_relocate(request: Request, inputs: dict[str, Any]) -> None: async def ha_relocate(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
+8 -2
View File
@@ -323,7 +323,7 @@ def register_lxc_handlers(registry: HandlerRegistry) -> None:
async def migrate_preconditions(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def migrate_preconditions(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
values = _values(inputs) values = _values(inputs)
await _lxc_resource(request, str(values["node"]), str(values["vmid"])) resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
target = values.get("target") target = values.get("target")
if target in {None, ""}: if target in {None, ""}:
raise ApiError(400, "parameter 'target' is required") raise ApiError(400, "parameter 'target' is required")
@@ -333,7 +333,13 @@ def register_lxc_handlers(registry: HandlerRegistry) -> None:
) )
if not exists: if not exists:
raise ApiError(404, "target node does not exist") raise ApiError(404, "target node does not exist")
return {"local_disks": [], "local_resources": [], "running": False} state = _state(resource["state"])
pre = state.get("migrate_preconditions")
payload = dict(pre) if isinstance(pre, dict) else {}
payload["running"] = str(state.get("status") or "") == "running"
payload.setdefault("local_disks", [])
payload.setdefault("local_resources", [])
return payload
async def migrate(request: Request, inputs: dict[str, Any]) -> str: async def migrate(request: Request, inputs: dict[str, Any]) -> str:
values = _values(inputs) values = _values(inputs)
+6 -18
View File
@@ -39,12 +39,8 @@ def register_lxc_extra_handlers(registry: HandlerRegistry) -> None:
values = _values(inputs) values = _values(inputs)
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"])) resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
state = _state(resource["state"]) state = _state(resource["state"])
ifaces = state.setdefault( ifaces = state.get("interfaces")
"interfaces", return [dict(item) for item in ifaces] if isinstance(ifaces, list) else []
[{"name": "eth0", "hwaddr": "02:00:00:00:00:11", "inet": "192.0.2.20/24"}],
)
await _save_state(request, resource["id"], state)
return list(ifaces) if isinstance(ifaces, list) else []
async def move_volume(request: Request, inputs: dict[str, Any]) -> str: async def move_volume(request: Request, inputs: dict[str, Any]) -> str:
values = _values(inputs) values = _values(inputs)
@@ -70,23 +66,15 @@ def register_lxc_extra_handlers(registry: HandlerRegistry) -> None:
values = _values(inputs) values = _values(inputs)
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"])) resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
state = _state(resource["state"]) state = _state(resource["state"])
rrd_state = state.setdefault("rrd", {"filename": f"pve-ct-{values['vmid']}.rrd"}) rrd_state = state.get("rrd")
await _save_state(request, resource["id"], state) return dict(rrd_state) if isinstance(rrd_state, dict) else {}
return dict(rrd_state)
async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
values = _values(inputs) values = _values(inputs)
resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"])) resource = await _lxc_resource(request, str(values["node"]), str(values["vmid"]))
state = _state(resource["state"]) state = _state(resource["state"])
series = state.setdefault( series = state.get("rrddata")
"rrddata", return [dict(item) for item in series] if isinstance(series, list) else []
[
{"time": 1_700_000_000, "cpu": 0.02, "mem": 64 * 1024 * 1024},
{"time": 1_700_000_060, "cpu": 0.03, "mem": 66 * 1024 * 1024},
],
)
await _save_state(request, resource["id"], state)
return list(series)
async def _console(request: Request, inputs: dict[str, Any], kind: str) -> dict[str, Any]: async def _console(request: Request, inputs: dict[str, Any], kind: str) -> dict[str, Any]:
values = _values(inputs) values = _values(inputs)
+16 -9
View File
@@ -12,13 +12,14 @@ from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs
def _mappings(metadata: dict[str, Any]) -> dict[str, Any]: def _mappings(metadata: dict[str, Any]) -> dict[str, Any]:
current = metadata.setdefault("mapping", {"dir": {}, "pci": {}, "usb": {}}) current = metadata.get("mapping")
if not isinstance(current, dict): if not isinstance(current, dict):
current = {"dir": {}, "pci": {}, "usb": {}} return {"dir": {}, "pci": {}, "usb": {}}
metadata["mapping"] = current return {
for kind in ("dir", "pci", "usb"): "dir": dict(current["dir"]) if isinstance(current.get("dir"), dict) else {},
current.setdefault(kind, {}) "pci": dict(current["pci"]) if isinstance(current.get("pci"), dict) else {},
return current "usb": dict(current["usb"]) if isinstance(current.get("usb"), dict) else {},
}
def register_mapping_handlers(registry: HandlerRegistry) -> None: def register_mapping_handlers(registry: HandlerRegistry) -> None:
@@ -42,12 +43,14 @@ def register_mapping_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
item_id = str(payload["id"]) item_id = str(payload["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _mappings(metadata)[kind] mapping = _mappings(metadata)
store = mapping[kind]
if item_id in store: if item_id in store:
raise ApiError(400, f"{kind} mapping '{item_id}' already exists") raise ApiError(400, f"{kind} mapping '{item_id}' already exists")
store[item_id] = { store[item_id] = {
key: value for key, value in payload.items() if key not in {"delete", "digest"} key: value for key, value in payload.items() if key not in {"delete", "digest"}
} }
metadata["mapping"] = mapping
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -62,7 +65,8 @@ def register_mapping_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
item_id = str(payload["id"]) item_id = str(payload["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _mappings(metadata)[kind] mapping = _mappings(metadata)
store = mapping[kind]
if item_id not in store: if item_id not in store:
raise ApiError(404, f"{kind} mapping does not exist") raise ApiError(404, f"{kind} mapping does not exist")
current = dict(store[item_id]) current = dict(store[item_id])
@@ -76,15 +80,18 @@ def register_mapping_handlers(registry: HandlerRegistry) -> None:
current[key] = value current[key] = value
current["id"] = item_id current["id"] = item_id
store[item_id] = current store[item_id] = current
metadata["mapping"] = mapping
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def delete(request: Request, inputs: dict[str, Any]) -> None: async def delete(request: Request, inputs: dict[str, Any]) -> None:
item_id = str(values(inputs)["id"]) item_id = str(values(inputs)["id"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _mappings(metadata)[kind] mapping = _mappings(metadata)
store = mapping[kind]
if item_id not in store: if item_id not in store:
raise ApiError(404, f"{kind} mapping does not exist") raise ApiError(404, f"{kind} mapping does not exist")
del store[item_id] del store[item_id]
metadata["mapping"] = mapping
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
registry.register(base, "GET", list_items) registry.register(base, "GET", list_items)
+24 -109
View File
@@ -2,7 +2,6 @@
from __future__ import annotations from __future__ import annotations
import copy
from typing import Any, cast from typing import Any, cast
from fastapi import Request from fastapi import Request
@@ -20,92 +19,13 @@ from app.handlers.common import (
from app.tasks.repository import TaskRepository from app.tasks.repository import TaskRepository
from app.tasks.upid import Upid from app.tasks.upid import Upid
DEFAULT_NODE_OPS: dict[str, Any] = {
"network": [
{
"iface": "vmbr0",
"type": "bridge",
"active": 1,
"method": "static",
"address": "10.0.0.10/24",
},
{
"iface": "vmbr1",
"type": "bridge",
"active": 1,
"method": "static",
"address": "10.10.0.10/24",
},
{"iface": "eno1", "type": "eth", "active": 1, "method": "manual"},
],
"disks": {
"list": [
{
"devpath": "/dev/sda",
"size": 1_000_000_000_000,
"model": "SIM-DISK-01",
"serial": "SIM0001",
"gpt": 1,
},
{
"devpath": "/dev/sdb",
"size": 2_000_000_000_000,
"model": "SIM-SSD-01",
"serial": "SIM0002",
"gpt": 0,
},
],
"directory": [],
"lvm": [],
"lvmthin": [],
"zfs": [],
"smart": {},
},
"services": {
"pveproxy": {"state": "running", "enabled": 1},
"pvedaemon": {"state": "running", "enabled": 1},
"pvestatd": {"state": "running", "enabled": 1},
"corosync": {"state": "running", "enabled": 1},
},
"apt": {
"packages": [
{
"Package": "pve-manager",
"Version": "9.2.3",
"OldVersion": "9.2.2",
"Status": "upgradable",
},
{"Package": "libpve-common-perl", "Version": "9.0.3", "Status": "installed"},
],
"repositories": [
{
"path": "/etc/apt/sources.list.d/pve-enterprise.list",
"enabled": 1,
"types": "deb",
"uri": "http://download.proxmox.com/debian/pve",
"suites": "bookworm",
"components": "pve-no-subscription",
}
],
"update": {"status": "stopped", "exitstatus": "OK"},
"changelogs": {},
},
}
def default_node_ops() -> dict[str, Any]:
return copy.deepcopy(DEFAULT_NODE_OPS)
async def load_node_ops(request: Request, node: str) -> dict[str, Any]: async def load_node_ops(request: Request, node: str) -> dict[str, Any]:
metadata = await node_metadata(request, node) metadata = await node_metadata(request, node)
ops = metadata.get("ops") ops = metadata.get("ops")
if isinstance(ops, dict) and ops: if isinstance(ops, dict):
return ops
ops = default_node_ops()
metadata["ops"] = ops
await save_node_metadata(request, node, metadata)
return ops return ops
return {}
async def save_node_ops(request: Request, node: str, ops: dict[str, Any]) -> None: async def save_node_ops(request: Request, node: str, ops: dict[str, Any]) -> None:
@@ -135,19 +55,18 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
name = str(values(inputs).get("name") or "pve-manager") name = str(values(inputs).get("name") or "pve-manager")
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
changelogs = ops.setdefault("apt", {}).setdefault("changelogs", {}) apt = ops.get("apt")
if name not in changelogs: changelogs = apt.get("changelogs") if isinstance(apt, dict) else None
changelogs[name] = f"simulated changelog for {name}\n\n * emulator build\n" if not isinstance(changelogs, dict):
await save_node_ops(request, node, ops) return ""
return str(changelogs[name]) return str(changelogs.get(name) or "")
async def apt_update_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def apt_update_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
update = ops.get("apt", {}).get("update", {"status": "stopped", "exitstatus": "OK"}) apt = ops.get("apt")
if isinstance(update, dict): update = apt.get("update") if isinstance(apt, dict) else None
return dict(update) return dict(update) if isinstance(update, dict) else {}
return {"status": "stopped", "exitstatus": "OK"}
async def apt_update_start(request: Request, inputs: dict[str, Any]) -> str: async def apt_update_start(request: Request, inputs: dict[str, Any]) -> str:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
@@ -230,11 +149,9 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
async def _disks(request: Request, node: str) -> dict[str, Any]: async def _disks(request: Request, node: str) -> dict[str, Any]:
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
disks = ops.setdefault("disks", default_node_ops()["disks"]) disks = ops.get("disks")
if not isinstance(disks, dict): if not isinstance(disks, dict):
disks = default_node_ops()["disks"] return {}
ops["disks"] = disks
await save_node_ops(request, node, ops)
return cast(dict[str, Any], disks) return cast(dict[str, Any], disks)
async def disks_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def disks_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
@@ -247,17 +164,9 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
disk = str(values(inputs).get("disk") or "/dev/sda") disk = str(values(inputs).get("disk") or "/dev/sda")
disks = await _disks(request, node) disks = await _disks(request, node)
smart = disks.setdefault("smart", {}) smart = disks.get("smart")
if disk not in smart: if not isinstance(smart, dict) or disk not in smart:
smart[disk] = { return {}
"health": "PASSED",
"type": "scsi",
"model": "SIM-DISK",
"serial": disk.rsplit("/", 1)[-1],
}
ops = await load_node_ops(request, node)
ops["disks"] = disks
await save_node_ops(request, node, ops)
return dict(smart[disk]) return dict(smart[disk])
async def disks_collection( async def disks_collection(
@@ -286,7 +195,10 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
if not disk: if not disk:
raise ApiError(400, "parameter verification failed - 'disk' missing") raise ApiError(400, "parameter verification failed - 'disk' missing")
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
disks = ops.setdefault("disks", default_node_ops()["disks"]) disks = ops.get("disks")
if not isinstance(disks, dict):
disks = {}
ops["disks"] = disks
items = list(disks.get("list") or []) items = list(disks.get("list") or [])
found = False found = False
for item in items: for item in items:
@@ -314,7 +226,9 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
if not disk: if not disk:
raise ApiError(400, "parameter verification failed - 'disk' missing") raise ApiError(400, "parameter verification failed - 'disk' missing")
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
disks = ops.setdefault("disks", default_node_ops()["disks"]) disks = ops.get("disks")
if not isinstance(disks, dict):
raise ApiError(404, "disk does not exist")
items = list(disks.get("list") or []) items = list(disks.get("list") or [])
for item in items: for item in items:
if item.get("devpath") == disk: if item.get("devpath") == disk:
@@ -324,7 +238,8 @@ def register_node_ops_handlers(registry: HandlerRegistry) -> None:
else: else:
raise ApiError(404, "disk does not exist") raise ApiError(404, "disk does not exist")
disks["list"] = items disks["list"] = items
smart = disks.setdefault("smart", {}) smart = disks.get("smart")
if isinstance(smart, dict):
smart.pop(disk, None) smart.pop(disk, None)
ops["disks"] = disks ops["disks"] = disks
await save_node_ops(request, node, ops) await save_node_ops(request, node, ops)
+105 -199
View File
@@ -2,7 +2,6 @@
from __future__ import annotations from __future__ import annotations
import copy
import json import json
import secrets import secrets
import time import time
@@ -13,128 +12,49 @@ from fastapi import Request
from app.api.errors import ApiError from app.api.errors import ApiError
from app.api.registry import HandlerRegistry from app.api.registry import HandlerRegistry
from app.handlers.common import database, require_node, subdirs, values from app.handlers.common import database, require_node, subdirs, values
from app.handlers.nodes import default_node_ops, load_node_ops, save_node_ops from app.handlers.nodes import load_node_ops, save_node_ops
from app.tasks.repository import TaskRepository from app.tasks.repository import TaskRepository
from app.tasks.upid import Upid from app.tasks.upid import Upid
DEFAULT_HARDWARE: dict[str, Any] = {
"pci": [
{
"id": "0000:00:1f.2",
"vendor_name": "Intel Corporation",
"device_name": "SATA Controller",
"iommugroup": 0,
},
{
"id": "0000:01:00.0",
"vendor_name": "NVIDIA Corporation",
"device_name": "GP102 [GeForce GTX 1080 Ti]",
"iommugroup": 1,
"mdev": 1,
},
],
"usb": [
{"busnum": 1, "devnum": 1, "level": 0, "port": "1", "prodid": "0002", "vendid": "1d6b"},
{"busnum": 2, "devnum": 2, "level": 1, "port": "2", "prodid": "5591", "vendid": "0781"},
],
"mdev": {
"0000:01:00.0": [
{"type": "nvidia-11", "available": 4, "description": "GRID profile"},
]
},
}
DEFAULT_SCAN: dict[str, list[dict[str, Any]]] = {
"cifs": [{"server": "files.local", "share": "backups"}],
"iscsi": [{"portal": "10.0.0.50:3260", "target": "iqn.2024-01.local:storage"}],
"lvm": [{"vg": "pve", "size": 500_000_000_000, "free": 100_000_000_000}],
"lvmthin": [{"lv": "data", "vg": "pve", "lv_size": 400_000_000_000}],
"nfs": [{"server": "nfs.local", "path": "/export/pve", "options": "vers=4"}],
"pbs": [{"server": "pbs.local", "datastore": "store1"}],
"zfs": [{"pool": "rpool", "name": "rpool/data", "size": 800_000_000_000}],
}
DEFAULT_SUBSCRIPTION: dict[str, Any] = {
"status": "notfound",
"message": "There is no subscription key",
"serverid": "SIMULATOR",
"sockets": 1,
"productname": "Proxmox VE",
"url": "https://www.proxmox.com/en/proxmox-virtual-environment/pricing",
}
DEFAULT_CONFIG: dict[str, Any] = {
"description": "Simulator node",
"startall-onboot-delay": 0,
"wakeonlan": "",
}
DEFAULT_DNS: dict[str, Any] = {
"search": "local",
"dns1": "1.1.1.1",
"dns2": "8.8.8.8",
"dns3": "",
}
DEFAULT_TIME: dict[str, Any] = {
"timezone": "UTC",
"time": 0,
"localtime": 0,
}
def _certificates(ops: dict[str, Any]) -> dict[str, Any]: def _certificates(ops: dict[str, Any]) -> dict[str, Any]:
certs = ops.setdefault( certs = ops.get("certificates")
"certificates",
{
"custom": None,
"acme": {"account": "default", "domains": [], "certificate": None},
"info": [],
},
)
if not isinstance(certs, dict): if not isinstance(certs, dict):
certs = {"custom": None, "acme": {}, "info": []} return {"custom": None, "acme": {}, "info": []}
ops["certificates"] = certs
certs.setdefault("acme", {"account": "default", "domains": [], "certificate": None})
certs.setdefault("info", [])
return certs return certs
def _hardware(ops: dict[str, Any]) -> dict[str, Any]: def _hardware(ops: dict[str, Any]) -> dict[str, Any]:
hardware = ops.get("hardware") hardware = ops.get("hardware")
if not isinstance(hardware, dict) or not hardware: if not isinstance(hardware, dict):
hardware = copy.deepcopy(DEFAULT_HARDWARE) return {"pci": [], "usb": [], "mdev": {}}
ops["hardware"] = hardware return {
hardware.setdefault("pci", copy.deepcopy(DEFAULT_HARDWARE["pci"])) "pci": list(hardware.get("pci") or []) if isinstance(hardware.get("pci"), list) else [],
hardware.setdefault("usb", copy.deepcopy(DEFAULT_HARDWARE["usb"])) "usb": list(hardware.get("usb") or []) if isinstance(hardware.get("usb"), list) else [],
hardware.setdefault("mdev", copy.deepcopy(DEFAULT_HARDWARE["mdev"])) "mdev": dict(hardware.get("mdev") or {}) if isinstance(hardware.get("mdev"), dict) else {},
return hardware }
def _scan_cache(ops: dict[str, Any]) -> dict[str, Any]: def _scan_cache(ops: dict[str, Any]) -> dict[str, Any]:
scan = ops.get("scan") scan = ops.get("scan")
if not isinstance(scan, dict) or not scan: if not isinstance(scan, dict):
scan = copy.deepcopy(DEFAULT_SCAN) return {}
ops["scan"] = scan
for key, value in DEFAULT_SCAN.items():
scan.setdefault(key, copy.deepcopy(value))
return scan return scan
def _subscription(ops: dict[str, Any]) -> dict[str, Any]: def _subscription(ops: dict[str, Any]) -> dict[str, Any]:
subscription = ops.get("subscription") subscription = ops.get("subscription")
if not isinstance(subscription, dict) or not subscription: if not isinstance(subscription, dict):
subscription = copy.deepcopy(DEFAULT_SUBSCRIPTION) return {}
ops["subscription"] = subscription
return subscription return subscription
def _disk_items(ops: dict[str, Any], kind: str) -> list[dict[str, Any]]: def _disk_items(ops: dict[str, Any], kind: str) -> list[dict[str, Any]]:
disks = ops.setdefault("disks", default_node_ops()["disks"]) disks = ops.get("disks")
if not isinstance(disks, dict): if not isinstance(disks, dict):
disks = default_node_ops()["disks"] disks = {}
ops["disks"] = disks ops["disks"] = disks
items = disks.setdefault(kind, []) items = disks.get(kind)
if not isinstance(items, list): if not isinstance(items, list):
items = [] items = []
disks[kind] = items disks[kind] = items
@@ -208,7 +128,11 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
} }
entry["name"] = name entry["name"] = name
items.append(entry) items.append(entry)
ops.setdefault("disks", default_node_ops()["disks"])[kind] = items disks = ops.get("disks")
if not isinstance(disks, dict):
disks = {}
ops["disks"] = disks
disks[kind] = items
await save_node_ops(request, node, ops) await save_node_ops(request, node, ops)
return entry return entry
@@ -360,35 +284,40 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
return subdirs("cpu", "cpu-flags", "machines", "migration") return subdirs("cpu", "cpu-flags", "machines", "migration")
async def capabilities_cpu(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def capabilities_cpu(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) node = str(values(inputs)["node"])
return [ await require_node(request, node)
{"name": "host", "vendor": "QEMU", "custom": 0}, ops = await load_node_ops(request, node)
{"name": "x86-64-v2-AES", "vendor": "QEMU", "custom": 0}, caps = ops.get("capabilities")
{"name": "kvm64", "vendor": "QEMU", "custom": 0}, items = caps.get("cpu") if isinstance(caps, dict) else None
] return [dict(item) for item in items] if isinstance(items, list) else []
async def capabilities_cpu_flags( async def capabilities_cpu_flags(
request: Request, inputs: dict[str, Any] request: Request, inputs: dict[str, Any]
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) node = str(values(inputs)["node"])
return [ await require_node(request, node)
{"name": "aes", "introduces": "Westmere"}, ops = await load_node_ops(request, node)
{"name": "avx", "introduces": "SandyBridge"}, caps = ops.get("capabilities")
{"name": "avx2", "introduces": "Haswell"}, items = caps.get("cpu_flags") if isinstance(caps, dict) else None
] return [dict(item) for item in items] if isinstance(items, list) else []
async def capabilities_machines( async def capabilities_machines(
request: Request, inputs: dict[str, Any] request: Request, inputs: dict[str, Any]
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) node = str(values(inputs)["node"])
return [ await require_node(request, node)
{"id": "pc-i440fx-9.0", "type": "i440fx", "version": "9.0"}, ops = await load_node_ops(request, node)
{"id": "pc-q35-9.0", "type": "q35", "version": "9.0"}, caps = ops.get("capabilities")
] items = caps.get("machines") if isinstance(caps, dict) else None
return [dict(item) for item in items] if isinstance(items, list) else []
async def capabilities_migration(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def capabilities_migration(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
await require_node(request, str(values(inputs)["node"])) node = str(values(inputs)["node"])
return {"network": "", "type": "secure", "enabled": 1} await require_node(request, node)
ops = await load_node_ops(request, node)
caps = ops.get("capabilities")
migration = caps.get("migration") if isinstance(caps, dict) else None
return dict(migration) if isinstance(migration, dict) else {}
async def hardware_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]: async def hardware_index(request: Request, inputs: dict[str, Any]) -> list[dict[str, str]]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
@@ -399,8 +328,6 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
hardware = _hardware(ops) hardware = _hardware(ops)
ops["hardware"] = hardware
await save_node_ops(request, node, ops)
return [dict(item) for item in hardware.get("pci", [])] return [dict(item) for item in hardware.get("pci", [])]
async def hardware_pci_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def hardware_pci_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -430,8 +357,6 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
hardware = _hardware(ops) hardware = _hardware(ops)
ops["hardware"] = hardware
await save_node_ops(request, node, ops)
return [dict(item) for item in hardware.get("usb", [])] return [dict(item) for item in hardware.get("usb", [])]
async def subscription_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def subscription_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -452,7 +377,10 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
current = _subscription(ops) current = _subscription(ops)
method = request.method.upper() method = request.method.upper()
if method == "DELETE": if method == "DELETE":
ops["subscription"] = copy.deepcopy(DEFAULT_SUBSCRIPTION) ops["subscription"] = {
"status": "notfound",
"message": "There is no subscription key",
}
await save_node_ops(request, node, ops) await save_node_ops(request, node, ops)
return None return None
if method == "POST": if method == "POST":
@@ -481,17 +409,8 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
items = ops.get("aplinfo") items = ops.get("aplinfo")
if not isinstance(items, list): if not isinstance(items, list):
items = [ return []
{ return [dict(item) for item in items if isinstance(item, dict)]
"package": "alpine-3-standard",
"section": "system",
"type": "lxc",
"version": "3.20",
}
]
ops["aplinfo"] = items
await save_node_ops(request, node, ops)
return [dict(item) for item in items]
async def aplinfo_download(request: Request, inputs: dict[str, Any]) -> str: async def aplinfo_download(request: Request, inputs: dict[str, Any]) -> str:
payload = values(inputs) payload = values(inputs)
@@ -514,7 +433,10 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
node = str(payload["node"]) node = str(payload["node"])
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
apt = ops.setdefault("apt", copy.deepcopy(default_node_ops()["apt"])) apt = ops.get("apt")
if not isinstance(apt, dict):
apt = {}
ops["apt"] = apt
repositories = list(apt.get("repositories") or []) repositories = list(apt.get("repositories") or [])
method = request.method.upper() method = request.method.upper()
if method == "POST": if method == "POST":
@@ -561,18 +483,14 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
config = ops.get("config") config = ops.get("config")
if not isinstance(config, dict): return dict(config) if isinstance(config, dict) else {}
config = copy.deepcopy(DEFAULT_CONFIG)
ops["config"] = config
await save_node_ops(request, node, ops)
return dict(config)
async def node_config_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_config_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = values(inputs) payload = values(inputs)
node = str(payload["node"]) node = str(payload["node"])
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
config = dict(ops.get("config") or DEFAULT_CONFIG) config = dict(ops.get("config") or {})
config.update( config.update(
{ {
key: value key: value
@@ -589,18 +507,14 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
dns = ops.get("dns") dns = ops.get("dns")
if not isinstance(dns, dict): return dict(dns) if isinstance(dns, dict) else {}
dns = copy.deepcopy(DEFAULT_DNS)
ops["dns"] = dns
await save_node_ops(request, node, ops)
return dict(dns)
async def dns_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def dns_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = values(inputs) payload = values(inputs)
node = str(payload["node"]) node = str(payload["node"])
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
dns = dict(ops.get("dns") or DEFAULT_DNS) dns = dict(ops.get("dns") or {})
dns.update( dns.update(
{ {
key: value key: value
@@ -616,13 +530,10 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
current = dict(ops.get("time") or DEFAULT_TIME) current = dict(ops.get("time") or {})
now = int(time.time()) now = int(time.time())
current["time"] = now current["time"] = now
current["localtime"] = now current["localtime"] = now
current.setdefault("timezone", "UTC")
ops["time"] = current
await save_node_ops(request, node, ops)
return current return current
async def time_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def time_put(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -630,7 +541,7 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
node = str(payload["node"]) node = str(payload["node"])
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
current = dict(ops.get("time") or DEFAULT_TIME) current = dict(ops.get("time") or {})
if "timezone" in payload: if "timezone" in payload:
current["timezone"] = str(payload["timezone"]) current["timezone"] = str(payload["timezone"])
now = int(time.time()) now = int(time.time())
@@ -668,12 +579,7 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
hosts = ops.get("hosts") hosts = ops.get("hosts")
if not isinstance(hosts, dict): if not isinstance(hosts, dict):
hosts = { return {"data": "", "digest": ""}
"data": f"127.0.0.1 localhost\n10.0.0.10 {node}\n",
"digest": secrets.token_hex(8),
}
ops["hosts"] = hosts
await save_node_ops(request, node, ops)
return {"data": str(hosts.get("data", "")), "digest": str(hosts.get("digest", ""))} return {"data": str(hosts.get("data", "")), "digest": str(hosts.get("digest", ""))}
async def hosts_post(request: Request, inputs: dict[str, Any]) -> None: async def hosts_post(request: Request, inputs: dict[str, Any]) -> None:
@@ -692,46 +598,50 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
start = int(values(inputs).get("startcursor") or values(inputs).get("start") or 0) start = int(values(inputs).get("startcursor") or values(inputs).get("start") or 0)
limit = int(values(inputs).get("limit") or 50) limit = int(values(inputs).get("limit") or 50)
lines = [ ops = await load_node_ops(request, node)
f"{index}: {node} systemd[1]: Started simulated service {index}." lines = ops.get("journal")
for index in range(start, start + limit) if not isinstance(lines, list):
] return []
return lines sliced = lines[start : start + limit]
return [str(item) for item in sliced]
async def syslog(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def syslog(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
await require_node(request, node) await require_node(request, node)
limit = int(values(inputs).get("limit") or 50) limit = int(values(inputs).get("limit") or 50)
return [ ops = await load_node_ops(request, node)
{"n": index, "t": f"{node} kernel: simulated syslog line {index}"} lines = ops.get("syslog")
for index in range(limit) if not isinstance(lines, list):
] return []
return [dict(item) for item in lines[:limit] if isinstance(item, dict)]
async def netstat(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def netstat(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
await require_node(request, node) await require_node(request, node)
return [ ops = await load_node_ops(request, node)
{"in": 1_000_000, "out": 900_000, "vnet": "vmbr0", "hwaddr": "bc:24:11:00:00:01"}, items = ops.get("netstat")
{"in": 500_000, "out": 450_000, "vnet": "vmbr1", "hwaddr": "bc:24:11:00:00:02"}, return [dict(item) for item in items] if isinstance(items, list) else []
]
async def report(request: Request, inputs: dict[str, Any]) -> str: async def report(request: Request, inputs: dict[str, Any]) -> str:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
await require_node(request, node) await require_node(request, node)
return f"==== Proxmox node report for {node} ====\nuptime: simulated\n" ops = await load_node_ops(request, node)
report_text = ops.get("report")
return str(report_text) if report_text is not None else ""
async def rrd(_request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
await require_node(_request, str(values(inputs)["node"])) node = str(values(inputs)["node"])
return {"filename": "/var/lib/rrdcached/db/pve-node.rrd"} await require_node(request, node)
ops = await load_node_ops(request, node)
rrd_state = ops.get("rrd")
return dict(rrd_state) if isinstance(rrd_state, dict) else {}
async def rrddata(_request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
await require_node(_request, str(values(inputs)["node"])) node = str(values(inputs)["node"])
now = int(time.time()) await require_node(request, node)
return [ ops = await load_node_ops(request, node)
{"time": now - 120, "cpu": 0.05, "memused": 1_000_000_000}, series = ops.get("rrddata")
{"time": now - 60, "cpu": 0.07, "memused": 1_100_000_000}, return [dict(item) for item in series] if isinstance(series, list) else []
{"time": now, "cpu": 0.04, "memused": 1_050_000_000},
]
async def startall(request: Request, inputs: dict[str, Any]) -> str: async def startall(request: Request, inputs: dict[str, Any]) -> str:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
@@ -809,7 +719,7 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
if not isinstance(ops.get("network"), list): if not isinstance(ops.get("network"), list):
ops["network"] = copy.deepcopy(default_node_ops()["network"]) ops["network"] = []
ops["network_applied"] = False ops["network_applied"] = False
await save_node_ops(request, node, ops) await save_node_ops(request, node, ops)
@@ -818,28 +728,24 @@ def register_nodes_extra_handlers(registry: HandlerRegistry) -> None:
await require_node(request, node) await require_node(request, node)
repo = str(values(inputs).get("repo") or "library/alpine") repo = str(values(inputs).get("repo") or "library/alpine")
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
cache = ops.setdefault("oci_tags", {}) cache = ops.get("oci_tags")
if repo not in cache: if not isinstance(cache, dict):
cache[repo] = [{"tag": "latest"}, {"tag": "3.20"}] return []
ops["oci_tags"] = cache items = cache.get(repo)
await save_node_ops(request, node, ops) if not isinstance(items, list):
return [dict(item) for item in cache[repo]] return []
return [dict(item) for item in items if isinstance(item, dict)]
async def query_url_metadata(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def query_url_metadata(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
await require_node(request, node) await require_node(request, node)
url = str(values(inputs).get("url") or "") url = str(values(inputs).get("url") or "")
ops = await load_node_ops(request, node) ops = await load_node_ops(request, node)
cache = ops.setdefault("url_metadata", {}) cache = ops.get("url_metadata")
if url not in cache: if not isinstance(cache, dict):
cache[url] = { return {}
"filename": url.rsplit("/", 1)[-1] or "download.bin", payload = cache.get(url)
"mimetype": "application/octet-stream", return dict(payload) if isinstance(payload, dict) else {}
"size": 1024,
}
ops["url_metadata"] = cache
await save_node_ops(request, node, ops)
return dict(cache[url])
async def vncwebsocket(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def vncwebsocket(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
node = str(values(inputs)["node"]) node = str(values(inputs)["node"])
+57 -50
View File
@@ -13,53 +13,40 @@ from app.handlers.common import cluster_metadata, save_cluster_metadata, subdirs
_SECRET_KEYS = frozenset({"token", "password", "secret"}) _SECRET_KEYS = frozenset({"token", "password", "secret"})
DEFAULT_MATCHER_FIELDS = [
{"name": "type", "type": "string"},
{"name": "hostname", "type": "string"},
{"name": "job-id", "type": "string"},
{"name": "severity", "type": "string"},
]
DEFAULT_MATCHER_FIELD_VALUES = [
{"field": "type", "value": "fencing"},
{"field": "type", "value": "package-updates"},
{"field": "type", "value": "replication"},
{"field": "type", "value": "system-mail"},
]
def _public(endpoint: dict[str, Any]) -> dict[str, Any]: def _public(endpoint: dict[str, Any]) -> dict[str, Any]:
return {key: value for key, value in endpoint.items() if key not in _SECRET_KEYS} return {key: value for key, value in endpoint.items() if key not in _SECRET_KEYS}
def _notifications(metadata: dict[str, Any]) -> dict[str, Any]: def _notifications(metadata: dict[str, Any]) -> dict[str, Any]:
current = metadata.setdefault( current = metadata.get("notifications")
"notifications",
{
"endpoints": {
"gotify": {},
"sendmail": {},
"smtp": {},
"webhook": {},
},
"matchers": {},
"tests": [],
},
)
if not isinstance(current, dict): if not isinstance(current, dict):
current = { return {
"endpoints": {"gotify": {}, "sendmail": {}, "smtp": {}, "webhook": {}}, "endpoints": {"gotify": {}, "sendmail": {}, "smtp": {}, "webhook": {}},
"matchers": {}, "matchers": {},
"tests": [], "tests": [],
} }
metadata["notifications"] = current endpoints = current.get("endpoints")
current.setdefault( if not isinstance(endpoints, dict):
"endpoints", endpoints = {}
{"gotify": {}, "sendmail": {}, "smtp": {}, "webhook": {}}, result = {
key: value
for key, value in current.items()
if key not in {"endpoints", "matchers", "tests"}
}
result["endpoints"] = {
"gotify": dict(endpoints["gotify"]) if isinstance(endpoints.get("gotify"), dict) else {},
"sendmail": (
dict(endpoints["sendmail"]) if isinstance(endpoints.get("sendmail"), dict) else {}
),
"smtp": dict(endpoints["smtp"]) if isinstance(endpoints.get("smtp"), dict) else {},
"webhook": dict(endpoints["webhook"]) if isinstance(endpoints.get("webhook"), dict) else {},
}
result["matchers"] = (
dict(current["matchers"]) if isinstance(current.get("matchers"), dict) else {}
) )
current.setdefault("matchers", {}) result["tests"] = list(current["tests"]) if isinstance(current.get("tests"), list) else []
current.setdefault("tests", []) return result
return current
def register_notifications_handlers(registry: HandlerRegistry) -> None: def register_notifications_handlers(registry: HandlerRegistry) -> None:
@@ -80,26 +67,28 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
async def list_endpoints(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def list_endpoints(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["endpoints"].setdefault(kind, {}) store = _notifications(metadata)["endpoints"].get(kind) or {}
return [_public({"name": name, **item}) for name, item in sorted(store.items())] return [_public({"name": name, **item}) for name, item in sorted(store.items())]
async def create(request: Request, inputs: dict[str, Any]) -> None: async def create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
name = str(payload["name"]) name = str(payload["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["endpoints"].setdefault(kind, {}) notifications = _notifications(metadata)
store = notifications["endpoints"].setdefault(kind, {})
if name in store: if name in store:
raise ApiError(400, f"{kind} endpoint '{name}' already exists") raise ApiError(400, f"{kind} endpoint '{name}' already exists")
entry = {key: payload[key] for key in create_keys if key in payload} entry = {key: payload[key] for key in create_keys if key in payload}
entry["name"] = name entry["name"] = name
entry.setdefault("disable", 0) entry.setdefault("disable", 0)
store[name] = entry store[name] = entry
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
name = str(values(inputs)["name"]) name = str(values(inputs)["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["endpoints"].setdefault(kind, {}) store = _notifications(metadata)["endpoints"].get(kind) or {}
if name not in store: if name not in store:
raise ApiError(404, f"{kind} endpoint does not exist") raise ApiError(404, f"{kind} endpoint does not exist")
return _public({"name": name, **store[name]}) return _public({"name": name, **store[name]})
@@ -108,7 +97,8 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
name = str(payload["name"]) name = str(payload["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["endpoints"].setdefault(kind, {}) notifications = _notifications(metadata)
store = notifications["endpoints"].setdefault(kind, {})
if name not in store: if name not in store:
raise ApiError(404, f"{kind} endpoint does not exist") raise ApiError(404, f"{kind} endpoint does not exist")
current = dict(store[name]) current = dict(store[name])
@@ -123,15 +113,18 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
current[key] = value current[key] = value
current["name"] = name current["name"] = name
store[name] = current store[name] = current
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def delete(request: Request, inputs: dict[str, Any]) -> None: async def delete(request: Request, inputs: dict[str, Any]) -> None:
name = str(values(inputs)["name"]) name = str(values(inputs)["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["endpoints"].setdefault(kind, {}) notifications = _notifications(metadata)
store = notifications["endpoints"].setdefault(kind, {})
if name not in store: if name not in store:
raise ApiError(404, f"{kind} endpoint does not exist") raise ApiError(404, f"{kind} endpoint does not exist")
del store[name] del store[name]
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
registry.register(base, "GET", list_endpoints) registry.register(base, "GET", list_endpoints)
@@ -149,12 +142,14 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
name = str(payload["name"]) name = str(payload["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["matchers"] notifications = _notifications(metadata)
store = notifications["matchers"]
if name in store: if name in store:
raise ApiError(400, f"matcher '{name}' already exists") raise ApiError(400, f"matcher '{name}' already exists")
store[name] = { store[name] = {
key: value for key, value in payload.items() if key not in {"delete", "digest"} key: value for key, value in payload.items() if key not in {"delete", "digest"}
} }
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def matchers_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def matchers_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -169,7 +164,8 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
name = str(payload["name"]) name = str(payload["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["matchers"] notifications = _notifications(metadata)
store = notifications["matchers"]
if name not in store: if name not in store:
raise ApiError(404, "matcher does not exist") raise ApiError(404, "matcher does not exist")
current = dict(store[name]) current = dict(store[name])
@@ -183,24 +179,35 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
current[key] = value current[key] = value
current["name"] = name current["name"] = name
store[name] = current store[name] = current
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def matchers_delete(request: Request, inputs: dict[str, Any]) -> None: async def matchers_delete(request: Request, inputs: dict[str, Any]) -> None:
name = str(values(inputs)["name"]) name = str(values(inputs)["name"])
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
store = _notifications(metadata)["matchers"] notifications = _notifications(metadata)
store = notifications["matchers"]
if name not in store: if name not in store:
raise ApiError(404, "matcher does not exist") raise ApiError(404, "matcher does not exist")
del store[name] del store[name]
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def matcher_fields(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def matcher_fields(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
return list(DEFAULT_MATCHER_FIELDS) metadata = await cluster_metadata(request)
fields = _notifications(metadata).get("matcher_fields")
if isinstance(fields, list):
return [dict(item) for item in fields if isinstance(item, dict)]
return []
async def matcher_field_values( async def matcher_field_values(
_request: Request, _inputs: dict[str, Any] request: Request, _inputs: dict[str, Any]
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
return list(DEFAULT_MATCHER_FIELD_VALUES) metadata = await cluster_metadata(request)
values_list = _notifications(metadata).get("matcher_field_values")
if isinstance(values_list, list):
return [dict(item) for item in values_list if isinstance(item, dict)]
return []
async def targets(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def targets(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
@@ -231,10 +238,10 @@ def register_notifications_handlers(registry: HandlerRegistry) -> None:
break break
if not found: if not found:
raise ApiError(404, "notification target does not exist") raise ApiError(404, "notification target does not exist")
tests = notifications.setdefault("tests", []) tests = list(notifications.get("tests") or [])
if not isinstance(tests, list):
tests = notifications["tests"] = []
tests.append({"name": name, "tested_at": int(time.time()), "ok": True}) tests.append({"name": name, "tested_at": int(time.time()), "ok": True})
notifications["tests"] = tests
metadata["notifications"] = notifications
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
registry.register("/cluster/notifications", "GET", index) registry.register("/cluster/notifications", "GET", index)
+2 -2
View File
@@ -10,7 +10,7 @@ from fastapi import Request
from app.api.errors import ApiError from app.api.errors import ApiError
from app.api.registry import HandlerRegistry from app.api.registry import HandlerRegistry
from app.handlers.common import database, state, values from app.handlers.common import database, require_value, state, values
from app.simulation.seed import CLUSTER_ID, stable_id from app.simulation.seed import CLUSTER_ID, stable_id
@@ -117,7 +117,7 @@ def register_pool_handlers(registry: HandlerRegistry) -> None:
) )
async def pool_delete(request: Request, inputs: dict[str, Any]) -> None: async def pool_delete(request: Request, inputs: dict[str, Any]) -> None:
poolid = str(values(inputs)["poolid"]) poolid = str(require_value(values(inputs), "poolid"))
status = await database(request).pool.execute( status = await database(request).pool.execute(
"DELETE FROM pools WHERE pool_id=$1", "DELETE FROM pools WHERE pool_id=$1",
poolid, poolid,
+76 -32
View File
@@ -33,6 +33,51 @@ def _state(value: object) -> dict[str, Any]:
return dict(cast(Mapping[str, Any], value)) return dict(cast(Mapping[str, Any], value))
# Runtime-only keys stored in ``resources.state``; never echo on GET /config.
_QEMU_CONFIG_INTERNAL = frozenset(
{
"agent",
"cloudinit_dump",
"config",
"interfaces",
"migrate_preconditions",
"rrd",
"rrddata",
"sendkey",
"status",
}
)
def _agent_config_string(value: object) -> str:
"""Proxmox QEMU config ``agent`` is a string (e.g. ``1`` / ``0``), not an object."""
if isinstance(value, dict):
return "1" if value.get("enabled", True) else "0"
if isinstance(value, bool):
return "1" if value else "0"
if value in (None, ""):
return "0"
text = str(value).strip()
return text or "0"
def _public_qemu_config(config: dict[str, Any], state: dict[str, Any]) -> dict[str, Any]:
"""Wire-format QEMU config for clients (bpg / pulumi-proxmoxve)."""
merged = {**state, **config}
payload: dict[str, Any] = {}
for key, value in merged.items():
if key in _QEMU_CONFIG_INTERNAL:
continue
if isinstance(value, dict | list):
continue
payload[key] = value
agent = config.get("agent", state.get("agent", "0"))
payload["agent"] = _agent_config_string(agent)
return payload
def register_qemu_handlers(registry: HandlerRegistry) -> None: def register_qemu_handlers(registry: HandlerRegistry) -> None:
async def qemu_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def qemu_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
node = str(_values(inputs)["node"]) node = str(_values(inputs)["node"])
@@ -117,7 +162,10 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
) )
if row is None: if row is None:
raise ApiError(404, "virtual machine does not exist") raise ApiError(404, "virtual machine does not exist")
return {"vmid": int(vmid), **_state(row["config"]), **_state(row["state"])} return {
"vmid": int(vmid),
**_public_qemu_config(_state(row["config"]), _state(row["state"])),
}
async def mutate(operation: str, request: Request, inputs: dict[str, Any]) -> str: async def mutate(operation: str, request: Request, inputs: dict[str, Any]) -> str:
values = _values(inputs) values = _values(inputs)
@@ -406,7 +454,7 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
async def migrate_preconditions(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def migrate_preconditions(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
values = _values(inputs) values = _values(inputs)
await _qemu_resource(request, str(values["node"]), str(values["vmid"])) resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
target = values.get("target") target = values.get("target")
if target in {None, ""}: if target in {None, ""}:
raise ApiError(400, "parameter 'target' is required") raise ApiError(400, "parameter 'target' is required")
@@ -416,7 +464,13 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
) )
if not exists: if not exists:
raise ApiError(404, "target node does not exist") raise ApiError(404, "target node does not exist")
return {"local_disks": [], "local_resources": [], "running": False} state = _state(resource["state"])
pre = state.get("migrate_preconditions")
payload = dict(pre) if isinstance(pre, dict) else {}
payload["running"] = str(state.get("status") or "") == "running"
payload.setdefault("local_disks", [])
payload.setdefault("local_resources", [])
return payload
async def migrate(request: Request, inputs: dict[str, Any]) -> str: async def migrate(request: Request, inputs: dict[str, Any]) -> str:
values = _values(inputs) values = _values(inputs)
@@ -544,39 +598,16 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
command: str, request: Request, inputs: dict[str, Any] command: str, request: Request, inputs: dict[str, Any]
) -> dict[str, Any]: ) -> dict[str, Any]:
resource = await _agent_resource(request, _values(inputs)) resource = await _agent_resource(request, _values(inputs))
config = _state(resource["config"])
vmid = str(_values(inputs)["vmid"])
results: dict[str, Any] = {
"info": {
"version": "9.2.0-simulator",
"supported_commands": [
{"name": name, "enabled": True, "success-response": True}
for name in ("guest-ping", "guest-info", "guest-get-osinfo")
],
},
"get-osinfo": {
"name": str(config.get("ostype", "linux")),
"pretty-name": "Proxmox Simulator Guest",
"version": "1.0",
"machine": "x86_64",
},
"get-host-name": {"host-name": str(config.get("name", f"vm-{vmid}"))},
"network-get-interfaces": [
{
"name": "eth0",
"hardware-address": "02:00:00:00:00:01",
"ip-addresses": [
{"ip-address": "192.0.2.10", "ip-address-type": "ipv4", "prefix": 24}
],
}
],
"ping": {},
}
if command == "get-time": if command == "get-time":
seconds = int( seconds = int(
await _database(request).pool.fetchval("SELECT extract(epoch from now())::bigint") await _database(request).pool.fetchval("SELECT extract(epoch from now())::bigint")
) )
return {"result": {"seconds": seconds, "nanoseconds": 0}} return {"result": {"seconds": seconds, "nanoseconds": 0}}
state = _state(resource["state"])
agent = state.get("agent")
results = agent.get("results") if isinstance(agent, dict) else None
if not isinstance(results, dict) or command not in results:
raise ApiError(404, f"agent command '{command}' has no stored result")
return {"result": results[command]} return {"result": results[command]}
async def agent_info(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def agent_info(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -586,6 +617,12 @@ def register_qemu_handlers(registry: HandlerRegistry) -> None:
return await agent_result("get-osinfo", request, inputs) return await agent_result("get-osinfo", request, inputs)
async def agent_hostname(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def agent_hostname(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
resource = await _agent_resource(request, _values(inputs))
state = _state(resource["state"])
config = _state(resource["config"])
name = state.get("name") or config.get("name")
if name:
return {"result": {"host-name": str(name)}}
return await agent_result("get-host-name", request, inputs) return await agent_result("get-host-name", request, inputs)
async def agent_network(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def agent_network(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -825,7 +862,14 @@ async def _agent_resource(request: Request, values: dict[str, Any]) -> Any:
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"])) resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
config = _state(resource["config"]) config = _state(resource["config"])
state = _state(resource["state"]) state = _state(resource["state"])
if str(config.get("agent", "0")).split(",", 1)[0].lower() not in {"1", "true", "yes"}: agent = state.get("agent")
if not isinstance(agent, dict):
agent = config.get("agent")
if isinstance(agent, dict):
enabled = bool(agent.get("enabled", True))
else:
enabled = str(config.get("agent", "0")).split(",", 1)[0].lower() in {"1", "true", "yes"}
if not enabled:
raise ApiError(409, "QEMU guest agent is not enabled") raise ApiError(409, "QEMU guest agent is not enabled")
if state.get("status") != "running": if state.get("status") != "running":
raise ApiError(409, "QEMU guest agent is not running") raise ApiError(409, "QEMU guest agent is not running")
+23 -49
View File
@@ -79,20 +79,10 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
async def _agent_blob(command: str, request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def _agent_blob(command: str, request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
resource = await _agent_resource(request, _values(inputs)) resource = await _agent_resource(request, _values(inputs))
state = _state(resource["state"]) state = _state(resource["state"])
agent = state.setdefault("agent", {}) agent = state.get("agent")
blobs = agent.setdefault("results", {}) blobs = agent.get("results") if isinstance(agent, dict) else None
defaults: dict[str, Any] = { if not isinstance(blobs, dict) or command not in blobs:
"get-users": [{"user": "root", "login-time": 0}], raise ApiError(404, f"agent command '{command}' has no stored result")
"get-fsinfo": [{"name": "/", "type": "ext4", "total-bytes": 32 * 1024**3}],
"get-memory-block-info": {"size": 1024**3},
"get-memory-blocks": [{"start": 0, "size": 1024**3}],
"get-timezone": {"zone": "UTC", "offset": 0},
"get-vcpus": [{"online": True, "can-offline": False}],
"fsfreeze-status": "thawed",
}
if command not in blobs:
blobs[command] = defaults.get(command, {})
await _save_guest_state(request, resource["id"], state)
return {"result": blobs[command]} return {"result": blobs[command]}
async def agent_users(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def agent_users(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
@@ -146,10 +136,10 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
resource = await _agent_resource(request, payload) resource = await _agent_resource(request, payload)
path = str(payload.get("file") or payload.get("path") or "/etc/hostname") path = str(payload.get("file") or payload.get("path") or "/etc/hostname")
state = _state(resource["state"]) state = _state(resource["state"])
files = state.setdefault("agent", {}).setdefault("files", {}) agent = state.get("agent")
if path not in files: files = agent.get("files") if isinstance(agent, dict) else None
files[path] = f"simulated:{path}\n" if not isinstance(files, dict) or path not in files:
await _save_guest_state(request, resource["id"], state) raise ApiError(404, "file does not exist")
content = str(files[path]) content = str(files[path])
return {"result": {"content": content, "truncated": True}} return {"result": {"content": content, "truncated": True}}
@@ -188,9 +178,15 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
async def agent_fstrim(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def agent_fstrim(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
resource = await _agent_resource(request, _values(inputs)) resource = await _agent_resource(request, _values(inputs))
state = _state(resource["state"]) state = _state(resource["state"])
state.setdefault("agent", {})["last_fstrim"] = True agent = dict(state.get("agent") or {})
agent["last_fstrim"] = True
state["agent"] = agent
await _save_guest_state(request, resource["id"], state) await _save_guest_state(request, resource["id"], state)
return {"result": {"paths": [{"path": "/", "trimmed": 0}]}} results = agent.get("results") if isinstance(agent.get("results"), dict) else {}
fstrim = results.get("fstrim") if isinstance(results, dict) else None
if isinstance(fstrim, dict):
return {"result": fstrim}
return {"result": {}}
async def agent_set_password(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def agent_set_password(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = _values(inputs) payload = _values(inputs)
@@ -258,45 +254,23 @@ def register_qemu_extra_handlers(registry: HandlerRegistry) -> None:
async def cloudinit_dump(request: Request, inputs: dict[str, Any]) -> str: async def cloudinit_dump(request: Request, inputs: dict[str, Any]) -> str:
values = _values(inputs) values = _values(inputs)
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"])) resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
config = _state(resource["config"]) state = _state(resource["state"])
return ( dump = state.get("cloudinit_dump")
f"#cloud-config\nhostname: {config.get('name', values['vmid'])}\n" return str(dump) if dump is not None else ""
f"manage_etc_hosts: true\n"
)
async def rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
values = _values(inputs) values = _values(inputs)
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"])) resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
state = _state(resource["state"]) state = _state(resource["state"])
rrd_state = state.setdefault("rrd", {"filename": f"pve-vm-{values['vmid']}.rrd"}) rrd_state = state.get("rrd")
await _save_guest_state(request, resource["id"], state) return dict(rrd_state) if isinstance(rrd_state, dict) else {}
return dict(rrd_state)
async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
values = _values(inputs) values = _values(inputs)
resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"])) resource = await _qemu_resource(request, str(values["node"]), str(values["vmid"]))
state = _state(resource["state"]) state = _state(resource["state"])
series = state.setdefault( series = state.get("rrddata")
"rrddata", return [dict(item) for item in series] if isinstance(series, list) else []
[
{
"time": 1_700_000_000,
"cpu": 0.05,
"mem": 256 * 1024 * 1024,
"netin": 0,
"netout": 0,
},
{
"time": 1_700_000_060,
"cpu": 0.08,
"mem": 260 * 1024 * 1024,
"netin": 100,
"netout": 80,
},
],
)
await _save_guest_state(request, resource["id"], state)
return list(series)
async def monitor(request: Request, inputs: dict[str, Any]) -> str: async def monitor(request: Request, inputs: dict[str, Any]) -> str:
values = _values(inputs) values = _values(inputs)
+124 -120
View File
@@ -21,52 +21,8 @@ _SECRET_KEYS = frozenset({"key", "token", "fingerprint"})
def _sdn(metadata: dict[str, Any]) -> dict[str, Any]: def _sdn(metadata: dict[str, Any]) -> dict[str, Any]:
current = metadata.setdefault( current = metadata.get("sdn")
"sdn", return current if isinstance(current, dict) else {}
{
"zones": {},
"vnets": {},
"controllers": {},
"dns": {},
"ipams": {},
"fabrics": {},
"fabric_nodes": {},
"prefix_lists": {},
"route_maps": {},
"lock": None,
"pending": False,
"running_version": 1,
},
)
if not isinstance(current, dict):
current = {
"zones": {},
"vnets": {},
"controllers": {},
"dns": {},
"ipams": {},
"fabrics": {},
"fabric_nodes": {},
"prefix_lists": {},
"route_maps": {},
"lock": None,
"pending": False,
"running_version": 1,
}
metadata["sdn"] = current
for key in (
"zones",
"vnets",
"controllers",
"dns",
"ipams",
"fabrics",
"fabric_nodes",
"prefix_lists",
"route_maps",
):
current.setdefault(key, {})
return current
def _public(item: dict[str, Any]) -> dict[str, Any]: def _public(item: dict[str, Any]) -> dict[str, Any]:
@@ -77,9 +33,14 @@ def _store_list(store: dict[str, Any], *, id_key: str) -> list[dict[str, Any]]:
return [_public({id_key: name, **item}) for name, item in sorted(store.items())] return [_public({id_key: name, **item}) for name, item in sorted(store.items())]
async def _load(request: Request) -> tuple[dict[str, Any], dict[str, Any]]: async def _load(
request: Request, *, for_write: bool = False
) -> tuple[dict[str, Any], dict[str, Any]]:
metadata = await cluster_metadata(request) metadata = await cluster_metadata(request)
return metadata, _sdn(metadata) sdn = _sdn(metadata)
if for_write and metadata.get("sdn") is not sdn:
metadata["sdn"] = sdn
return metadata, sdn
def register_sdn_handlers(registry: HandlerRegistry) -> None: def register_sdn_handlers(registry: HandlerRegistry) -> None:
@@ -100,7 +61,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def apply(request: Request, inputs: dict[str, Any]) -> None: async def apply(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
lock = sdn.get("lock") lock = sdn.get("lock")
token = payload.get("lock-token") token = payload.get("lock-token")
if lock and token and lock.get("token") != token: if lock and token and lock.get("token") != token:
@@ -112,7 +73,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def lock_create(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def lock_create(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
if sdn.get("lock") and not values(inputs).get("allow-pending"): if sdn.get("lock") and not values(inputs).get("allow-pending"):
raise ApiError(400, "SDN is already locked") raise ApiError(400, "SDN is already locked")
token = secrets.token_hex(8) token = secrets.token_hex(8)
@@ -122,7 +83,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def lock_delete(request: Request, inputs: dict[str, Any]) -> None: async def lock_delete(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
lock = sdn.get("lock") lock = sdn.get("lock")
if lock is None: if lock is None:
return None return None
@@ -133,14 +94,14 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def rollback(request: Request, inputs: dict[str, Any]) -> None: async def rollback(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
sdn["pending"] = False sdn["pending"] = False
if payload.get("release-lock"): if payload.get("release-lock"):
sdn["lock"] = None sdn["lock"] = None
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def dry_run(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def dry_run(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
return [ return [
{"type": "zone", "name": name, "action": "noop"} {"type": "zone", "name": name, "action": "noop"}
for name in sorted(sdn.get("zones") or {}) for name in sorted(sdn.get("zones") or {})
@@ -154,7 +115,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
create_required: str | None = None, create_required: str | None = None,
) -> None: ) -> None:
async def list_items(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def list_items(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
items = _store_list(sdn.get(store_key) or {}, id_key=id_param) items = _store_list(sdn.get(store_key) or {}, id_key=id_param)
type_filter = values(inputs).get("type") type_filter = values(inputs).get("type")
if type_filter: if type_filter:
@@ -164,7 +125,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def create(request: Request, inputs: dict[str, Any]) -> None: async def create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
item_id = str(payload[create_required or id_param]) item_id = str(payload[create_required or id_param])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault(store_key, {}) store = sdn.setdefault(store_key, {})
if item_id in store: if item_id in store:
raise ApiError(400, f"{store_key} '{item_id}' already exists") raise ApiError(400, f"{store_key} '{item_id}' already exists")
@@ -179,7 +140,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
item_id = str(values(inputs)[id_param]) item_id = str(values(inputs)[id_param])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get(store_key) or {}).get(item_id) item = (sdn.get(store_key) or {}).get(item_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, f"{store_key} entry does not exist") raise ApiError(404, f"{store_key} entry does not exist")
@@ -188,7 +149,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def update(request: Request, inputs: dict[str, Any]) -> None: async def update(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
item_id = str(payload[id_param]) item_id = str(payload[id_param])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault(store_key, {}) store = sdn.setdefault(store_key, {})
if item_id not in store: if item_id not in store:
raise ApiError(404, f"{store_key} entry does not exist") raise ApiError(404, f"{store_key} entry does not exist")
@@ -208,7 +169,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def delete(request: Request, inputs: dict[str, Any]) -> None: async def delete(request: Request, inputs: dict[str, Any]) -> None:
item_id = str(values(inputs)[id_param]) item_id = str(values(inputs)[id_param])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault(store_key, {}) store = sdn.setdefault(store_key, {})
if item_id not in store: if item_id not in store:
raise ApiError(404, f"{store_key} entry does not exist") raise ApiError(404, f"{store_key} entry does not exist")
@@ -230,20 +191,20 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def ipam_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def ipam_status(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
ipam = str(values(inputs)["ipam"]) ipam = str(values(inputs)["ipam"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
if ipam not in (sdn.get("ipams") or {}): if ipam not in (sdn.get("ipams") or {}):
raise ApiError(404, "ipam does not exist") raise ApiError(404, "ipam does not exist")
return {"status": "ok", "ipam": ipam} return {"status": "ok", "ipam": ipam}
# vnets + nested # vnets + nested
async def vnets_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def vnets_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
return _store_list(sdn.get("vnets") or {}, id_key="vnet") return _store_list(sdn.get("vnets") or {}, id_key="vnet")
async def vnets_create(request: Request, inputs: dict[str, Any]) -> None: async def vnets_create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("vnets", {}) store = sdn.setdefault("vnets", {})
if vnet in store: if vnet in store:
raise ApiError(400, f"vnet '{vnet}' already exists") raise ApiError(400, f"vnet '{vnet}' already exists")
@@ -259,7 +220,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def vnet_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def vnet_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
vnet = str(values(inputs)["vnet"]) vnet = str(values(inputs)["vnet"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("vnets") or {}).get(vnet) item = (sdn.get("vnets") or {}).get(vnet)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "vnet does not exist") raise ApiError(404, "vnet does not exist")
@@ -268,7 +229,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def vnet_update(request: Request, inputs: dict[str, Any]) -> None: async def vnet_update(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("vnets", {}) store = sdn.setdefault("vnets", {})
if vnet not in store: if vnet not in store:
raise ApiError(404, "vnet does not exist") raise ApiError(404, "vnet does not exist")
@@ -288,7 +249,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def vnet_delete(request: Request, inputs: dict[str, Any]) -> None: async def vnet_delete(request: Request, inputs: dict[str, Any]) -> None:
vnet = str(values(inputs)["vnet"]) vnet = str(values(inputs)["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("vnets", {}) store = sdn.setdefault("vnets", {})
if vnet not in store: if vnet not in store:
raise ApiError(404, "vnet does not exist") raise ApiError(404, "vnet does not exist")
@@ -307,7 +268,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def subnets_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def subnets_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
vnet = str(values(inputs)["vnet"]) vnet = str(values(inputs)["vnet"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
return _store_list(item.get("subnets") or {}, id_key="subnet") return _store_list(item.get("subnets") or {}, id_key="subnet")
@@ -315,7 +276,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
subnet = str(payload["subnet"]) subnet = str(payload["subnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
subnets = item.setdefault("subnets", {}) subnets = item.setdefault("subnets", {})
if subnet in subnets: if subnet in subnets:
@@ -331,7 +292,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
subnet = str(payload["subnet"]) subnet = str(payload["subnet"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
data = (item.get("subnets") or {}).get(subnet) data = (item.get("subnets") or {}).get(subnet)
if not isinstance(data, dict): if not isinstance(data, dict):
@@ -342,7 +303,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
subnet = str(payload["subnet"]) subnet = str(payload["subnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
subnets = item.setdefault("subnets", {}) subnets = item.setdefault("subnets", {})
if subnet not in subnets: if subnet not in subnets:
@@ -365,7 +326,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
subnet = str(payload["subnet"]) subnet = str(payload["subnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
subnets = item.setdefault("subnets", {}) subnets = item.setdefault("subnets", {})
if subnet not in subnets: if subnet not in subnets:
@@ -377,7 +338,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def ips_create(request: Request, inputs: dict[str, Any]) -> None: async def ips_create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
ips = item.setdefault("ips", []) ips = item.setdefault("ips", [])
if not isinstance(ips, list): if not isinstance(ips, list):
@@ -398,7 +359,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def ips_delete(request: Request, inputs: dict[str, Any]) -> None: async def ips_delete(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
ips = item.setdefault("ips", []) ips = item.setdefault("ips", [])
if not isinstance(ips, list): if not isinstance(ips, list):
@@ -416,14 +377,14 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fw_options_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def fw_options_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
vnet = str(values(inputs)["vnet"]) vnet = str(values(inputs)["vnet"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
return dict(item.get("firewall", {}).get("options") or {"enable": 0}) return dict(item.get("firewall", {}).get("options") or {"enable": 0})
async def fw_options_put(request: Request, inputs: dict[str, Any]) -> None: async def fw_options_put(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
options = dict(item.setdefault("firewall", {}).setdefault("options", {"enable": 0})) options = dict(item.setdefault("firewall", {}).setdefault("options", {"enable": 0}))
for key, value in payload.items(): for key, value in payload.items():
@@ -435,7 +396,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fw_rules_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def fw_rules_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
vnet = str(values(inputs)["vnet"]) vnet = str(values(inputs)["vnet"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
rules = item.get("firewall", {}).get("rules") or [] rules = item.get("firewall", {}).get("rules") or []
return list(rules) if isinstance(rules, list) else [] return list(rules) if isinstance(rules, list) else []
@@ -443,7 +404,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fw_rules_create(request: Request, inputs: dict[str, Any]) -> None: async def fw_rules_create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
rules = item.setdefault("firewall", {}).setdefault("rules", []) rules = item.setdefault("firewall", {}).setdefault("rules", [])
if not isinstance(rules, list): if not isinstance(rules, list):
@@ -464,7 +425,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
pos = int(payload["pos"]) pos = int(payload["pos"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
rules = item.setdefault("firewall", {}).setdefault("rules", []) rules = item.setdefault("firewall", {}).setdefault("rules", [])
if not isinstance(rules, list) or pos < 0 or pos >= len(rules): if not isinstance(rules, list) or pos < 0 or pos >= len(rules):
@@ -479,7 +440,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
vnet = str(payload["vnet"]) vnet = str(payload["vnet"])
pos = int(payload["pos"]) pos = int(payload["pos"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = await _vnet(sdn, vnet) item = await _vnet(sdn, vnet)
rules = item.setdefault("firewall", {}).setdefault("rules", []) rules = item.setdefault("firewall", {}).setdefault("rules", [])
if not isinstance(rules, list) or pos < 0 or pos >= len(rules): if not isinstance(rules, list) or pos < 0 or pos >= len(rules):
@@ -492,7 +453,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
return subdirs("all", "fabric", "node") return subdirs("all", "fabric", "node")
async def fabrics_all(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def fabrics_all(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
return _store_list(sdn.get("fabrics") or {}, id_key="id") return _store_list(sdn.get("fabrics") or {}, id_key="id")
async def fabric_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def fabric_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
@@ -501,7 +462,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fabric_create(request: Request, inputs: dict[str, Any]) -> None: async def fabric_create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
fabric_id = str(payload["id"]) fabric_id = str(payload["id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("fabrics", {}) store = sdn.setdefault("fabrics", {})
if fabric_id in store: if fabric_id in store:
raise ApiError(400, f"fabric '{fabric_id}' already exists") raise ApiError(400, f"fabric '{fabric_id}' already exists")
@@ -514,7 +475,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fabric_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def fabric_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
fabric_id = str(values(inputs)["id"]) fabric_id = str(values(inputs)["id"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("fabrics") or {}).get(fabric_id) item = (sdn.get("fabrics") or {}).get(fabric_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "fabric does not exist") raise ApiError(404, "fabric does not exist")
@@ -523,7 +484,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fabric_update(request: Request, inputs: dict[str, Any]) -> None: async def fabric_update(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
fabric_id = str(payload["id"]) fabric_id = str(payload["id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("fabrics", {}) store = sdn.setdefault("fabrics", {})
if fabric_id not in store: if fabric_id not in store:
raise ApiError(404, "fabric does not exist") raise ApiError(404, "fabric does not exist")
@@ -543,7 +504,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def fabric_delete(request: Request, inputs: dict[str, Any]) -> None: async def fabric_delete(request: Request, inputs: dict[str, Any]) -> None:
fabric_id = str(values(inputs)["id"]) fabric_id = str(values(inputs)["id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("fabrics", {}) store = sdn.setdefault("fabrics", {})
if fabric_id not in store: if fabric_id not in store:
raise ApiError(404, "fabric does not exist") raise ApiError(404, "fabric does not exist")
@@ -554,7 +515,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
await save_cluster_metadata(request, metadata) await save_cluster_metadata(request, metadata)
async def fabric_nodes_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def fabric_nodes_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
fabric_id = values(inputs).get("fabric_id") fabric_id = values(inputs).get("fabric_id")
nodes = sdn.get("fabric_nodes") or {} nodes = sdn.get("fabric_nodes") or {}
result: list[dict[str, Any]] = [] result: list[dict[str, Any]] = []
@@ -571,7 +532,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
fabric_id = str(payload["fabric_id"]) fabric_id = str(payload["fabric_id"])
node_id = str(payload["node_id"]) node_id = str(payload["node_id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
if fabric_id not in (sdn.get("fabrics") or {}): if fabric_id not in (sdn.get("fabrics") or {}):
raise ApiError(404, "fabric does not exist") raise ApiError(404, "fabric does not exist")
store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {}) store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {})
@@ -588,7 +549,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
fabric_id = str(payload["fabric_id"]) fabric_id = str(payload["fabric_id"])
node_id = str(payload["node_id"]) node_id = str(payload["node_id"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = ((sdn.get("fabric_nodes") or {}).get(fabric_id) or {}).get(node_id) item = ((sdn.get("fabric_nodes") or {}).get(fabric_id) or {}).get(node_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "fabric node does not exist") raise ApiError(404, "fabric node does not exist")
@@ -598,7 +559,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
fabric_id = str(payload["fabric_id"]) fabric_id = str(payload["fabric_id"])
node_id = str(payload["node_id"]) node_id = str(payload["node_id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {}) store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {})
if node_id not in store: if node_id not in store:
raise ApiError(404, "fabric node does not exist") raise ApiError(404, "fabric node does not exist")
@@ -620,7 +581,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
fabric_id = str(payload["fabric_id"]) fabric_id = str(payload["fabric_id"])
node_id = str(payload["node_id"]) node_id = str(payload["node_id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {}) store = sdn.setdefault("fabric_nodes", {}).setdefault(fabric_id, {})
if node_id not in store: if node_id not in store:
raise ApiError(404, "fabric node does not exist") raise ApiError(404, "fabric node does not exist")
@@ -630,13 +591,13 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
# prefix lists # prefix lists
async def prefix_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]: async def prefix_list(request: Request, _inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
return _store_list(sdn.get("prefix_lists") or {}, id_key="id") return _store_list(sdn.get("prefix_lists") or {}, id_key="id")
async def prefix_create(request: Request, inputs: dict[str, Any]) -> None: async def prefix_create(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
list_id = str(payload["id"]) list_id = str(payload["id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("prefix_lists", {}) store = sdn.setdefault("prefix_lists", {})
if list_id in store: if list_id in store:
raise ApiError(400, f"prefix-list '{list_id}' already exists") raise ApiError(400, f"prefix-list '{list_id}' already exists")
@@ -654,7 +615,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def prefix_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def prefix_get(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
list_id = str(values(inputs)["id"]) list_id = str(values(inputs)["id"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("prefix_lists") or {}).get(list_id) item = (sdn.get("prefix_lists") or {}).get(list_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -663,7 +624,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def prefix_update(request: Request, inputs: dict[str, Any]) -> None: async def prefix_update(request: Request, inputs: dict[str, Any]) -> None:
payload = values(inputs) payload = values(inputs)
list_id = str(payload["id"]) list_id = str(payload["id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("prefix_lists", {}) store = sdn.setdefault("prefix_lists", {})
if list_id not in store: if list_id not in store:
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -676,7 +637,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def prefix_delete(request: Request, inputs: dict[str, Any]) -> None: async def prefix_delete(request: Request, inputs: dict[str, Any]) -> None:
list_id = str(values(inputs)["id"]) list_id = str(values(inputs)["id"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
store = sdn.setdefault("prefix_lists", {}) store = sdn.setdefault("prefix_lists", {})
if list_id not in store: if list_id not in store:
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -686,7 +647,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def prefix_entries(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def prefix_entries(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
list_id = str(values(inputs)["id"]) list_id = str(values(inputs)["id"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("prefix_lists") or {}).get(list_id) item = (sdn.get("prefix_lists") or {}).get(list_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -699,7 +660,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
list_id = str(payload["id"]) list_id = str(payload["id"])
seq = str(payload.get("seq") or secrets.randbelow(10000)) seq = str(payload.get("seq") or secrets.randbelow(10000))
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("prefix_lists") or {}).get(list_id) item = (sdn.get("prefix_lists") or {}).get(list_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -720,7 +681,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
list_id = str(payload["id"]) list_id = str(payload["id"])
seq = str(payload["url_seq"]) seq = str(payload["url_seq"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("prefix_lists") or {}).get(list_id) item = (sdn.get("prefix_lists") or {}).get(list_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -733,7 +694,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
list_id = str(payload["id"]) list_id = str(payload["id"])
seq = str(payload["url_seq"]) seq = str(payload["url_seq"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("prefix_lists") or {}).get(list_id) item = (sdn.get("prefix_lists") or {}).get(list_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -752,7 +713,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
list_id = str(payload["id"]) list_id = str(payload["id"])
seq = str(payload["url_seq"]) seq = str(payload["url_seq"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("prefix_lists") or {}).get(list_id) item = (sdn.get("prefix_lists") or {}).get(list_id)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "prefix-list does not exist") raise ApiError(404, "prefix-list does not exist")
@@ -767,16 +728,44 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def route_maps_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]: async def route_maps_index(_request: Request, _inputs: dict[str, Any]) -> list[dict[str, str]]:
return subdirs("entries") return subdirs("entries")
def _route_map_entries(item: object) -> dict[str, Any]:
if not isinstance(item, dict):
return {}
nested = item.get("entries")
if isinstance(nested, dict):
return {
str(key): value
for key, value in nested.items()
if isinstance(value, dict) and str(key).isdigit()
}
return {
str(key): value
for key, value in item.items()
if isinstance(value, dict) and str(key).isdigit()
}
def _route_map_entries_mutable(sdn: dict[str, Any], map_id: str) -> dict[str, Any]:
bucket = sdn.setdefault("route_maps", {}).setdefault(map_id, {})
if not isinstance(bucket, dict):
bucket = {}
sdn["route_maps"][map_id] = bucket
if "entries" in bucket or "name" in bucket:
entries = bucket.setdefault("entries", {})
if not isinstance(entries, dict):
entries = {}
bucket["entries"] = entries
return entries
return bucket
async def route_entries_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def route_entries_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
maps = sdn.get("route_maps") or {} maps = sdn.get("route_maps") or {}
route_map_id = values(inputs).get("route-map-id") route_map_id = values(inputs).get("route-map-id")
result: list[dict[str, Any]] = [] result: list[dict[str, Any]] = []
for map_id, entries in sorted(maps.items()): for map_id, raw_entries in sorted(maps.items()):
if route_map_id and map_id != route_map_id: if route_map_id and map_id != route_map_id:
continue continue
if not isinstance(entries, dict): entries = _route_map_entries(raw_entries)
continue
for order, entry in sorted(entries.items(), key=lambda pair: int(pair[0])): for order, entry in sorted(entries.items(), key=lambda pair: int(pair[0])):
result.append({"route-map-id": map_id, "order": int(order), **entry}) result.append({"route-map-id": map_id, "order": int(order), **entry})
return result return result
@@ -785,8 +774,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
map_id = str(payload["route-map-id"]) map_id = str(payload["route-map-id"])
order = str(payload.get("order") or 10) order = str(payload.get("order") or 10)
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
entries = sdn.setdefault("route_maps", {}).setdefault(map_id, {}) entries = _route_map_entries_mutable(sdn, map_id)
if order in entries: if order in entries:
raise ApiError(400, f"route-map entry '{order}' already exists") raise ApiError(400, f"route-map entry '{order}' already exists")
entries[order] = { entries[order] = {
@@ -809,8 +798,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
map_id = str(payload["route-map-id"]) map_id = str(payload["route-map-id"])
order = str(payload["order"]) order = str(payload["order"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
entry = ((sdn.get("route_maps") or {}).get(map_id) or {}).get(order) entry = _route_map_entries((sdn.get("route_maps") or {}).get(map_id)).get(order)
if not isinstance(entry, dict): if not isinstance(entry, dict):
raise ApiError(404, "route-map entry does not exist") raise ApiError(404, "route-map entry does not exist")
return {"route-map-id": map_id, "order": int(order), **entry} return {"route-map-id": map_id, "order": int(order), **entry}
@@ -819,8 +808,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
map_id = str(payload["route-map-id"]) map_id = str(payload["route-map-id"])
order = str(payload["order"]) order = str(payload["order"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
entries = sdn.setdefault("route_maps", {}).setdefault(map_id, {}) entries = _route_map_entries_mutable(sdn, map_id)
if order not in entries: if order not in entries:
raise ApiError(404, "route-map entry does not exist") raise ApiError(404, "route-map entry does not exist")
current = dict(entries[order]) current = dict(entries[order])
@@ -837,8 +826,8 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
map_id = str(payload["route-map-id"]) map_id = str(payload["route-map-id"])
order = str(payload["order"]) order = str(payload["order"])
metadata, sdn = await _load(request) metadata, sdn = await _load(request, for_write=True)
entries = sdn.setdefault("route_maps", {}).setdefault(map_id, {}) entries = _route_map_entries_mutable(sdn, map_id)
if order not in entries: if order not in entries:
raise ApiError(404, "route-map entry does not exist") raise ApiError(404, "route-map entry does not exist")
del entries[order] del entries[order]
@@ -852,13 +841,13 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def node_zones(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def node_zones(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
return _store_list(sdn.get("zones") or {}, id_key="zone") return _store_list(sdn.get("zones") or {}, id_key="zone")
async def node_zone(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_zone(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
zone = str(values(inputs)["zone"]) zone = str(values(inputs)["zone"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
item = (sdn.get("zones") or {}).get(zone) item = (sdn.get("zones") or {}).get(zone)
if not isinstance(item, dict): if not isinstance(item, dict):
raise ApiError(404, "zone does not exist") raise ApiError(404, "zone does not exist")
@@ -866,13 +855,16 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def node_zone_bridges(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def node_zone_bridges(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
zone = await node_zone(request, inputs) zone = await node_zone(request, inputs)
bridge = zone.get("bridge") or f"vmbr-{zone.get('zone')}" bridges = zone.get("bridges")
return [{"iface": bridge, "active": 1}] if isinstance(bridges, list):
return [dict(item) for item in bridges if isinstance(item, dict)]
bridge = zone.get("bridge")
return [{"iface": bridge, "active": 1}] if bridge else []
async def node_zone_content(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def node_zone_content(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
zone = str(values(inputs)["zone"]) zone = str(values(inputs)["zone"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
return [ return [
{"vnet": name, **item} {"vnet": name, **item}
for name, item in sorted((sdn.get("vnets") or {}).items()) for name, item in sorted((sdn.get("vnets") or {}).items())
@@ -881,7 +873,11 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def node_zone_ip_vrf(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_zone_ip_vrf(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
zone = await node_zone(request, inputs) zone = await node_zone(request, inputs)
return {"zone": zone.get("zone"), "vrf": f"vrf-{zone.get('zone')}", "table": 100} return {
"zone": zone.get("zone"),
"vrf": zone.get("vrf") or "",
"table": zone.get("table") if zone.get("table") is not None else 0,
}
async def node_vnet(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_vnet(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
@@ -889,7 +885,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
async def node_vnet_mac_vrf(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_vnet_mac_vrf(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
vnet = await node_vnet(request, inputs) vnet = await node_vnet(request, inputs)
return {"vnet": vnet.get("vnet"), "mac-vrf": f"macvrf-{vnet.get('vnet')}"} return {"vnet": vnet.get("vnet"), "mac-vrf": vnet.get("mac-vrf") or ""}
async def node_fabric(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def node_fabric(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
@@ -901,7 +897,7 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
) -> list[dict[str, Any]]: ) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
fabric = str(values(inputs)["fabric"]) fabric = str(values(inputs)["fabric"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request, for_write=True)
nodes = (sdn.get("fabric_nodes") or {}).get(fabric) or {} nodes = (sdn.get("fabric_nodes") or {}).get(fabric) or {}
result = [] result = []
for node_id, item in nodes.items(): for node_id, item in nodes.items():
@@ -919,15 +915,23 @@ def register_sdn_handlers(registry: HandlerRegistry) -> None:
fabric = str(values(inputs)["fabric"]) fabric = str(values(inputs)["fabric"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request)
nodes = (sdn.get("fabric_nodes") or {}).get(fabric) or {} nodes = (sdn.get("fabric_nodes") or {}).get(fabric) or {}
return [{"node": node_id, "state": "up"} for node_id in sorted(nodes)] return [
{
"node": node_id,
"state": item.get("state") if isinstance(item, dict) else "",
}
for node_id, item in sorted(nodes.items())
]
async def node_fabric_routes(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def node_fabric_routes(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
await require_node(request, str(values(inputs)["node"])) await require_node(request, str(values(inputs)["node"]))
fabric = str(values(inputs)["fabric"]) fabric = str(values(inputs)["fabric"])
_metadata, sdn = await _load(request) _metadata, sdn = await _load(request)
item = (sdn.get("fabrics") or {}).get(fabric) or {} item = (sdn.get("fabrics") or {}).get(fabric) or {}
prefix = item.get("ip_prefix") or "10.0.0.0/24" routes = item.get("routes") if isinstance(item, dict) else None
return [{"dst": prefix, "protocol": item.get("protocol") or "ospf"}] if isinstance(routes, list):
return [dict(route) for route in routes if isinstance(route, dict)]
return []
# registrations # registrations
registry.register("/cluster/sdn", "GET", index) registry.register("/cluster/sdn", "GET", index)
+52 -22
View File
@@ -9,7 +9,15 @@ from fastapi import Request
from app.api.errors import ApiError from app.api.errors import ApiError
from app.api.registry import HandlerRegistry from app.api.registry import HandlerRegistry
from app.handlers.common import database, require_node, state, storage_payload, subdirs, values from app.handlers.common import (
database,
require_node,
require_value,
state,
storage_payload,
subdirs,
values,
)
from app.simulation.seed import CLUSTER_ID, stable_id from app.simulation.seed import CLUSTER_ID, stable_id
@@ -485,18 +493,29 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
async def file_restore_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def file_restore_list(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
await _storage_row(request, None, str(payload["storage"])) row = await _storage_row(request, None, str(payload["storage"]))
filepath = str(payload.get("filepath") or "/") filepath = str(payload.get("filepath") or "/")
return [{"filepath": filepath.rstrip("/") + "/etc", "type": "d", "text": "etc"}] config = state(row["config"])
restore = config.get("file_restore")
if not isinstance(restore, dict):
return []
items = restore.get(filepath) or restore.get(filepath.rstrip("/") or "/")
return [dict(item) for item in items] if isinstance(items, list) else []
async def file_restore_download(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def file_restore_download(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
await _storage_row(request, None, str(payload["storage"])) row = await _storage_row(request, None, str(payload["storage"]))
config = state(row["config"])
downloads = config.get("file_restore_downloads")
filepath = str(payload.get("filepath") or "/")
if isinstance(downloads, dict) and filepath in downloads:
entry = downloads[filepath]
return dict(entry) if isinstance(entry, dict) else {"filepath": filepath}
return { return {
"download-url": f"/api2/json/nodes/{payload['node']}/storage/" "download-url": f"/api2/json/nodes/{payload['node']}/storage/"
f"{payload['storage']}/file-restore/download", f"{payload['storage']}/file-restore/download",
"filepath": payload.get("filepath") or "/", "filepath": filepath,
"volume": payload.get("volume"), "volume": payload.get("volume"),
} }
@@ -504,40 +523,51 @@ def register_storage_handlers(registry: HandlerRegistry) -> None:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
row = await _storage_row(request, None, str(payload["storage"])) row = await _storage_row(request, None, str(payload["storage"]))
config = state(row["config"])
identity = config.get("identity")
if isinstance(identity, dict):
return {
"storage": str(row["storage_id"]),
"type": str(identity.get("type") or row["storage_type"]),
"fingerprint": str(identity.get("fingerprint") or ""),
}
return { return {
"storage": str(row["storage_id"]), "storage": str(row["storage_id"]),
"type": str(row["storage_type"]), "type": str(row["storage_type"]),
"fingerprint": f"sim-{row['storage_id']}", "fingerprint": "",
} }
async def import_metadata(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def import_metadata(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
storage_id = str(payload["storage"]) storage_id = str(payload["storage"])
volume = str(payload["volume"]) volume = str(require_value(payload, "volume"))
await _storage_row(request, None, storage_id) row = await _storage_row(request, None, storage_id)
return { config = state(row["config"])
"type": "qemu", by_volume = config.get("import_metadata_by_volume")
"source": volume, if isinstance(by_volume, dict) and volume in by_volume:
"disks": {"scsi0": f"{storage_id}:0/vm-import.raw"}, meta = by_volume[volume]
"net0": "virtio,bridge=vmbr0", return dict(meta) if isinstance(meta, dict) else {}
} meta = config.get("import_metadata")
if isinstance(meta, dict):
return {"source": volume, **meta}
return {}
async def storage_rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]: async def storage_rrd(request: Request, inputs: dict[str, Any]) -> dict[str, Any]:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
storage_id = str(payload["storage"]) row = await _storage_row(request, None, str(payload["storage"]))
await _storage_row(request, None, storage_id) config = state(row["config"])
return {"filename": f"pve-storage-{storage_id}.rrd"} rrd_state = config.get("rrd")
return dict(rrd_state) if isinstance(rrd_state, dict) else {}
async def storage_rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]: async def storage_rrddata(request: Request, inputs: dict[str, Any]) -> list[dict[str, Any]]:
payload = values(inputs) payload = values(inputs)
await require_node(request, str(payload["node"])) await require_node(request, str(payload["node"]))
await _storage_row(request, None, str(payload["storage"])) row = await _storage_row(request, None, str(payload["storage"]))
return [ config = state(row["config"])
{"time": 1_700_000_000, "used": 10, "total": 100}, series = config.get("rrddata")
{"time": 1_700_000_060, "used": 12, "total": 100}, return [dict(item) for item in series] if isinstance(series, list) else []
]
registry.register("/storage", "GET", storage_ids) registry.register("/storage", "GET", storage_ids)
registry.register("/storage", "POST", storage_create) registry.register("/storage", "POST", storage_create)
+1127 -20
View File
File diff suppressed because it is too large Load Diff
+5 -5
View File
@@ -159,16 +159,16 @@ async def probe_major(
verb = method.verb.upper() verb = method.verb.upper()
url = f"/api2/json{render_path(path_template)}" url = f"/api2/json{render_path(path_template)}"
headers = {"CSRFPreventionToken": csrf} if verb != "GET" else {} headers = {"CSRFPreventionToken": csrf} if verb != "GET" else {}
body = body_for(method, path_template) if verb in {"PUT", "POST"} else None payload = body_for(method, path_template)
try: try:
if verb == "GET": if verb == "GET":
response = await client.get(url, headers=headers) response = await client.get(url, headers=headers, params=payload or None)
elif verb == "PUT": elif verb == "PUT":
response = await client.put(url, data=body or {}, headers=headers) response = await client.put(url, data=payload or {}, headers=headers)
elif verb == "POST": elif verb == "POST":
response = await client.post(url, data=body or {}, headers=headers) response = await client.post(url, data=payload or {}, headers=headers)
elif verb == "DELETE": elif verb == "DELETE":
response = await client.delete(url, headers=headers) response = await client.request("DELETE", url, data=payload or {}, headers=headers)
else: else:
continue continue
except Exception as exc: except Exception as exc:
+3 -1
View File
@@ -62,10 +62,12 @@ def qemu_handler(repository: TaskRepository, clock: Clock) -> TaskHandler:
async def _create(repository: TaskRepository, task: Task) -> dict[str, Any]: async def _create(repository: TaskRepository, task: Task) -> dict[str, Any]:
from app.simulation.seed import enrich_guest_state
node, vmid = str(task.payload["node"]), int(task.payload["vmid"]) node, vmid = str(task.payload["node"]), int(task.payload["vmid"])
config = dict(task.payload.get("config", {})) config = dict(task.payload.get("config", {}))
resource_id = uuid.uuid4() resource_id = uuid.uuid4()
state = {"status": "stopped", **config} state = enrich_guest_state({"status": "stopped", **config}, kind="qemu", vmid=str(vmid))
async with repository.pool.acquire() as connection: async with repository.pool.acquire() as connection:
async with connection.transaction(): async with connection.transaction():
node_row = await connection.fetchrow( node_row = await connection.fetchrow(
+310 -30
View File
@@ -26,22 +26,22 @@
--muted: #5c6778; --muted: #5c6778;
--accent: #ff8500; --accent: #ff8500;
--accent-hover: #ff6a00; --accent-hover: #ff6a00;
--link: #1e88ff; --link: #d97706;
--ok: #22c55e; --ok: #22c55e;
--ok-bg: #e6f8ed; --ok-bg: #e6f8ed;
--warn: #9a6700; --warn: #9a6700;
--warn-bg: #fff8e6; --warn-bg: #fff8e6;
--err: #b42318; --err: #b42318;
--err-bg: #fdecea; --err-bg: #fdecea;
--info: #1e88ff; --info: #d97706;
--info-bg: #e8f2ff; --info-bg: #fff4e8;
--debug: #5c6778; --debug: #5c6778;
--debug-bg: #eef1f6; --debug-bg: #eef1f6;
--surface-raised: #fafbfc; --surface-raised: #fafbfc;
--surface-hover: #f3f5f8; --surface-hover: #f3f5f8;
--surface-toolbar: #f6f8fb; --surface-toolbar: #f6f8fb;
--surface-muted-hover: #eef1f6; --surface-muted-hover: #eef1f6;
--surface-accent: #eef4ff; --surface-accent: #fff4e8;
--surface-danger: #fffafa; --surface-danger: #fffafa;
--border-strong: #c8d0db; --border-strong: #c8d0db;
--brand-ink: #0f1419; --brand-ink: #0f1419;
@@ -50,14 +50,14 @@
--brand-logo-bg: var(--overlay-head-bg); --brand-logo-bg: var(--overlay-head-bg);
--brand-logo-border: var(--border-strong); --brand-logo-border: var(--border-strong);
--brand-subtle: #97a1b1; --brand-subtle: #97a1b1;
--meta-desc-bg: #f4f7fb; --meta-desc-bg: #fff8f0;
--meta-desc-border: #7eb6ff; --meta-desc-border: #ffb366;
--meta-desc-text: #1e88ff; --meta-desc-text: #c45a00;
--overlay-head-bg: #eef1f6; --overlay-head-bg: #eef1f6;
--brand-accent: #ff8500; --brand-accent: #ff8500;
--brand-accent-hover: #ff6a00; --brand-accent-hover: #ff6a00;
--panel-accent: #22c55e; --panel-accent: #ff8500;
--panel-accent-hover: #16a34a; --panel-accent-hover: #ff6a00;
--mono: "SF Mono", "Cascadia Code", Consolas, monospace; --mono: "SF Mono", "Cascadia Code", Consolas, monospace;
--sans: "Segoe UI", system-ui, sans-serif; --sans: "Segoe UI", system-ui, sans-serif;
--header-h: 52px; --header-h: 52px;
@@ -74,22 +74,22 @@
--muted: #b0bac8; --muted: #b0bac8;
--accent: #ff8a1f; --accent: #ff8a1f;
--accent-hover: #ffaa55; --accent-hover: #ffaa55;
--link: #93c5fd; --link: #ffb366;
--ok: #86efac; --ok: #86efac;
--ok-bg: #1a3d28; --ok-bg: #1a3d28;
--warn: #f0c866; --warn: #f0c866;
--warn-bg: #3d3218; --warn-bg: #3d3218;
--err: #fca5a5; --err: #fca5a5;
--err-bg: #3d2222; --err-bg: #3d2222;
--info: #93c5fd; --info: #ffb366;
--info-bg: #1a2d4a; --info-bg: #3d2a18;
--debug: #b0bac8; --debug: #b0bac8;
--debug-bg: #283240; --debug-bg: #283240;
--surface-raised: #242e3d; --surface-raised: #242e3d;
--surface-hover: #2f3a4b; --surface-hover: #2f3a4b;
--surface-toolbar: #151c27; --surface-toolbar: #151c27;
--surface-muted-hover: #2f3a4b; --surface-muted-hover: #2f3a4b;
--surface-accent: #1e3150; --surface-accent: #3d2a18;
--surface-danger: #3a2626; --surface-danger: #3a2626;
--brand-ink: #f2f5f9; --brand-ink: #f2f5f9;
--brand-logo-core: #ffffff; --brand-logo-core: #ffffff;
@@ -97,14 +97,14 @@
--brand-logo-bg: var(--overlay-head-bg); --brand-logo-bg: var(--overlay-head-bg);
--brand-logo-border: var(--border-strong); --brand-logo-border: var(--border-strong);
--brand-subtle: #9aa8ba; --brand-subtle: #9aa8ba;
--meta-desc-bg: #1e2836; --meta-desc-bg: #2a2218;
--meta-desc-border: #3d5a85; --meta-desc-border: #8a5a28;
--meta-desc-text: #c8d8ef; --meta-desc-text: #ffc078;
--overlay-head-bg: #242e3d; --overlay-head-bg: #242e3d;
--brand-accent: #ff8a1f; --brand-accent: #ff8a1f;
--brand-accent-hover: #ffaa55; --brand-accent-hover: #ffaa55;
--panel-accent: #4ade80; --panel-accent: #ff8a1f;
--panel-accent-hover: #86efac; --panel-accent-hover: #ffaa55;
} }
html[data-theme="dark"] .header-tool-btn { html[data-theme="dark"] .header-tool-btn {
@@ -537,7 +537,7 @@
} }
.header-tool-btn.data-badge-btn.loaded .data-badge-dot { .header-tool-btn.data-badge-btn.loaded .data-badge-dot {
background: var(--ok); background: var(--brand-accent);
} }
.help-demo-panel { .help-demo-panel {
@@ -1124,7 +1124,7 @@
grid-row: 1; grid-row: 1;
display: grid; display: grid;
position: relative; position: relative;
font-size: 15px; font-size: 16.5px;
font-weight: 700; font-weight: 700;
letter-spacing: 0.07em; letter-spacing: 0.07em;
text-transform: uppercase; text-transform: uppercase;
@@ -1158,18 +1158,22 @@
.workspace-brand-sub { .workspace-brand-sub {
grid-row: 2; grid-row: 2;
display: block; display: block;
box-sizing: border-box;
width: 100%; width: 100%;
max-width: 100%;
min-width: 0; min-width: 0;
font-size: 9.75px; font-size: 7.75px;
font-weight: 600; font-weight: 500;
letter-spacing: 0.02em; letter-spacing: 0.14em;
line-height: 1.2; line-height: 1.15;
text-transform: uppercase; text-transform: uppercase;
color: var(--muted); text-align: left;
color: var(--brand-subtle);
white-space: nowrap; white-space: nowrap;
text-align: justify; }
text-align-last: justify;
word-spacing: 0; html[data-theme="dark"] .workspace-brand-sub {
color: var(--border-strong);
} }
.workspace-topbar-actions { .workspace-topbar-actions {
@@ -1769,6 +1773,7 @@
.data-drawer, .data-drawer,
.url-drawer, .url-drawer,
.endpoints-drawer, .endpoints-drawer,
.upid-drawer,
.help-drawer, .help-drawer,
.ui-modal { .ui-modal {
--accent: var(--panel-accent); --accent: var(--panel-accent);
@@ -1783,6 +1788,7 @@
.data-drawer-head, .data-drawer-head,
.url-drawer-head, .url-drawer-head,
.endpoints-drawer-head, .endpoints-drawer-head,
.upid-drawer-head,
.help-drawer-head { .help-drawer-head {
display: flex; display: flex;
align-items: center; align-items: center;
@@ -1803,6 +1809,7 @@
.data-drawer-head h2, .data-drawer-head h2,
.url-drawer-head h2, .url-drawer-head h2,
.endpoints-drawer-head h2, .endpoints-drawer-head h2,
.upid-drawer-head h2,
.help-drawer-head h2 { .help-drawer-head h2 {
margin: 0; margin: 0;
font-size: 11px; font-size: 11px;
@@ -1828,6 +1835,7 @@
.catalog-drawer-head .icon-btn, .catalog-drawer-head .icon-btn,
.environment-drawer-head .icon-btn, .environment-drawer-head .icon-btn,
.data-drawer-head .icon-btn, .data-drawer-head .icon-btn,
.upid-drawer-head .icon-btn,
.url-drawer-head .icon-btn, .url-drawer-head .icon-btn,
.endpoints-drawer-head .icon-btn, .endpoints-drawer-head .icon-btn,
.help-drawer-head .icon-btn { .help-drawer-head .icon-btn {
@@ -1842,6 +1850,7 @@
.catalog-drawer-head .icon-btn:hover, .catalog-drawer-head .icon-btn:hover,
.environment-drawer-head .icon-btn:hover, .environment-drawer-head .icon-btn:hover,
.data-drawer-head .icon-btn:hover, .data-drawer-head .icon-btn:hover,
.upid-drawer-head .icon-btn:hover,
.url-drawer-head .icon-btn:hover, .url-drawer-head .icon-btn:hover,
.endpoints-drawer-head .icon-btn:hover, .endpoints-drawer-head .icon-btn:hover,
.help-drawer-head .icon-btn:hover { .help-drawer-head .icon-btn:hover {
@@ -1869,6 +1878,7 @@
.catalog-backdrop, .catalog-backdrop,
.environment-backdrop, .environment-backdrop,
.data-backdrop, .data-backdrop,
.upid-backdrop,
.url-backdrop, .url-backdrop,
.endpoints-backdrop, .endpoints-backdrop,
.help-backdrop { .help-backdrop {
@@ -1887,6 +1897,7 @@
.catalog-backdrop.open, .catalog-backdrop.open,
.environment-backdrop.open, .environment-backdrop.open,
.data-backdrop.open, .data-backdrop.open,
.upid-backdrop.open,
.url-backdrop.open, .url-backdrop.open,
.endpoints-backdrop.open, .endpoints-backdrop.open,
.help-backdrop.open { .help-backdrop.open {
@@ -1900,6 +1911,7 @@
.catalog-drawer, .catalog-drawer,
.environment-drawer, .environment-drawer,
.data-drawer, .data-drawer,
.upid-drawer,
.url-drawer, .url-drawer,
.endpoints-drawer, .endpoints-drawer,
.help-drawer { .help-drawer {
@@ -1926,6 +1938,7 @@
.catalog-drawer.open, .catalog-drawer.open,
.environment-drawer.open, .environment-drawer.open,
.data-drawer.open, .data-drawer.open,
.upid-drawer.open,
.url-drawer.open, .url-drawer.open,
.endpoints-drawer.open, .endpoints-drawer.open,
.help-drawer.open { .help-drawer.open {
@@ -1938,6 +1951,7 @@
.auth-drawer-body, .auth-drawer-body,
.environment-drawer-body, .environment-drawer-body,
.data-drawer-body, .data-drawer-body,
.upid-drawer-body,
.url-drawer-body, .url-drawer-body,
.endpoints-drawer-body, .endpoints-drawer-body,
.help-drawer-body { .help-drawer-body {
@@ -1959,6 +1973,7 @@
.catalog-drawer-body .btn, .catalog-drawer-body .btn,
.environment-drawer-body .btn, .environment-drawer-body .btn,
.data-drawer-body .btn, .data-drawer-body .btn,
.upid-drawer-body .btn,
.url-drawer-body .btn, .url-drawer-body .btn,
.endpoints-drawer-body .btn, .endpoints-drawer-body .btn,
.help-drawer-body .btn, .help-drawer-body .btn,
@@ -1973,6 +1988,80 @@
line-height: 1.2; line-height: 1.2;
} }
.upid-panel {
display: flex;
flex-direction: column;
gap: 14px;
}
.upid-hint {
margin: 0;
color: var(--muted);
font-size: 12px;
line-height: 1.45;
}
.upid-hint code {
font-family: var(--mono);
font-size: 11px;
}
.upid-form {
display: flex;
flex-direction: column;
gap: 12px;
}
.upid-form .field label {
display: block;
margin-bottom: 6px;
font-size: 11px;
font-weight: 600;
letter-spacing: 0.04em;
text-transform: uppercase;
color: var(--muted);
}
.upid-form .field input {
width: 100%;
box-sizing: border-box;
padding: 10px 12px;
border: 1px solid var(--border);
border-radius: 8px;
background: var(--surface-raised);
color: var(--text);
font-family: var(--mono);
font-size: 12px;
}
.upid-form .field input:focus {
outline: none;
border-color: var(--brand-accent);
}
.upid-actions {
display: flex;
flex-wrap: wrap;
gap: 8px;
}
.upid-output {
margin: 0;
min-height: 220px;
max-height: min(55vh, 520px);
overflow: auto;
padding: 12px 14px;
border: 1px solid var(--border);
border-radius: 8px;
background: var(--surface-raised);
color: var(--text);
font-family: var(--mono);
font-size: 12px;
line-height: 1.45;
white-space: pre-wrap;
word-break: break-word;
}
.help-drawer-body { .help-drawer-body {
display: flex; display: flex;
flex-direction: column; flex-direction: column;
@@ -3173,7 +3262,7 @@
<span class="workspace-brand-name-ghost" aria-hidden="true">PROXMOX</span> <span class="workspace-brand-name-ghost" aria-hidden="true">PROXMOX</span>
<span class="workspace-brand-name-text"><span class="workspace-brand-core">PRO</span><span class="workspace-brand-x">X</span><span class="workspace-brand-core">MO</span><span class="workspace-brand-x">X</span></span> <span class="workspace-brand-name-text"><span class="workspace-brand-core">PRO</span><span class="workspace-brand-x">X</span><span class="workspace-brand-core">MO</span><span class="workspace-brand-x">X</span></span>
</span> </span>
<span class="workspace-brand-sub">API&nbsp;Simulator</span> <span class="workspace-brand-sub">API Simulator</span>
</span> </span>
</a> </a>
<div class="workspace-topbar-actions"> <div class="workspace-topbar-actions">
@@ -3188,6 +3277,7 @@
<span>History</span> <span>History</span>
<span class="history-badge-count" id="history-badge-count"></span> <span class="history-badge-count" id="history-badge-count"></span>
</button> </button>
<button type="button" class="header-tool-btn ok upid-badge-btn" id="upid-badge" aria-expanded="false" aria-controls="upid-drawer" data-tooltip="Poll task status and logs by UPID">UPID</button>
<button type="button" class="header-tool-btn ok data-badge-btn" id="data-badge" aria-expanded="false" aria-controls="data-drawer"> <button type="button" class="header-tool-btn ok data-badge-btn" id="data-badge" aria-expanded="false" aria-controls="data-drawer">
<span>Data</span> <span>Data</span>
<span class="data-badge-dot" aria-hidden="true"></span> <span class="data-badge-dot" aria-hidden="true"></span>
@@ -3431,6 +3521,42 @@
</div> </div>
</aside> </aside>
<div class="upid-backdrop" id="upid-backdrop" aria-hidden="true"></div>
<aside class="upid-drawer" id="upid-drawer" aria-labelledby="upid-drawer-title" aria-hidden="true">
<div class="upid-drawer-head">
<h2 id="upid-drawer-title">UPID</h2>
<button class="icon-btn" id="upid-drawer-close" type="button" data-tooltip="Close panel" aria-label="Close panel">
<svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M6 6l12 12M18 6L6 18"/>
</svg>
</button>
</div>
<div class="upid-drawer-body">
<div class="upid-panel">
<p class="upid-hint">
Async mutations return a Proxmox-style <strong>UPID</strong>. Poll
<code>/nodes/{node}/tasks/{upid}/status</code> until the task finishes, then inspect the log.
</p>
<div class="upid-form">
<div class="field">
<label for="upid-node">Node</label>
<input id="upid-node" value="pve01" spellcheck="false" autocomplete="off">
</div>
<div class="field">
<label for="upid-value">UPID</label>
<input id="upid-value" placeholder="UPID:pve01:… returned by a mutation" spellcheck="false" autocomplete="off">
</div>
<div class="upid-actions">
<button class="btn btn-primary btn-sm" id="btn-upid-status" type="button">Task status</button>
<button class="btn btn-sm" id="btn-upid-log" type="button">Task log</button>
<button class="btn btn-sm" id="btn-upid-from-response" type="button">From last response</button>
</div>
</div>
<pre class="upid-output" id="upid-output" aria-live="polite">Paste a UPID or send an async request, then poll status / log.</pre>
</div>
</div>
</aside>
<div class="url-backdrop" id="url-backdrop" aria-hidden="true"></div> <div class="url-backdrop" id="url-backdrop" aria-hidden="true"></div>
<aside class="url-drawer" id="url-drawer" aria-labelledby="url-drawer-title" aria-hidden="true"> <aside class="url-drawer" id="url-drawer" aria-labelledby="url-drawer-title" aria-hidden="true">
<div class="url-drawer-head"> <div class="url-drawer-head">
@@ -3699,6 +3825,7 @@
history: readStoredHistory(), history: readStoredHistory(),
demoState: null, demoState: null,
lastResponseRaw: "", lastResponseRaw: "",
lastUpid: null,
}; };
let methodDetailsRequestId = 0; let methodDetailsRequestId = 0;
@@ -3742,6 +3869,16 @@
dataDrawer: document.getElementById("data-drawer"), dataDrawer: document.getElementById("data-drawer"),
dataDrawerClose: document.getElementById("data-drawer-close"), dataDrawerClose: document.getElementById("data-drawer-close"),
dataPanel: document.getElementById("data-panel"), dataPanel: document.getElementById("data-panel"),
upidBadge: document.getElementById("upid-badge"),
upidBackdrop: document.getElementById("upid-backdrop"),
upidDrawer: document.getElementById("upid-drawer"),
upidDrawerClose: document.getElementById("upid-drawer-close"),
upidNode: document.getElementById("upid-node"),
upidValue: document.getElementById("upid-value"),
upidOutput: document.getElementById("upid-output"),
btnUpidStatus: document.getElementById("btn-upid-status"),
btnUpidLog: document.getElementById("btn-upid-log"),
btnUpidFromResponse: document.getElementById("btn-upid-from-response"),
catalogImplFill: document.getElementById("catalog-impl-fill"), catalogImplFill: document.getElementById("catalog-impl-fill"),
catalogImplValue: document.getElementById("catalog-impl-value"), catalogImplValue: document.getElementById("catalog-impl-value"),
endpointsBadge: document.getElementById("endpoints-badge"), endpointsBadge: document.getElementById("endpoints-badge"),
@@ -5114,6 +5251,124 @@
toggleDataDrawer(false); toggleDataDrawer(false);
} }
function toggleUpidDrawer(force) {
if (!els.upidDrawer || !els.upidBackdrop || !els.upidBadge) return;
const open = typeof force === "boolean" ? force : !els.upidDrawer.classList.contains("open");
els.upidDrawer.classList.toggle("open", open);
els.upidBackdrop.classList.toggle("open", open);
els.upidBadge.setAttribute("aria-expanded", open ? "true" : "false");
els.upidDrawer.setAttribute("aria-hidden", open ? "false" : "true");
els.upidBackdrop.setAttribute("aria-hidden", open ? "false" : "true");
if (open && state.lastUpid && els.upidValue && !els.upidValue.value.trim()) {
applyCapturedUpid(state.lastUpid);
}
}
function closeUpidDrawer() {
toggleUpidDrawer(false);
}
function parseUpidNode(upid) {
const match = String(upid || "").match(/^UPID:([^:]+):/i);
return match ? match[1] : null;
}
function extractUpidFromPayload(payload) {
if (typeof payload === "string" && /^UPID:/i.test(payload.trim())) {
return payload.trim();
}
if (!payload || typeof payload !== "object") return null;
if (typeof payload.data === "string" && /^UPID:/i.test(payload.data.trim())) {
return payload.data.trim();
}
if (typeof payload.upid === "string" && /^UPID:/i.test(payload.upid.trim())) {
return payload.upid.trim();
}
if (payload.data && typeof payload.data === "object") {
if (typeof payload.data.upid === "string" && /^UPID:/i.test(payload.data.upid.trim())) {
return payload.data.upid.trim();
}
}
return null;
}
function applyCapturedUpid(upid) {
if (!upid || !els.upidValue) return false;
els.upidValue.value = upid;
const node = parseUpidNode(upid);
if (node && els.upidNode) els.upidNode.value = node;
return true;
}
function captureUpidFromResult(result) {
const upid = extractUpidFromPayload(result?.body);
if (!upid) return;
state.lastUpid = upid;
applyCapturedUpid(upid);
if (els.upidBadge) {
els.upidBadge.classList.remove("warn");
els.upidBadge.classList.add("ok");
els.upidBadge.dataset.tooltip = `Last UPID captured · ${upid}`;
}
}
function showUpidOutput(value) {
if (!els.upidOutput) return;
if (value !== null && typeof value === "object") {
els.upidOutput.textContent = JSON.stringify(value, null, 2);
} else {
els.upidOutput.textContent = String(value ?? "");
}
}
async function fetchUpidTask(kind) {
const node = (els.upidNode?.value || "").trim();
const upid = (els.upidValue?.value || "").trim();
if (!node) {
showUpidOutput("Node is required.");
toast("Enter a node name", "warn");
return;
}
if (!upid) {
showUpidOutput("UPID is required.");
toast("Enter a UPID", "warn");
return;
}
if (!state.ticket) {
toast("Sign in first — task endpoints require authentication", "warn");
}
const suffix = kind === "log" ? "log" : "status";
const path = `/nodes/${encodeURIComponent(node)}/tasks/${encodeURIComponent(upid)}/${suffix}`;
setLoading(true);
try {
const result = await api("GET", path);
showUpidOutput({ status: result.status, data: result.body });
const toastLevel = result.status >= 500 ? "error" : result.status >= 400 ? "warn" : "ok";
toast(`UPID ${suffix} · HTTP ${result.status}`, toastLevel);
} catch (error) {
showUpidOutput(String(error));
toast(`UPID ${suffix} failed`, "error");
} finally {
setLoading(false);
}
}
function fillUpidFromLastResponse() {
const fromState = state.lastUpid || extractUpidFromPayload(
(() => {
try { return JSON.parse(state.lastResponseRaw); } catch { return state.lastResponseRaw; }
})(),
);
if (!fromState) {
toast("No UPID found in the last response", "warn");
showUpidOutput("Last response did not contain a UPID string in data.");
return;
}
applyCapturedUpid(fromState);
state.lastUpid = fromState;
toast("UPID loaded from last response", "ok");
}
function buildDemoDataHtml(data) { function buildDemoDataHtml(data) {
const loaded = Boolean(data?.loaded); const loaded = Boolean(data?.loaded);
const cephPiB = data?.ceph_capacity_pib; const cephPiB = data?.ceph_capacity_pib;
@@ -5399,7 +5654,7 @@
<path d="M11 6h2"/> <path d="M11 6h2"/>
</svg> </svg>
</a> </a>
<a class="help-about-social-github" href="https://github.com/inecs" target="_blank" rel="noopener noreferrer" aria-label="GitHub" data-tooltip="github.com/inecs"> <a class="help-about-social-github" href="https://github.com/sergeyantropoff" target="_blank" rel="noopener noreferrer" aria-label="GitHub" data-tooltip="github.com/sergeyantropoff">
<svg viewBox="0 0 24 24" aria-hidden="true"> <svg viewBox="0 0 24 24" aria-hidden="true">
<path d="M12 2a10 10 0 0 0-3.2 19.5c.5.1.7-.2.7-.5v-1.8c-3 .7-3.6-1.3-3.6-1.3-.5-1.1-1.1-1.4-1.1-1.4-.9-.6.1-.6.1-.6 1 .1 1.5 1 1.5 1 .9 1.5 2.4 1.1 3 .8.1-.7.4-1.1.7-1.4-2.4-.3-5-1.2-5-5.5a4.3 4.3 0 0 1 1.2-3 4.3 4.3 0 0 1 3.2-1.3c1.2 0 2.2.4 3 1a7.5 7.5 0 0 1 4.6-1.6c.5 0 1 .1 1.5.2a4.3 4.3 0 0 1 1.2 3c0 3.3-2.6 4.1-5 4.3-1.1.9-1 1.4-1 3.6v2.7c0 .3.2.6.7.5A10 10 0 0 0 12 2z"/> <path d="M12 2a10 10 0 0 0-3.2 19.5c.5.1.7-.2.7-.5v-1.8c-3 .7-3.6-1.3-3.6-1.3-.5-1.1-1.1-1.4-1.1-1.4-.9-.6.1-.6.1-.6 1 .1 1.5 1 1.5 1 .9 1.5 2.4 1.1 3 .8.1-.7.4-1.1.7-1.4-2.4-.3-5-1.2-5-5.5a4.3 4.3 0 0 1 1.2-3 4.3 4.3 0 0 1 3.2-1.3c1.2 0 2.2.4 3 1a7.5 7.5 0 0 1 4.6-1.6c.5 0 1 .1 1.5.2a4.3 4.3 0 0 1 1.2 3c0 3.3-2.6 4.1-5 4.3-1.1.9-1 1.4-1 3.6v2.7c0 .3.2.6.7.5A10 10 0 0 0 12 2z"/>
</svg> </svg>
@@ -5542,6 +5797,7 @@
if (except !== "params") closeParamsDrawer(); if (except !== "params") closeParamsDrawer();
if (except !== "environment") closeEnvironmentDrawer(); if (except !== "environment") closeEnvironmentDrawer();
if (except !== "data") closeDataDrawer(); if (except !== "data") closeDataDrawer();
if (except !== "upid") closeUpidDrawer();
if (except !== "url") closeUrlDrawer(); if (except !== "url") closeUrlDrawer();
if (except !== "endpoints") closeEndpointsDrawer(); if (except !== "endpoints") closeEndpointsDrawer();
if (except !== "help") closeHelpDrawer(); if (except !== "help") closeHelpDrawer();
@@ -6495,6 +6751,7 @@
try { try {
const result = await api(method, path.startsWith("/") ? path : `/${path}`, body); const result = await api(method, path.startsWith("/") ? path : `/${path}`, body);
showResponse(result); showResponse(result);
captureUpidFromResult(result);
updateResolvedPath(); updateResolvedPath();
pushHistory( pushHistory(
method, method,
@@ -6647,6 +6904,11 @@
closeOtherPanels("data"); closeOtherPanels("data");
toggleDataDrawer(!els.dataDrawer.classList.contains("open")); toggleDataDrawer(!els.dataDrawer.classList.contains("open"));
}); });
els.upidBadge?.addEventListener("click", (e) => {
e.stopPropagation();
closeOtherPanels("upid");
toggleUpidDrawer(!els.upidDrawer.classList.contains("open"));
});
els.themeToggle.addEventListener("click", () => toggleTheme()); els.themeToggle.addEventListener("click", () => toggleTheme());
els.helpBadge.addEventListener("click", (e) => { els.helpBadge.addEventListener("click", (e) => {
e.stopPropagation(); e.stopPropagation();
@@ -6671,6 +6933,24 @@
els.environmentBackdrop.addEventListener("click", () => closeEnvironmentDrawer()); els.environmentBackdrop.addEventListener("click", () => closeEnvironmentDrawer());
els.dataDrawerClose.addEventListener("click", () => closeDataDrawer()); els.dataDrawerClose.addEventListener("click", () => closeDataDrawer());
els.dataBackdrop.addEventListener("click", () => closeDataDrawer()); els.dataBackdrop.addEventListener("click", () => closeDataDrawer());
els.upidDrawerClose?.addEventListener("click", () => closeUpidDrawer());
els.upidBackdrop?.addEventListener("click", () => closeUpidDrawer());
els.btnUpidStatus?.addEventListener("click", () => {
fetchUpidTask("status").catch((error) => showError(String(error)));
});
els.btnUpidLog?.addEventListener("click", () => {
fetchUpidTask("log").catch((error) => showError(String(error)));
});
els.btnUpidFromResponse?.addEventListener("click", () => fillUpidFromLastResponse());
els.upidValue?.addEventListener("keydown", (e) => {
if (e.key === "Enter") {
fetchUpidTask("status").catch((error) => showError(String(error)));
}
});
els.upidValue?.addEventListener("input", () => {
const node = parseUpidNode(els.upidValue.value);
if (node && els.upidNode) els.upidNode.value = node;
});
els.historyDrawerClose.addEventListener("click", () => closeHistoryDrawer()); els.historyDrawerClose.addEventListener("click", () => closeHistoryDrawer());
els.historyBackdrop.addEventListener("click", () => closeHistoryDrawer()); els.historyBackdrop.addEventListener("click", () => closeHistoryDrawer());
els.historyClear.addEventListener("click", () => { els.historyClear.addEventListener("click", () => {
+51 -2
View File
@@ -1,12 +1,28 @@
# Quick start with the published Docker Hub runtime image. # Quick start with the published Docker Hub runtime image.
# #
# WARNING: Laboratory / CI only.
# Default TICKET_SIGNING_KEY and PostgreSQL password are intentional lab
# defaults. Do NOT expose host :8006 to untrusted networks without replacing
# secrets and adding your own controls. See SECURITY.md and docs/security.md.
#
# Requires this repository checkout (compose file).
#
# docker compose -f docker-compose.release.yml up -d # docker compose -f docker-compose.release.yml up -d
# docker compose -f docker-compose.release.yml run --rm --entrypoint python simulator -m app.simulation.seed_cli # docker compose -f docker-compose.release.yml run --rm --entrypoint python \
# simulator -m app.simulation.seed_cli
# curl -sS http://localhost:8006/health/ready
# #
# Override the image tag: # Override the image tag:
# IMAGE_TAG=0.1.0 docker compose -f docker-compose.release.yml up -d # IMAGE_TAG=0.1.0 docker compose -f docker-compose.release.yml up -d
# #
# Change TICKET_SIGNING_KEY before exposing the stack beyond a local lab. # Rotate lab secrets before any shared or networked demo:
# TICKET_SIGNING_KEY=$(openssl rand -hex 32) \
# POSTGRES_PASSWORD=$(openssl rand -hex 16) \
# docker compose -f docker-compose.release.yml up -d
#
# Optional HTTPS for proxmoxer-style clients (separate port):
# docker compose -f docker-compose.release.yml --profile tls up -d
# curl -sk https://localhost:8443/health/ready
name: proxmox-api-simulator-release name: proxmox-api-simulator-release
@@ -18,6 +34,7 @@ x-app-env: &app-env
CONTRACT_SNAPSHOT: /app/contracts/pve-9.2.3.json CONTRACT_SNAPSHOT: /app/contracts/pve-9.2.3.json
COMPATIBILITY_EVIDENCE: /app/evidence/pve-9.2.3.json COMPATIBILITY_EVIDENCE: /app/evidence/pve-9.2.3.json
LOG_LEVEL: ${LOG_LEVEL:-INFO} LOG_LEVEL: ${LOG_LEVEL:-INFO}
# Lab default — replace for any shared or networked use.
TICKET_SIGNING_KEY: ${TICKET_SIGNING_KEY:-development-only-signing-key-change-me} TICKET_SIGNING_KEY: ${TICKET_SIGNING_KEY:-development-only-signing-key-change-me}
TASK_WORKER_CONCURRENCY: ${TASK_WORKER_CONCURRENCY:-2} TASK_WORKER_CONCURRENCY: ${TASK_WORKER_CONCURRENCY:-2}
SIMULATION_TIME_SCALE: ${SIMULATION_TIME_SCALE:-10} SIMULATION_TIME_SCALE: ${SIMULATION_TIME_SCALE:-10}
@@ -86,3 +103,35 @@ services:
start_period: 20s start_period: 20s
ports: ports:
- "${SIMULATOR_PORT:-8006}:8006" - "${SIMULATOR_PORT:-8006}:8006"
tls-gateway:
profiles: [tls]
image: nginx:1.28.0-alpine
restart: unless-stopped
networks: [simulator]
depends_on:
simulator:
condition: service_healthy
ports:
- "${TLS_GATEWAY_PORT:-8443}:8443"
volumes:
- ./docker/tls/gateway.conf:/etc/nginx/conf.d/default.conf:ro
- ./docker/tls/server.crt:/etc/nginx/tls/server.crt:ro
- ./docker/tls/server.key:/etc/nginx/tls/server.key:ro
healthcheck:
test:
[
"CMD-SHELL",
"wget -qO- --no-check-certificate https://127.0.0.1:8443/health/live || exit 1",
]
interval: 10s
timeout: 3s
retries: 5
start_period: 5s
read_only: true
tmpfs:
- /var/cache/nginx
- /tmp
- /var/run
security_opt:
- no-new-privileges:true
+8 -4
View File
@@ -12,8 +12,9 @@ x-dev-env: &dev-env
TEST_DATABASE_URL: postgresql://proxmox:proxmox@postgres:5432/proxmox_simulator TEST_DATABASE_URL: postgresql://proxmox:proxmox@postgres:5432/proxmox_simulator
CONTRACT_SNAPSHOT: /workspace/contracts/e61a893e996d05d376579226e7dfbedbcfce8b71787adacffbc557e6e35901c1/snapshot.json CONTRACT_SNAPSHOT: /workspace/contracts/e61a893e996d05d376579226e7dfbedbcfce8b71787adacffbc557e6e35901c1/snapshot.json
COMPATIBILITY_EVIDENCE: /workspace/evidence/pve-9.2.3.json COMPATIBILITY_EVIDENCE: /workspace/evidence/pve-9.2.3.json
PROXMOXER_HOST: tls-gateway # proxmoxer is HTTPS-only — set when using --profile tls (see Makefile test-compatibility)
PROXMOXER_PORT: "8443" PROXMOXER_HOST: ${PROXMOXER_HOST:-}
PROXMOXER_PORT: ${PROXMOXER_PORT:-8443}
LOG_LEVEL: INFO LOG_LEVEL: INFO
TICKET_SIGNING_KEY: development-only-signing-key-change-me TICKET_SIGNING_KEY: development-only-signing-key-change-me
@@ -108,9 +109,12 @@ services:
retries: 8 retries: 8
start_period: 20s start_period: 20s
ports: ports:
- "8006:8006" - "${SIMULATOR_PORT:-8006}:8006"
# Optional HTTPS front for clients that cannot speak plain HTTP (proxmoxer).
# Default lab URL is http://localhost:8006/ — K8s TLS is Ingress-only.
tls-gateway: tls-gateway:
profiles: [tls]
image: nginx:1.28.0-alpine image: nginx:1.28.0-alpine
restart: unless-stopped restart: unless-stopped
networks: [simulator] networks: [simulator]
@@ -118,7 +122,7 @@ services:
simulator: simulator:
condition: service_healthy condition: service_healthy
ports: ports:
- "8007:8443" - "${TLS_GATEWAY_PORT:-8443}:8443"
volumes: volumes:
- ./docker/tls/gateway.conf:/etc/nginx/conf.d/default.conf:ro - ./docker/tls/gateway.conf:/etc/nginx/conf.d/default.conf:ro
- ./docker/tls/server.crt:/etc/nginx/tls/server.crt:ro - ./docker/tls/server.crt:/etc/nginx/tls/server.crt:ro
+23 -2
View File
@@ -1,5 +1,26 @@
# Development HTTPS gateway for proxmoxer and other TLS clients. # Optional / internal TLS terminator.
# Upstream hostnames are resolved at request time via Docker embedded DNS. #
# - Host lab default is plain HTTP :8006 on the simulator (no TLS).
# - Kubernetes HTTPS terminates at Ingress (cert-manager).
# - This gateway is for HTTPS-only clients (proxmoxer, pulumi-proxmoxve).
# - Also listen on :80: some bridged providers rewrite https://host:8443 → http://host
# on later requests (delete); without :80 those calls fail.
server {
listen 80;
server_name _;
resolver 127.0.0.11 valid=10s ipv6=off;
location / {
set $upstream simulator:8006;
proxy_pass http://$upstream;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Request-ID $request_id;
}
}
server { server {
listen 8443 ssl; listen 8443 ssl;
+30
View File
@@ -0,0 +1,30 @@
**Language / Язык:** [English](README.md) | [Русский](ru/README.md)
# Documentation
Guides for the Proxmox VE API simulator. Switch language with the header on each
page. Russian mirrors live under [`ru/`](ru/README.md).
| Guide | Description |
|---|---|
| [Getting started](getting-started.md) | First successful lab session |
| [Configuration](configuration.md) | Environment variables and Compose |
| [Authentication](authentication.md) | Tickets, CSRF, API tokens, ACLs |
| [API versions](api-versions.md) | Contracts 69 and hot-swap |
| [Clients & examples](clients.md) | Python, Go, Java, Perl, Ansible, Terraform, Pulumi |
| [Seed profiles](seed-profiles.md) | Deterministic cluster fixtures |
| [API surface](api-surface.md) | Routing, handlers, fallbacks |
| [Domains](domains/README.md) | QEMU, LXC, storage, HA, SDN, … |
| [Web UI](web-ui.md) | Interactive console and catalogs |
| [Operations](operations.md) | Migrate, reseed, upgrade, Hub publish |
| [Docker Hub overview](docker-hub-overview.md) | Paste-ready Hub repository description |
| [Kubernetes / Helm](kubernetes.md) | Hub image + Ingress + Let's Encrypt |
| [Security](security.md) | Lab threat model and credentials |
| [Observability](observability.md) | Health endpoints and logging |
| [Troubleshooting](troubleshooting.md) | Common failure modes |
| [FAQ](faq.md) | Short answers |
| [Architecture](architecture.md) | Component boundaries |
| [Compatibility](compatibility.md) | Evidence model and release matrix |
Runnable cookbooks: [`examples/`](../examples/README.md).
Integration suites: [`pulumi-tests/`](../pulumi-tests/README.md).
+4 -1
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](api-surface.md) | [Русский](ru/api-surface.md)
# API surface # API surface
## Request path ## Request path
@@ -33,7 +35,8 @@ not part of the product contract. See the workspace durable-simulator rule.
- Interactive FastAPI docs: `/docs` - Interactive FastAPI docs: `/docs`
- Web UI method inspector: `/` → catalog → method - Web UI method inspector: `/` → catalog → method
- UI APIs: `/ui/api/catalog`, `/ui/api/method`, `/ui/api/compatibility` - UI APIs: `/ui/api/versions`, `/ui/api/catalog`, `/ui/api/method`,
`/ui/api/compatibility`, `/ui/api/contract/apply`, `/ui/api/demo/*`
## Compatibility endpoints ## Compatibility endpoints
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](api-versions.md) | [Русский](ru/api-versions.md)
# API versions (PVE 69) # API versions (PVE 69)
The simulator ships authoritative imported contracts for four Proxmox VE majors. The simulator ships authoritative imported contracts for four Proxmox VE majors.
+3 -1
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](architecture.md) | [Русский](ru/architecture.md)
# Architecture # Architecture
## Goals ## Goals
@@ -30,7 +32,7 @@ flowchart LR
Obs["Logs / Prometheus / OpenTelemetry"] Obs["Logs / Prometheus / OpenTelemetry"]
Client -->|"/api2/json"| API Client -->|"/api2/json"| API
Admin -->|"CLI and /_simulator"| API Admin -->|"CLI, Make/Helm, Web UI /ui/api"| API
Docs -->|"explicit import only"| Importer Docs -->|"explicit import only"| Importer
Importer --> Contract Importer --> Contract
Contract --> DB Contract --> DB
+5 -2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](authentication.md) | [Русский](ru/authentication.md)
# Authentication # Authentication
The simulator implements Proxmox-compatible ticket and API-token authentication The simulator implements Proxmox-compatible ticket and API-token authentication
@@ -54,8 +56,9 @@ beyond its owner.
## Seeded development principals ## Seeded development principals
Loaded by every standard seed profile (unless replaced by UI demo unload Seeded for **every** profile — including `minimal` and after Web UI demo unload.
`minimal`): Unload shrinks guests/nodes/storages; lab principals and tokens are still
inserted by `apply_seed`:
| Principal | Password | Token | Notes | | Principal | Password | Token | Notes |
|---|---|---|---| |---|---|---|---|
+18 -27
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](clients.md) | [Русский](ru/clients.md)
# Clients # Clients
Use the simulator from common automation stacks. Each cookbook aims for the Use the simulator from common automation stacks. Each cookbook aims for the
@@ -13,39 +15,28 @@ same laboratory flow where the tool allows it:
## Connection matrix ## Connection matrix
| Stack | Transport | Notes | Docs | Code | Real Proxmox VE clients talk to **HTTPS `:8006`**. This labs Compose stack
publishes plain **HTTP `:8006`** (same port number). HTTPS belongs on
**Kubernetes Ingress** (cert-manager). Clients that cannot speak HTTP
(proxmoxer) use the optional profile: `docker compose --profile tls`
`https://localhost:8443/` (see [Ports and TLS](configuration.md#ports-and-tls)).
| Stack | Compose transport | Notes | Docs | Code |
|---|---|---|---|---| |---|---|---|---|---|
| Python (proxmoxer) | HTTPS `:8007` | Unmodified library; `verify_ssl=False` for local cert | [guide](examples/python-proxmoxer.md) | [`examples/python`](../examples/python) | | Python (proxmoxer) | HTTPS `:8443` (`--profile tls`) | HTTPS-only library; `verify_ssl=False` for lab cert | [guide](examples/python-proxmoxer.md) | [`examples/python`](../examples/python) |
| Python (requests) | HTTP `:8006` | Raw `/api2/json` | [guide](examples/python-requests.md) | [`examples/python`](../examples/python) | | Python (requests) | HTTP `:8006` | Raw `/api2/json` | [guide](examples/python-requests.md) | [`examples/python`](../examples/python) |
| Go | HTTP `:8006` | stdlib `net/http` | [guide](examples/go.md) | [`examples/go`](../examples/go) | | Go | HTTP `:8006` | stdlib `net/http` | [guide](examples/go.md) | [`examples/go`](../examples/go) |
| Java | HTTP `:8006` | Java 11+ `HttpClient` | [guide](examples/java.md) | [`examples/java`](../examples/java) | | Java | HTTP `:8006` | Java 11+ `HttpClient` | [guide](examples/java.md) | [`examples/java`](../examples/java) |
| Perl | HTTP `:8006` | `HTTP::Tiny` + JSON | [guide](examples/perl.md) | [`examples/perl`](../examples/perl) | | Perl | HTTP `:8006` | `HTTP::Tiny` + JSON | [guide](examples/perl.md) | [`examples/perl`](../examples/perl) |
| Ansible | HTTP `:8006` | `uri` module cookbook | [guide](examples/ansible.md) | [`examples/ansible`](../examples/ansible) | | Ansible | HTTP `:8006` | `uri` module cookbook | [guide](examples/ansible.md) | [`examples/ansible`](../examples/ansible) |
| Terraform | HTTPS `:8007` | Provider + insecure TLS for local gateway | [guide](examples/terraform.md) | [`examples/terraform`](../examples/terraform) | | Terraform | HTTP `:8006` (or TLS `:8443`) | Prefer HTTP; use `insecure` only with `--profile tls` | [guide](examples/terraform.md) | [`examples/terraform`](../examples/terraform) |
| Pulumi | HTTPS `:8007` | Python program against the API | [guide](examples/pulumi.md) | [`examples/pulumi`](../examples/pulumi) | | Pulumi | HTTP `:8006` | `pulumi-proxmoxve` or HTTP cookbooks | [guide](examples/pulumi.md) | [`examples/pulumi`](../examples/pulumi) |
Shared prerequisites: [examples overview](examples/overview.md). On Kubernetes with Ingress + cert-manager, point every client at
`https://<your-host>/` instead.
## Credentials (seed) ## More
| Use | Value | - Cookbooks index: [examples/overview.md](examples/overview.md)
|---|---| - Troubleshooting: [examples/troubleshooting-clients.md](examples/troubleshooting-clients.md)
| User | `root@pam` | - Pulumi full suite: [`pulumi-tests/`](../pulumi-tests/README.md)
| Password | `secret` |
| Token | `root@pam!automation=automation-secret` |
| Default node (`small`) | `pve01` |
## API major
Pin the major before long runs:
- Cold start: `CONTRACT_SNAPSHOT`
- Runtime: Web UI apply or `POST /ui/api/contract/apply?major=N`
Confirm with `GET /api2/json/version`. Coverage is **100%** for declared methods
on majors 69.
## Troubleshooting clients
See [troubleshooting-clients](examples/troubleshooting-clients.md) and the
global [Troubleshooting](troubleshooting.md) guide.
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](compatibility-0.1.0.md) | [Русский](ru/compatibility-0.1.0.md)
# Compatibility report — 0.1.0 # Compatibility report — 0.1.0
This report records evidence for simulator release 0.1.0 against the bundled This report records evidence for simulator release 0.1.0 against the bundled
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](compatibility.md) | [Русский](ru/compatibility.md)
# Compatibility # Compatibility
This document explains how the simulator claims compatibility with Proxmox VE This document explains how the simulator claims compatibility with Proxmox VE
+32 -3
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](configuration.md) | [Русский](ru/configuration.md)
# Configuration # Configuration
Application settings are loaded from the environment (see `.env.example`). Application settings are loaded from the environment (see `.env.example`).
@@ -53,14 +55,41 @@ any accidental gap surfaces as HTTP 501.
| `SEED_LARGE_NODES` | Node count for `large` | | `SEED_LARGE_NODES` | Node count for `large` |
| `SEED_LARGE_RESOURCES` | Guest count for `large` (default 10000) | | `SEED_LARGE_RESOURCES` | Guest count for `large` (default 10000) |
| `TEST_DATABASE_URL` | Integration-test DSN | | `TEST_DATABASE_URL` | Integration-test DSN |
| `PROXMOXER_HOST` / `PROXMOXER_PORT` | Compatibility test client target (`tls-gateway` / `8443` in Compose) | | `PROXMOXER_HOST` / `PROXMOXER_PORT` | Compatibility test client target with `--profile tls` (`tls-gateway` / `8443`) |
## Ports and TLS ## Ports and TLS
### Real Proxmox VE (reference)
On a physical / production PVE node the management API listens on **HTTPS
`:8006`** (`/api2/json/...`). Related management ports (not separate REST APIs):
| Port | Protocol | Role |
|---|---|---|
| `8006` | TCP, HTTPS | Web UI + REST API |
| `3128` | TCP | SPICE proxy (graphical console) |
| `59005999` | TCP (WebSocket) | VNC web console |
| `22` | TCP | SSH / cluster actions |
| `54055412` | UDP | Corosync cluster traffic |
Port **`8007`** is **not** the PVE API — it is the usual Proxmox Backup Server
(PBS) management port. Do not point PVE clients at `:8007` on real hardware.
### Simulator lab endpoints
| Endpoint | Use | | Endpoint | Use |
|---|---| |---|---|
| `http://localhost:8006` | Direct HTTP (curl, browsers, most examples) | | `http://localhost:8006` | Primary client URL — simulator (curl, browsers, requests, Terraform, …) |
| `https://localhost:8007` | TLS gateway for TLS-assuming clients (proxmoxer, etc.) | | `https://localhost:8443` | Optional — `docker compose --profile tls` for proxmoxer-style HTTPS-only clients |
Compose publishes plain **HTTP on host `:8006`** (same port number as real PVE,
which uses HTTPS). The simulator process speaks HTTP on `:8006` inside the Docker
network as well. For Kubernetes, TLS terminates at Ingress (cert-manager). Host
**`:8007` is not used** for the lab API (on real hardware that port is typically
PBS, not PVE).
Optional Compose TLS: `docker compose --profile tls` starts an nginx gateway on
host `:8443` (requires `docker/tls/`). It proxies to `simulator:8006`.
The checked-in certificate under `docker/tls/` is disposable development The checked-in certificate under `docker/tls/` is disposable development
material. Never reuse it outside local labs. See [Security](security.md). material. Never reuse it outside local labs. See [Security](security.md).
+40
View File
@@ -0,0 +1,40 @@
# Docker Hub overview (paste into Hub)
Copy the block below into the **Full description** of
[`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator)
so Hub wording matches GitHub (stateful simulator — not a thin mock).
---
**proxmox-api-simulator** — stateful asynchronous Proxmox VE API simulator for
labs and CI. PostgreSQL-backed mutations, durable UPIDs, official API contracts
for PVE 69, and the same `/api2/json` surface clients already speak.
**Laboratory / CI only.** Default credentials and signing keys are intentional
lab defaults. Do **not** expose this image to the public Internet without
replacing secrets and adding your own network controls.
### Quick start
```bash
# from a git checkout (needs docker-compose.release.yml + docker/tls/)
docker compose -f docker-compose.release.yml up -d
docker compose -f docker-compose.release.yml run --rm --entrypoint python \
simulator -m app.simulation.seed_cli
curl -sS http://localhost:8006/health/ready
curl -sS http://localhost:8006/api2/json/version
```
- HTTP API + Web UI: `http://localhost:8006/`
- Optional HTTPS for proxmoxer: `docker compose --profile tls``https://localhost:8443/`
- Seeded admin: `root@pam` / `secret`
- Source & docs: https://github.com/sergeyantropoff/proxmox-api-simulator
- Helm chart: `helm/proxmox-api-simulator` in the same repository
### Tags
| Tag | Meaning |
|---|---|
| `0.1.0`, `…` | Immutable release from `pyproject.toml` / `make release` |
| `latest` | Most recent `make release` (when `PUSH_LATEST=1`) |
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](README.md) | [Русский](../ru/domains/README.md)
# Domain guides # Domain guides
These pages summarize durable semantics by area. For exhaustive method lists, These pages summarize durable semantics by area. For exhaustive method lists,
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](access.md) | [Русский](../ru/domains/access.md)
# Access # Access
Durable identity and authorization: users, groups, roles, ACL entries, realms, Durable identity and authorization: users, groups, roles, ACL entries, realms,
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](ceph.md) | [Русский](../ru/domains/ceph.md)
# Ceph # Ceph
Ceph-related API paths persist simulated cluster, pool, OSD, and monitor state. Ceph-related API paths persist simulated cluster, pool, OSD, and monitor state.
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](cluster-extras.md) | [Русский](../ru/domains/cluster-extras.md)
# Cluster extras # Cluster extras
Additional cluster-scoped domains with durable handlers: Additional cluster-scoped domains with durable handlers:
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](core-cluster.md) | [Русский](../ru/domains/core-cluster.md)
# Core & cluster # Core & cluster
## Version ## Version
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](firewall.md) | [Русский](../ru/domains/firewall.md)
# Firewall # Firewall
Cluster, node, and guest firewall configuration — rules, aliases, IP sets, Cluster, node, and guest firewall configuration — rules, aliases, IP sets,
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](ha.md) | [Русский](../ru/domains/ha.md)
# HA # HA
High-availability groups, resources, status, and rules persist in cluster High-availability groups, resources, status, and rules persist in cluster
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](lxc.md) | [Русский](../ru/domains/lxc.md)
# LXC # LXC
Container APIs mirror the QEMU lifecycle patterns where the contract declares Container APIs mirror the QEMU lifecycle patterns where the contract declares
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](pools.md) | [Русский](../ru/domains/pools.md)
# Pools # Pools
Pool CRUD and resource membership are fully covered and durable. The `medium` Pool CRUD and resource membership are fully covered and durable. The `medium`
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](qemu.md) | [Русский](../ru/domains/qemu.md)
# QEMU # QEMU
Full contract surface for QEMU guests on the active major, including: Full contract surface for QEMU guests on the active major, including:
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](sdn.md) | [Русский](../ru/domains/sdn.md)
# SDN # SDN
Software-defined networking handlers cover declared zones, VNets, subnets, Software-defined networking handlers cover declared zones, VNets, subnets,
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](storage-backup.md) | [Русский](../ru/domains/storage-backup.md)
# Storage & backup # Storage & backup
## Storage ## Storage
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](tasks.md) | [Русский](../ru/domains/tasks.md)
# Tasks # Tasks
Long-running operations return a Proxmox-style **UPID**. Task rows, events, Long-running operations return a Proxmox-style **UPID**. Task rows, events,
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](ansible.md) | [Русский](../ru/examples/ansible.md)
# Ansible # Ansible
Playbook uses the `uri` module against HTTP `:8006` with token auth, then Playbook uses the `uri` module against HTTP `:8006` with token auth, then
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](go.md) | [Русский](../ru/examples/go.md)
# Go # Go
Uses the Go standard library against `http://localhost:8006` with API-token Uses the Go standard library against `http://localhost:8006` with API-token
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](java.md) | [Русский](../ru/examples/java.md)
# Java # Java
Java 11+ `HttpClient` cookbook using API-token auth against `:8006`. Java 11+ `HttpClient` cookbook using API-token auth against `:8006`.
+3 -1
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](overview.md) | [Русский](../ru/examples/overview.md)
# Client examples overview # Client examples overview
## Bring-up checklist ## Bring-up checklist
@@ -20,7 +22,7 @@ curl -s -X POST 'http://localhost:8006/ui/api/contract/apply?major=8'
| URL | When | | URL | When |
|---|---| |---|---|
| `http://localhost:8006` | curl, Go, Java, Perl, Ansible, requests | | `http://localhost:8006` | curl, Go, Java, Perl, Ansible, requests |
| `https://localhost:8007` | proxmoxer, many Terraform/Pulumi TLS clients | | `http://localhost:8006` | proxmoxer, many Terraform/Pulumi TLS clients |
## Auth quick reference ## Auth quick reference
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](perl.md) | [Русский](../ru/examples/perl.md)
# Perl # Perl
`HTTP::Tiny` + JSON cookbook with API-token auth. `HTTP::Tiny` + JSON cookbook with API-token auth.
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](pulumi.md) | [Русский](../ru/examples/pulumi.md)
# Pulumi # Pulumi
Python Pulumi program that drives the simulator over HTTPS using token auth via Python Pulumi program that drives the simulator over HTTPS using token auth via
+3 -1
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](python-proxmoxer.md) | [Русский](../ru/examples/python-proxmoxer.md)
# Python — proxmoxer # Python — proxmoxer
Canonical library path against the HTTPS gateway. Canonical library path against the HTTPS gateway.
@@ -11,7 +13,7 @@ python examples/python/proxmoxer_cookbook.py
``` ```
Environment overrides: `PVE_HOST` (default `localhost`), `PVE_PORT` (default Environment overrides: `PVE_HOST` (default `localhost`), `PVE_PORT` (default
`8007`), `PVE_USER`, `PVE_PASSWORD`, or token via `PVE_TOKEN_NAME` / `8006`), `PVE_USER`, `PVE_PASSWORD`, or token via `PVE_TOKEN_NAME` /
`PVE_TOKEN_VALUE`. `PVE_TOKEN_VALUE`.
## Notes ## Notes
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](python-requests.md) | [Русский](../ru/examples/python-requests.md)
# Python — requests # Python — requests
Raw HTTP against `:8006` without proxmoxer. Raw HTTP against `:8006` without proxmoxer.
+3 -1
View File
@@ -1,7 +1,9 @@
**Language / Язык:** [English](terraform.md) | [Русский](../ru/examples/terraform.md)
# Terraform # Terraform
Example uses a Proxmox provider pointed at the local HTTPS gateway Example uses a Proxmox provider pointed at the local HTTPS gateway
(`https://localhost:8007`) with `insecure = true` for the development (`http://localhost:8006`) with `insecure = true` for the development
certificate. certificate.
```bash ```bash
+3 -1
View File
@@ -1,8 +1,10 @@
**Language / Язык:** [English](troubleshooting-clients.md) | [Русский](../ru/examples/troubleshooting-clients.md)
# Troubleshooting clients # Troubleshooting clients
| Symptom | Fix | | Symptom | Fix |
|---|---| |---|---|
| TLS certificate errors | Use `:8007` with verify disabled **only** locally, or use HTTP `:8006` | | TLS certificate errors | Compose default is `http://localhost:8006` (no TLS). For proxmoxer use `docker compose --profile tls` and `https://localhost:8443` with verify disabled **only** locally (`curl -sk`, `verify_ssl=False`, `insecure=true`). On Kubernetes use your Ingress hostname with cert-manager TLS. |
| CSRF failure | Send `CSRFPreventionToken` with ticket mutations; prefer token auth in scripts | | CSRF failure | Send `CSRFPreventionToken` with ticket mutations; prefer token auth in scripts |
| Node not found | `small` seed uses `pve01` | | Node not found | `small` seed uses `pve01` |
| 403 on power | You may be using `auditor@pve` / readonly token — switch to root or operator | | 403 on power | You may be using `auditor@pve` / readonly token — switch to root or operator |
+18 -2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](faq.md) | [Русский](ru/faq.md)
# FAQ # FAQ
## Is this a real Proxmox hypervisor? ## Is this a real Proxmox hypervisor?
@@ -14,7 +16,9 @@ See [API versions](api-versions.md) and [Compatibility](compatibility.md).
## Can I use this in CI for Terraform / Ansible / custom clients? ## Can I use this in CI for Terraform / Ansible / custom clients?
Yes. That is a primary use case. Pin the API major, seed a profile, and point Yes. That is a primary use case. Pin the API major, seed a profile, and point
clients at HTTP `:8006` or HTTPS `:8007`. See [Clients](clients.md). clients at **HTTP `:8006`** (Compose) or your Ingress **HTTPS** hostname on
Kubernetes. See [Clients](clients.md). For the
Pulumi surface suite see [`pulumi-tests/`](../pulumi-tests/README.md).
## Why do some OpenID / LDAP / ACME / Ceph calls “succeed” without remotes? ## Why do some OpenID / LDAP / ACME / Ceph calls “succeed” without remotes?
@@ -40,4 +44,16 @@ Hub image. Ingress + cert-manager Let's Encrypt is supported — see
## Which node name does the small seed use? ## Which node name does the small seed use?
`pve01`. `pve01`. Profiles `medium` and `ha-demo` use **`pve1` / `pve2` / `pve3`**.
## What ports does real Proxmox VE use vs this simulator?
Real PVE serves the Web UI and REST API on **HTTPS `:8006`** only. Related
management ports include SPICE `:3128`, VNC `:59005999`, SSH `:22`, and
Corosync UDP `:54055412`. Port `:8007` on real hardware is typically
**Proxmox Backup Server**, not PVE.
This lab publishes plain **HTTP `:8006`** in Compose (same port number as real
PVE). HTTPS belongs on Kubernetes Ingress. Optional proxmoxer TLS:
`docker compose --profile tls` on `:8443`. Host `:8007` is **not** used. Details:
[Ports and TLS](configuration.md#ports-and-tls).
+22 -12
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](getting-started.md) | [Русский](ru/getting-started.md)
# Getting started # Getting started
Bring up a local laboratory cluster, authenticate, and exercise a first Bring up a local laboratory cluster, authenticate, and exercise a first
@@ -17,15 +19,18 @@ Python toolchain for day-to-day use.
|---|---| |---|---|
| [Published image](#1a-published-image-docker-hub) | Fastest lab using `inecs/proxmox-api-simulator` | | [Published image](#1a-published-image-docker-hub) | Fastest lab using `inecs/proxmox-api-simulator` |
| [Helm / Kubernetes](kubernetes.md) | Cluster install with Ingress + Let's Encrypt | | [Helm / Kubernetes](kubernetes.md) | Cluster install with Ingress + Let's Encrypt |
| [Development checkout](#1b-development-checkout) | Contribute / bind-mount source / HTTPS gateway on `:8007` | | [Development checkout](#1b-development-checkout) | Contribute / bind-mount source / HTTP API on `:8006` |
## 1a. Published image (Docker Hub) ## 1a. Published image (Docker Hub)
Uses [`docker-compose.release.yml`](../docker-compose.release.yml) — PostgreSQL + Uses [`docker-compose.release.yml`](../docker-compose.release.yml) — PostgreSQL +
runtime simulator from Hub. No source build required. runtime simulator from Hub. No source build required.
> Laboratory / CI only — rotate `TICKET_SIGNING_KEY` and the DB password before
> any shared or networked demo. See [SECURITY.md](../SECURITY.md).
```bash ```bash
# from this repository, or download docker-compose.release.yml alone # from this repository (compose file + docker/tls/)
docker compose -f docker-compose.release.yml pull docker compose -f docker-compose.release.yml pull
docker compose -f docker-compose.release.yml up -d docker compose -f docker-compose.release.yml up -d
docker compose -f docker-compose.release.yml run --rm --entrypoint python \ docker compose -f docker-compose.release.yml run --rm --entrypoint python \
@@ -47,7 +52,7 @@ make release-seed PROFILE=small
| Host port | Service | | Host port | Service |
|---|---| |---|---|
| `8006` | HTTP API + Web UI | | `8006` | HTTP API + Web UI (same port as real PVE; real PVE uses HTTPS) |
| `5432` | PostgreSQL (localhost only) | | `5432` | PostgreSQL (localhost only) |
Migrations run automatically via the `migrate` one-shot service. Migrations run automatically via the `migrate` one-shot service.
@@ -65,17 +70,22 @@ Services:
| Host port | Service | | Host port | Service |
|---|---| |---|---|
| `8006` | HTTP API + Web UI | | `8006` | HTTP API + Web UI (same port as real PVE; real PVE uses HTTPS) |
| `8007` | HTTPS nginx gateway → simulator |
| `5432` | PostgreSQL (localhost only) | | `5432` | PostgreSQL (localhost only) |
On real Proxmox VE the REST API is **only** `https://<host>:8006/api2/json/...`.
The lab publishes plain **HTTP** on host `:8006`; see
[Ports and TLS](configuration.md#ports-and-tls). Optional HTTPS for proxmoxer:
`docker compose --profile tls``https://localhost:8443/`. Host `:8007` is
**not** used (on hardware it is typically PBS, not PVE API).
Migrations apply automatically before the simulator becomes ready. Migrations apply automatically before the simulator becomes ready.
## 2. Wait until ready ## 2. Wait until ready
```bash ```bash
curl http://localhost:8006/health/live curl -sS http://localhost:8006/health/live
curl http://localhost:8006/health/ready curl -sS http://localhost:8006/health/ready
``` ```
`/health/ready` returns HTTP 503 until PostgreSQL is reachable **and** the `/health/ready` returns HTTP 503 until PostgreSQL is reachable **and** the
@@ -94,7 +104,7 @@ local storages, and the standard development principals. See
## 4. Check the API version ## 4. Check the API version
```bash ```bash
curl -s http://localhost:8006/api2/json/version | jq . curl -sS http://localhost:8006/api2/json/version | jq .
``` ```
The cold-start contract defaults to the bundled PVE **9.2.3** snapshot in Docker The cold-start contract defaults to the bundled PVE **9.2.3** snapshot in Docker
@@ -104,7 +114,7 @@ Compose. Switch majors 69 from the Web UI or
## 5. Authenticate ## 5. Authenticate
```bash ```bash
curl -s -X POST \ curl -sS -X POST \
-d 'username=root@pam&password=secret' \ -d 'username=root@pam&password=secret' \
http://localhost:8006/api2/json/access/ticket | jq . http://localhost:8006/api2/json/access/ticket | jq .
``` ```
@@ -120,10 +130,10 @@ Details: [Authentication](authentication.md).
```bash ```bash
# replace TICKET / CSRF from the previous response # replace TICKET / CSRF from the previous response
curl -s -H "Cookie: PVEAuthCookie=$TICKET" \ curl -sS -H "Cookie: PVEAuthCookie=$TICKET" \
http://localhost:8006/api2/json/nodes/pve01/qemu | jq . http://localhost:8006/api2/json/nodes/pve01/qemu | jq .
curl -s -X POST \ curl -sS -X POST \
-H "Cookie: PVEAuthCookie=$TICKET" \ -H "Cookie: PVEAuthCookie=$TICKET" \
-H "CSRFPreventionToken: $CSRF" \ -H "CSRFPreventionToken: $CSRF" \
http://localhost:8006/api2/json/nodes/pve01/qemu/100/status/start | jq . http://localhost:8006/api2/json/nodes/pve01/qemu/100/status/start | jq .
@@ -132,7 +142,7 @@ curl -s -X POST \
Async operations return a UPID string. Poll until the task finishes: Async operations return a UPID string. Poll until the task finishes:
```bash ```bash
curl -s -H "Cookie: PVEAuthCookie=$TICKET" \ curl -sS -H "Cookie: PVEAuthCookie=$TICKET" \
"http://localhost:8006/api2/json/nodes/pve01/tasks/${UPID}/status" | jq . "http://localhost:8006/api2/json/nodes/pve01/tasks/${UPID}/status" | jq .
``` ```
+36 -6
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](kubernetes.md) | [Русский](ru/kubernetes.md)
# Kubernetes / Helm # Kubernetes / Helm
Deploy the published Docker Hub runtime image with the chart in Deploy the published Docker Hub runtime image with the chart in
@@ -5,6 +7,18 @@ Deploy the published Docker Hub runtime image with the chart in
Image: [`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator) Image: [`inecs/proxmox-api-simulator`](https://hub.docker.com/r/inecs/proxmox-api-simulator)
> **Laboratory / CI only.** Chart defaults include weak placeholder secrets.
> Always override `secret.ticketSigningKey` and `postgresql.auth.password`
> before any shared or Internet-facing install. See [SECURITY.md](../SECURITY.md).
## Transport note (Compose vs Helm)
| Path | Client URL |
|---|---|
| Local Compose (`docker-compose*.yml`) | **HTTP** `:8006` (simulator process) |
| Helm Service / `kubectl port-forward` | **HTTP** `:8006` (simulator process; TLS terminates at Ingress if enabled) |
| Helm Ingress + cert-manager | **HTTPS** on your hostname |
## Prerequisites ## Prerequisites
- Kubernetes 1.27+ (or comparable) - Kubernetes 1.27+ (or comparable)
@@ -28,8 +42,8 @@ helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
-n proxmox-sim --create-namespace \ -n proxmox-sim --create-namespace \
-f ./helm/proxmox-api-simulator/values-ingress-example.yaml \ -f ./helm/proxmox-api-simulator/values-ingress-example.yaml \
--set certManager.email=you@example.com \ --set certManager.email=you@example.com \
--set ingress.hosts[0].host=pve-sim.example.com \ --set 'ingress.hosts[0].host=pve-sim.example.com' \
--set ingress.tls[0].hosts[0]=pve-sim.example.com \ --set 'ingress.tls[0].hosts[0]=pve-sim.example.com' \
--set secret.ticketSigningKey="$(openssl rand -hex 32)" \ --set secret.ticketSigningKey="$(openssl rand -hex 32)" \
--set postgresql.auth.password="$(openssl rand -hex 16)" --set postgresql.auth.password="$(openssl rand -hex 16)"
``` ```
@@ -68,9 +82,10 @@ helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
-f ./helm/proxmox-api-simulator/values-ingress-example.yaml \ -f ./helm/proxmox-api-simulator/values-ingress-example.yaml \
--set certManager.email=you@example.com \ --set certManager.email=you@example.com \
--set certManager.useStaging=true \ --set certManager.useStaging=true \
--set ingress.hosts[0].host=pve-sim.example.com \ --set 'ingress.hosts[0].host=pve-sim.example.com' \
--set ingress.tls[0].hosts[0]=pve-sim.example.com \ --set 'ingress.tls[0].hosts[0]=pve-sim.example.com' \
--set secret.ticketSigningKey="$(openssl rand -hex 32)" --set secret.ticketSigningKey="$(openssl rand -hex 32)" \
--set postgresql.auth.password="$(openssl rand -hex 16)"
``` ```
Browsers will not trust the staging CA — use `curl -k` while testing. Flip Browsers will not trust the staging CA — use `curl -k` while testing. Flip
@@ -88,7 +103,8 @@ helm upgrade --install pve-sim ./helm/proxmox-api-simulator \
kubectl -n proxmox-sim port-forward svc/pve-sim-proxmox-api-simulator 8006:8006 kubectl -n proxmox-sim port-forward svc/pve-sim-proxmox-api-simulator 8006:8006
``` ```
Open http://127.0.0.1:8006/ Open http://127.0.0.1:8006/ (plain HTTP — the chart does not ship the Compose
TLS gateway; use Ingress for HTTPS).
## External PostgreSQL ## External PostgreSQL
@@ -131,6 +147,20 @@ certManager:
issuerName: your-existing-issuer issuerName: your-existing-issuer
``` ```
## Local chart validation
From the repository root (requires Helm 3.14+):
```bash
make helm-lint
make helm-template
```
`helm lint` should report 0 failures (an informational note that Chart.yaml has no
`icon` is expected). `helm template` renders Deployment (with a migrate
initContainer by default), Service, Secret, PostgreSQL StatefulSet, optional
standalone migrate Job (`migrate.asJob`), seed Job, Ingress, and ClusterIssuers.
## Operations ## Operations
```bash ```bash
+2
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](observability.md) | [Русский](ru/observability.md)
# Observability # Observability
## Health ## Health
+18 -5
View File
@@ -1,3 +1,5 @@
**Language / Язык:** [English](operations.md) | [Русский](ru/operations.md)
# Operations # Operations
## Day-2 commands ## Day-2 commands
@@ -82,6 +84,14 @@ Published tags:
- `inecs/proxmox-api-simulator:<version>` - `inecs/proxmox-api-simulator:<version>`
- `inecs/proxmox-api-simulator:latest` (unless `PUSH_LATEST=0`) - `inecs/proxmox-api-simulator:latest` (unless `PUSH_LATEST=0`)
After publishing, paste
[Docker Hub overview](docker-hub-overview.md) into the Hub repository
description if it drifted, and keep GitHub “About” wording aligned
(“stateful Proxmox VE API simulator” — not a thin mock).
CI on GitHub Actions runs `make ci` plus Compose/Helm validation on every push
and PR to `main` (see `.github/workflows/ci.yml`).
## Quick start with the published compose file ## Quick start with the published compose file
[`docker-compose.release.yml`](../docker-compose.release.yml) pulls the Hub [`docker-compose.release.yml`](../docker-compose.release.yml) pulls the Hub
@@ -92,7 +102,7 @@ docker compose -f docker-compose.release.yml up -d
docker compose -f docker-compose.release.yml run --rm --entrypoint python \ docker compose -f docker-compose.release.yml run --rm --entrypoint python \
simulator -m app.simulation.seed_cli simulator -m app.simulation.seed_cli
curl http://localhost:8006/health/ready curl -sS http://localhost:8006/health/ready
open http://localhost:8006/ open http://localhost:8006/
``` ```
@@ -110,12 +120,14 @@ Useful overrides:
|---|---|---| |---|---|---|
| `DOCKER_IMAGE` | `inecs/proxmox-api-simulator` | Image repository | | `DOCKER_IMAGE` | `inecs/proxmox-api-simulator` | Image repository |
| `IMAGE_TAG` | `latest` | Tag to pull | | `IMAGE_TAG` | `latest` | Tag to pull |
| `SIMULATOR_PORT` | `8006` | Host HTTP port | | `SIMULATOR_PORT` | `8006` | Host HTTP port (simulator) |
| `TICKET_SIGNING_KEY` | lab default | Change outside toy labs | | `TICKET_SIGNING_KEY` | lab default | Change outside toy labs |
| `POSTGRES_PASSWORD` | `proxmox` | DB password | | `POSTGRES_PASSWORD` | `proxmox` | DB password |
This stack is HTTP-only. The development Compose file still provides the local Both development and release Compose publish **HTTP `:8006`** on the host
HTTPS gateway on `:8007` for TLS-assuming clients. (same port as real PVE, which uses HTTPS). Optional HTTPS for proxmoxer-style
clients: `docker compose --profile tls` on host `:8443`. See
[Ports and TLS](configuration.md#ports-and-tls).
For Kubernetes with public TLS (cert-manager / Let's Encrypt), use the Helm For Kubernetes with public TLS (cert-manager / Let's Encrypt), use the Helm
chart — see [Kubernetes / Helm](kubernetes.md). chart — see [Kubernetes / Helm](kubernetes.md).
@@ -126,7 +138,8 @@ chart — see [Kubernetes / Helm](kubernetes.md).
2. Run migrations. 2. Run migrations.
3. Confirm `/health/ready`. 3. Confirm `/health/ready`.
4. Re-check `/admin/compatibility` and `/api2/json/version`. 4. Re-check `/admin/compatibility` and `/api2/json/version`.
5. Re-run `make test-compatibility` if you validate external clients in CI. 5. Re-run `make test-compatibility` if you validate external clients in CI
(seeds the **medium** profile — `pve1`/`pve2`/`pve3` — for migration smoke).
## Resetting a lab ## Resetting a lab
+30
View File
@@ -0,0 +1,30 @@
**Language / Язык:** [English](../README.md) | [Русский](README.md)
# Документация
Руководства по симулятору Proxmox VE API. Переключайте язык с помощью заголовка на
каждой странице. Русские версии находятся в каталоге [`ru/`](README.md).
| Руководство | Описание |
|---|---|
| [Быстрый старт](getting-started.md) | Первая успешная лабораторная сессия |
| [Конфигурация](configuration.md) | Переменные окружения и Compose |
| [Аутентификация](authentication.md) | Тикеты, CSRF, API-токены, ACL |
| [Версии API](api-versions.md) | Контракты 6–9 и горячая замена |
| [Клиенты и примеры](clients.md) | Python, Go, Java, Perl, Ansible, Terraform, Pulumi |
| [Профили seed](seed-profiles.md) | Детерминированные фикстуры кластера |
| [Поверхность API](api-surface.md) | Маршрутизация, обработчики, fallback |
| [Домены](domains/README.md) | QEMU, LXC, storage, HA, SDN, … |
| [Web UI](web-ui.md) | Интерактивная консоль и каталоги |
| [Эксплуатация](operations.md) | Миграция, reseed, обновление, публикация в Hub |
| [Обзор Docker Hub](../docker-hub-overview.md) | Готовый текст описания репозитория Hub (EN) |
| [Kubernetes / Helm](kubernetes.md) | Образ Hub + Ingress + Let's Encrypt |
| [Безопасность](security.md) | Модель угроз лаборатории и учётные данные |
| [Наблюдаемость](observability.md) | Эндпоинты health и логирование |
| [Устранение неполадок](troubleshooting.md) | Типичные сбои |
| [FAQ](faq.md) | Краткие ответы |
| [Архитектура](architecture.md) | Границы компонентов |
| [Совместимость](compatibility.md) | Модель evidence и матрица релизов |
Исполняемые cookbook'и: [`examples/`](../../examples/README.ru.md).
Интеграционные наборы: [`pulumi-tests/`](../../pulumi-tests/README.ru.md).
+86
View File
@@ -0,0 +1,86 @@
**Language / Язык:** [English](../api-surface.md) | [Русский](api-surface.md)
# Поверхность API
## Путь запроса
1. Middleware назначает или пробрасывает request ID.
2. Активный снимок контракта выбирает объявленные пути и схемы.
3. Аутентификация разрешает принципала (тикет или API-токен).
4. Проверки ACL / привилегий выполняются до раскрытия или изменения ресурсов.
5. Path, query и body валидируются по схемам, производным от контракта.
6. Семантический обработчик выполняется против состояния в PostgreSQL.
7. Долгие операции создают durable-задачу (+ lock при необходимости) и возвращают UPID.
8. Ответы используют Proxmox-конверт под `/api2/json` или `/api2/extjs`.
## Два рендерера
Каждый метод контракта регистрируется под обоими:
- `/api2/json/...`
- `/api2/extjs/...`
Клиенты и Web UI обычно используют JSON-рендерер.
## Обработчики vs контракты
- **Declared** — присутствует во импортированном снимке API Viewer для мажора.
- **Implemented** — для этого verb + path зарегистрирован семантический обработчик.
- Мажорные версии **69** имеют **100%** implemented-покрытие для объявленных методов.
Обработчики должны сохранять эффекты create/update/delete. Пустые no-op мутации не
входят в продуктовый контракт. См. workspace durable-simulator rule.
## OpenAPI и исследование
- Интерактивная документация FastAPI: `/docs`
- Инспектор методов Web UI: `/` → catalog → method
- UI API: `/ui/api/versions`, `/ui/api/catalog`, `/ui/api/method`,
`/ui/api/compatibility`, `/ui/api/contract/apply`, `/ui/api/demo/*`
## Эндпоинты совместимости
| Путь | Формат |
|---|---|
| `/admin/compatibility` | JSON |
| `/admin/compatibility.md` | Markdown |
| `/admin/compatibility.html` | HTML |
Отчёты следуют активному runtime-контракту после горячей замены.
## Задачи (UPID)
Асинхронная работа (power гостя, clone, migrate, многие delete, backup, …)
возвращает UPID. Опрашивайте:
```text
GET /nodes/{node}/tasks/{upid}/status
GET /nodes/{node}/tasks/{upid}/log
```
Workers забирают задачи через `FOR UPDATE SKIP LOCKED`, продлевают аренды и
восстанавливаются после перезапуска процесса. HTTP 200 на запрос мутации означает
«принято», а не «гость уже в финальном состоянии».
## Ошибки (типичные)
| Статус | Типичная причина |
|---|---|
| 401 | Отсутствует/невалидный тикет или токен |
| 403 | Отказ ACL или отсутствует CSRF при мутации по тикету |
| 409 | Конфликт VMID, недопустимый переход состояния, contention lock |
| 501 | Обработчик отсутствует (не должно появляться для объявленных методов на 6–9) |
| 503 | Сбой готовности (database / migrations) |
## Импорт контрактов
```bash
make shell
proxmox-api-contract validate path/to/source.json
proxmox-api-contract --store contracts import --file path/to/source.json --version 9.2.3
proxmox-api-contract --store contracts list
proxmox-api-contract diff old.json new.json --format markdown
```
Удалённый импорт требует HTTPS, allowlist официальных хостов, лимиты
size/redirect/timeout и неизменяемые ревизии с checksum.
+79
View File
@@ -0,0 +1,79 @@
**Language / Язык:** [English](../api-versions.md) | [Русский](api-versions.md)
# Версии API (PVE 69)
Симулятор поставляет авторитетные импортированные контракты для четырёх мажорных
версий Proxmox VE. Покрытие реестра обработчиков **100% проверено** для каждой:
| Мажор | Исходная версия | Объявленных методов | Покрытие обработчиками |
|---|---|---:|---:|
| 6 | 6.4-15 | 504 | 100% |
| 7 | 7.4-16 | 540 | 100% |
| 8 | 8.4.5 | 605 | 100% |
| 9 | 9.2.3 | 675 | 100% |
Более старые мажорные версии переиспользуют текущие семантические обработчики плюс
синонимы путей, зарегистрированные в `app/handlers/legacy_aliases.py` (например,
исторические написания путей Ceph и backup).
## Холодный старт
Задайте `CONTRACT_SNAPSHOT` путь к нормализованному снимку. Docker Compose по
умолчанию закрепляет встроенную ревизию PVE **9.2.3**.
`GET /api2/json/version` возвращает поля, производные от `source_version` **активного**
снимка.
## Горячая замена (runtime)
Просмотрите любой мажор в каталоге Web UI, затем **Apply as runtime**, или вызовите:
```http
POST /ui/api/contract/apply?major=7
```
Эффекты:
- Маршруты `/api2/json` и `/api2/extjs` в памяти заменяются под блокировкой
приложения.
- `/version`, OpenAPI, метаданные реализации и состояние совместимости обновляются
для нового мажора.
- Изменение **локально для процесса** и **не сохраняется**.
- Перезапуск восстанавливает `CONTRACT_SNAPSHOT`.
Просмотр каталога (`GET /ui/api/catalog?major=N`) **сам по себе** не меняет
runtime; меняет только apply.
### Рекомендации для клиентов
- Явно закрепляйте мажор в CI (env холодного старта **или** apply + проверка
`/version` перед набором тестов).
- Горячая замена на лету может инвалидировать предположения клиента о схемах и
путях — избегайте во время длинных прогонов Terraform/Ansible, если прогон не
владеет переключением.
- После apply перепроверьте `/admin/compatibility` для активного runtime.
## Режимы fallback
`CONTRACT_FALLBACK` управляет поведением для необъявленных обработчиков:
| Значение | Поведение |
|---|---|
| `error` (по умолчанию) | HTTP 501 с явным сообщением в стиле pending-handler |
| `schema-default` | Синтез возвращаемого значения из схемы контракта |
| `fixture` | Только fixture-данные, встроенные в контракт метода |
При полном покрытии обработчиков активного контракта объявленные методы не должны
попадать в fallback. Оставляйте `error`, чтобы регрессии оставались видимыми.
## Evidence vs реестр
**Покрытие реестра** означает, что у каждого объявленного метода зарегистрирован
семантический обработчик (нет систематического 501 для этого контракта).
**Verified** в смысле этого проекта — мажорные версии прогоняются через наборы
совместимости и автоматизацию на наличие обработчиков для 6–9. Многомерный evidence
JSON может со временем расширяться для более глубоких edge-case заявлений;
предпочитайте живой `/admin/compatibility`, когда процесс запущен.
См. [Совместимость](compatibility.md).
+240
View File
@@ -0,0 +1,240 @@
**Language / Язык:** [English](../architecture.md) | [Русский](architecture.md)
# Архитектура
## Цели
`proxmox-api-simulator` — stateful асинхронный эмулятор Proxmox VE API. Главная
цель проектирования — измеримая совместимость с контрактом: маршруты, валидация,
аутентификация, права доступа, формы ответов, переходы состояния и персистентные
долгоживущие задачи проверяются независимо, а не объявляются «универсально
совместимыми». В комплекте majors **69** поставляются с **100%** регистрацией
семантических обработчиков для каждого объявленного метода контракта и поддержкой
горячей замены между этими majors во время работы.
Для обычной работы симулятору не нужна живая установка Proxmox. Официальные
артефакты API и санитизированные наблюдения импортируются заранее и хранятся как
версионируемые снимки.
## Контекст системы
```mermaid
flowchart LR
Client["API clients<br/>proxmoxer / Terraform / Ansible"]
Admin["Simulator operator"]
Docs["Official Proxmox API Viewer"]
API["FastAPI application"]
Importer["Contract importer and CLI"]
Contract["Versioned API contract"]
Engine["Simulation engine"]
Worker["Persistent task workers"]
DB[(PostgreSQL)]
Obs["Logs / Prometheus / OpenTelemetry"]
Client -->|"/api2/json"| API
Admin -->|"CLI, Make/Helm, Web UI /ui/api"| API
Docs -->|"explicit import only"| Importer
Importer --> Contract
Contract --> DB
API --> Contract
API --> Engine
Engine --> DB
Engine --> Worker
Worker --> DB
API --> Obs
Worker --> Obs
```
## Архитектура компонентов
```mermaid
flowchart TB
subgraph ContractPlane["API contract plane"]
Sources["Remote, local, and recorded sources"] --> Parse["Source adapters and parser"]
Parse --> Normalize["Version-independent normalized model"]
Normalize --> Validate["Validation, checksums, manifests"]
Validate --> Registry["Contract registry"]
Registry --> Diff["Semantic version diff"]
Registry --> Routes["Dynamic route and schema factory"]
Registry --> Reports["Compatibility reports"]
end
subgraph RequestPlane["Request plane"]
Middleware["Request ID, logging, metrics"] --> Auth["Ticket or API-token authentication"]
Auth --> Permission["ACL and privilege evaluation"]
Permission --> Input["Contract-driven request validation"]
Input --> Handler["Semantic handler registry"]
Handler --> Render["Proxmox response and error renderer"]
end
subgraph SimulationPlane["Simulation plane"]
Handler --> Services["Node, QEMU, LXC, storage services"]
Services --> State["State machines and resource locks"]
Services --> Tasks["Transactional persistent tasks"]
Tasks --> Workers["asyncio workers with PostgreSQL leases"]
Faults["Scenarios, faults, virtual clock"] --> Services
end
Routes --> Input
Registry --> Permission
State --> PG[(PostgreSQL)]
Workers --> PG
Auth --> PG
```
## Границы и направление зависимостей
Плоскость контракта владеет объявленными фактами API. Она импортирует
исходные артефакты, сохраняет неизвестные поля источника, формирует
детерминированный нормализованный JSON и предоставляет неизменяемые
версионируемые контракты. Она не знает о состоянии ВМ и не выполняет операции.
Плоскость симуляции владеет изменяемым состоянием кластера и семантикой
операций. Она использует доменные модели и репозитории, не зависящие от FastAPI
и структур контракта, специфичных для источника. PostgreSQL — система записи
для ресурсов, состояния безопасности, блокировок, сценариев и задач.
Долговечные задачи подтверждаются только после совместной фиксации строки задачи,
события, ключа идемпотентности и опциональной блокировки ресурса. Воркеры
захватывают задачи через `SKIP LOCKED`, продлевают аренды в реальном времени,
сохраняют прогресс и append-only логи/события и позволяют повторно захватить
просроченную работу после сбоя процесса. Lifespan владеет ограниченным набором
asyncio-воркеров и ждёт упорядоченного завершения; PostgreSQL остаётся очередью
и источником истины между репликами.
Длительности симуляции используют внедрённые часы: реальные, ускоренные или
продвигаемые вручную. Операции ВМ — явные переходы конечного автомата, а
засеянные правила сбоев оцениваются детерминированно. Аренды воркеров намеренно
исключены из виртуального времени: они используют wall time PostgreSQL и
monotonic sleep процесса, чтобы приостановленный или ускоренный сценарий не
нарушил безопасность распределённых воркеров.
Секреты аутентификации хранятся как salted scrypt-хеши. Сессионные тикеты
подписаны и имеют срок действия; мутационные запросы используют CSRF-токены,
привязанные к тикету. Привилегии API-токена пересекаются с эффективными
распространёнными ACL владельца-принципала, поэтому токен не может эскалировать
права владельца. Логи редактируют распознанные представления тикетов, паролей и
токенов перед записью.
API-слой — адаптер. Он аутентифицирует, авторизует, валидирует по выбранному
контракту, диспетчеризует семантический обработчик и формирует
версионно-совместимый ответ. Маршрут без семантического обработчика явно
сообщается как неподдерживаемый, если оператор не включил нестандартный режим
fallback.
Зависимости направлены внутрь: HTTP- и CLI-адаптеры зависят от прикладных
сервисов; прикладные сервисы — от доменных интерфейсов; PostgreSQL, файлы
контрактов, метрики и часы реализуют эти интерфейсы. Доменные сервисы никогда не
импортируют FastAPI.
## Жизненный цикл запроса
1. Middleware назначает или проверяет request ID и запускает безопасную
структурированную телеметрию.
2. Выбранный профиль совместимости разрешает неизменяемый снимок API и
версионно-специфичное поведение.
3. Аутентификация определяет принципала без раскрытия учётных данных в логах.
4. Объявленные контрактом и специфичные для обработчика права проверяются до
раскрытия или изменения ресурсов.
5. Значения path, query и body валидируются схемами, полученными из контракта.
6. Семантический обработчик выполняется через прикладной сервис и явную границу
транзакции.
7. Долгие операции атомарно обновляют блокировку ресурса и создают
персистентную задачу, затем возвращают её UPID.
8. Рендерер ответа применяет Proxmox-обёртку, заголовки, cookies и
версионно-специфичные шаблоны ошибок.
## Персистентность и конкурентность
Используется `asyncpg` напрямую. Репозитории принимают явное соединение или
контекст транзакции; SQL параметризован и расположен рядом с репозиторием.
Изменяемые глобальные переменные процесса не являются авторитетным состоянием.
Воркеры захватывают задачи через `FOR UPDATE SKIP LOCKED`, устанавливают
продлеваемые аренды и используют метаданные идемпотентности для восстановления
после сбоя процесса. Состояние ресурса, блокировки ресурсов и создание задачи
изменяются в одной транзакции, когда это требуется. Оптимистичные колонки версии
обнаруживают конкурентные обновления, а ограничения БД защищают инварианты,
например уникальность VMID в пределах кластера.
Application lifespan владеет пулом соединений и ограниченным набором
asyncio-задач воркеров. При shutdown захват прекращается, выполняемая работа
достигает безопасной границы, отмена происходит только после настроенного grace
period, затем пул закрывается.
## Получение контракта и доверие
Сетевой доступ ограничен явными командами import и recorder. Импортёры
принудительно используют HTTPS, по умолчанию allowlist официальных хостов,
лимиты размера ответа и редиректов, таймауты и ограниченные повторы. Каждый
сырой артефакт неизменяем и имеет SHA-256 checksum. Его manifest фиксирует
происхождение, версию, предупреждения парсера и checksum нормализованного
снимка. Локальные снимки позволяют запуску и тестам работать офлайн.
Объявленная документация и санитизированное наблюдаемое поведение остаются
разделёнными. Профиль совместимости выбирает поведение `strict-docs`, `observed`
или `hybrid` без разброса проверок версий по сервисам.
## Модель безопасности
- Пароли и секреты API-токенов хранятся только как password hash.
- Тикеты подписаны, краткоживущие и редактируются в телеметрии.
- Мутации с ticket-аутентификацией требуют CSRF-валидации; запросы с API-токеном
CSRF не требуют.
- Интерактивный Web UI и вспомогательные `/admin/compatibility*` — лабораторные
поверхности без отдельного admin-токена в текущей сборке; границей доверия
является сетевая экспозиция.
- Контейнеры в упакованных образах работают от непривилегированного пользователя.
## Горячая замена контракта во время работы
При холодном старте загружается `CONTRACT_SNAPSHOT`. Операторы могут заменить
таблицу маршрутов в памяти для majors 6–9 через
`POST /ui/api/contract/apply?major=N` (также доступно в Web UI). Замена
обновляет `/version`, OpenAPI и состояние совместимости и действует только в
пределах процесса (перезапуск восстанавливает снимок из env).
## Наблюдаемость
JSON-логи содержат request ID, шаблон маршрута, статус, длительность и
редактированные поля идентичности. Процессные экспортёры Prometheus/OpenTelemetry
пока не поставляются; обработчики Proxmox `/cluster/metrics*` симулируют только
конфигурацию metrics-server PVE.
## Стратегия тестирования
Unit-тесты покрывают обработку контракта и доменные правила. Интеграционные
тесты проверяют репозитории, транзакции, воркеров и lifespan на PostgreSQL.
Наборы contract и compatibility нацелены на majors **69** с **100%** покрытием
реестра обработчиков. Внешний proxmoxer smoke выполняется против TLS-шлюза
Compose. Concurrency-тесты проверяют аренды задач и переходы состояния.
Готовность БД включает последнюю упакованную версию миграции, а не только
успешный connectivity-запрос. Воркеры повторяют неудачные захваты, пока не
появятся таблицы миграций. Нормализованные записи ресурсов используют
compare-and-swap обновления версии через типизированный репозиторий, поэтому
устаревшие писатели получают domain conflict.
## Модель развёртывания
На контейнер приходится один процесс Uvicorn. Горизонтальные реплики
координируются через PostgreSQL, а не через локальные очереди. Миграции БД и
операции seed — явные команды и в Kubernetes становятся отдельными job. PostgreSQL
включён в локальный Docker Compose, но в production chart — внешняя зависимость.
## Архитектурные решения
1. Маршруты FastAPI регистрируются из нормализованных снимков при старте;
сотни вручную поддерживаемых объявлений маршрутов не нужны.
2. SQLAlchemy не используется. Прямые asyncpg-репозитории делают поведение
транзакций и конкурентности явным.
3. Задачи на PostgreSQL — граница долговечности; фоновые задачи FastAPI и
in-memory очереди не используются для критичной работы.
4. Совместимость capability-driven и версионирована, а не реализована через
разбросанные условия по строкам версий.
5. Отсутствующие обработчики честно завершаются через `CONTRACT_FALLBACK`
(по умолчанию `error` → HTTP 501). Majors 69 поставляются с полной
регистрацией обработчиков, поэтому объявленные методы не должны попадать на
этот путь при нормальной работе.
6. Лабораторная документация и cookbooks живут в `docs/` и `examples/`; внутренние
research/prompt-заметки не входят в пользовательское руководство.
+85
View File
@@ -0,0 +1,85 @@
**Language / Язык:** [English](../authentication.md) | [Русский](authentication.md)
# Аутентификация
Симулятор реализует аутентификацию Proxmox-совместимыми тикетами и API-токенами
с проверкой ACL для не-root принципалов.
## Вход по тикету
```http
POST /api2/json/access/ticket
Content-Type: application/x-www-form-urlencoded
username=root@pam&password=secret
```
Успешный ответ включает:
- `ticket` — также устанавливается как HttpOnly cookie `PVEAuthCookie` (SameSite=Strict)
- `CSRFPreventionToken` — обязателен для мутаций с аутентификацией по тикету
- `username` и связанные поля идентичности
Тикеты подписываются HMAC с `TICKET_SIGNING_KEY`, по умолчанию истекают через два часа
и допускают небольшой сдвиг часов в будущее.
### Правила CSRF
| Запрос | Сессия по тикету | API-токен |
|---|---|---|
| `GET` / `HEAD` / `OPTIONS` | Достаточно cookie (или тикета) | Заголовок `Authorization` |
| Другие методы | Cookie **и** заголовок `CSRFPreventionToken` | CSRF **не** требуется |
```bash
curl -X POST \
-H "Cookie: PVEAuthCookie=$TICKET" \
-H "CSRFPreventionToken: $CSRF" \
-d '...' \
http://localhost:8006/api2/json/nodes/pve01/qemu/100/status/start
```
## API-токены
Формат заголовка:
```http
Authorization: PVEAPIToken=USER@REALM!TOKENID=SECRET
```
Секреты хранятся только как scrypt-хеши. Создание и явная регенерация возвращают
plaintext-секрет **один раз**; list и read его никогда не выводят. Удаление токена
немедленно его инвалидирует.
Привилегии токена — **пересечение** привилегий токена и эффективных (прямых +
унаследованных) ACL владельца. Токен не может эскалировать права выше владельца.
## Seeded development-принципалы
Сидятся **каждым** профилем — включая `minimal` и после demo unload в Web UI.
Unload уменьшает guests/nodes/storages; лабораторные принципалы и токены
`apply_seed` всё равно вставляет:
| Принципал | Пароль | Токен | Примечания |
|---|---|---|---|
| `root@pam` | `secret` | `automation` / `automation-secret` | Полный доступ по тикету; токен всё равно ограничен при ограниченных привилегиях |
| `auditor@pve` | `auditor-secret` | `readonly` / `readonly-secret` | Унаследованный auditor ACL — чтение OK, power ops запрещены |
| `operator@pve` | `operator@pve-password` | `operator` / `operator-secret` | VM audit/power на `/vms` |
| `storage@pve` | `storage@pve-password` | `storage` / `storage-secret` | Область datastore на `/storage` |
Эти учётные данные **только для лаборатории**. Смените или отключите их перед
выходом в сеть за пределы вашей рабочей станции.
## Root vs ACL
Root-сессии по тикету обходят обычные проверки ACL в Proxmox-совместимом смысле,
используемом этим симулятором. Отдельные API-токены остаются ограниченными. Тесты
совместимости проверяют разделение привилегий для персон auditor/operator/storage.
## Связанные пути
- Тикет: `/access/ticket`
- Пользователи / группы / роли / ACL / realm'ы / permissions
- Токены: `/access/users/{userid}/token[/{tokenid}]`
- TFA и OpenID: durable локальное состояние; **без** живых вызовов IdP
См. доменное руководство [Access](../domains/access.md).
+42
View File
@@ -0,0 +1,42 @@
**Language / Язык:** [English](../clients.md) | [Русский](clients.md)
# Клиенты
Используйте симулятор из обычных стеков автоматизации. Каждый cookbook стремится
к одному лабораторному сценарию, где инструмент это позволяет:
1. Аутентификация (ticket + CSRF **или** API token)
2. Чтение `version` / nodes / списка QEMU
3. Создание VM (принять UPID)
4. Опрос статуса задачи
5. Start / stop
6. Чтение статуса
7. Delete / cleanup
## Матрица подключений
Клиенты реального Proxmox VE ходят на **HTTPS `:8006`**. Compose в этой
лаборатории публикует plain **HTTP `:8006`** (тот же номер порта). HTTPS —
на **Kubernetes Ingress** (cert-manager). Клиенты без HTTP (proxmoxer):
`docker compose --profile tls``https://localhost:8443/` (см.
[Порты и TLS](configuration.md#порты-и-tls)).
| Стек | Транспорт Compose | Заметки | Docs | Code |
|---|---|---|---|---|
| Python (proxmoxer) | HTTPS `:8443` (`--profile tls`) | Только HTTPS; `verify_ssl=False` для lab cert | [руководство](examples/python-proxmoxer.md) | [`examples/python`](../../examples/python) |
| Python (requests) | HTTP `:8006` | Сырой `/api2/json` | [руководство](examples/python-requests.md) | [`examples/python`](../../examples/python) |
| Go | HTTP `:8006` | stdlib `net/http` | [руководство](examples/go.md) | [`examples/go`](../../examples/go) |
| Java | HTTP `:8006` | Java 11+ `HttpClient` | [руководство](examples/java.md) | [`examples/java`](../../examples/java) |
| Perl | HTTP `:8006` | `HTTP::Tiny` + JSON | [руководство](examples/perl.md) | [`examples/perl`](../../examples/perl) |
| Ansible | HTTP `:8006` | Cookbook модуля `uri` | [руководство](examples/ansible.md) | [`examples/ansible`](../../examples/ansible) |
| Terraform | HTTP `:8006` (или TLS `:8443`) | Предпочитайте HTTP; `insecure` только с `--profile tls` | [руководство](examples/terraform.md) | [`examples/terraform`](../../examples/terraform) |
| Pulumi | HTTP `:8006` | `pulumi-proxmoxve` или HTTP cookbooks | [руководство](examples/pulumi.md) | [`examples/pulumi`](../../examples/pulumi) |
В Kubernetes с Ingress + cert-manager направляйте клиентов на
`https://<ваш-хост>/`.
## Дальше
- Индекс cookbook: [examples/overview.md](examples/overview.md)
- Troubleshooting: [examples/troubleshooting-clients.md](examples/troubleshooting-clients.md)
- Полный Pulumi suite: [`pulumi-tests/`](../../pulumi-tests/README.ru.md)
+109
View File
@@ -0,0 +1,109 @@
**Language / Язык:** [English](../compatibility-0.1.0.md) | [Русский](compatibility-0.1.0.md)
# Отчёт о совместимости — 0.1.0
Этот отчёт фиксирует evidence для релиза симулятора 0.1.0 относительно bundled
контрактов Proxmox VE API (majors 69). Это матрица ограничений для измерений
*качества / внешней интеграции*, а не заявление общей совместимости с
гипервизором Proxmox. Покрытие реестра обработчиков относительно каждого
contract snapshot — **100%** для majors 6–9: у каждого объявленного метода есть
семантический обработчик.
Обзор для пользователя — в [compatibility.md](compatibility.md). Актуальные
machine-readable counts всегда доступны из `/admin/compatibility``.md` /
`.html`). Предпочитайте этот endpoint, когда симулятор запущен.
## Сводка (основной контракт PVE 9.2.3)
| Уровень | Методы | Доля контракта | Evidence |
|---|---:|---:|---|
| Declared and dynamically routed | 675 | 100% | Bundled API Viewer snapshot |
| Stateful semantics implemented | **675** | **100%** | Handler registry ∩ contract |
| Observed / verified surface ledger | **675** | **100%** | `evidence/pve-9.2.3.json` |
| All 13 compatibility dimensions | **675** | **100%** | Full ledger claims + group smoke suite |
| Schema-only / unsupported (HTTP 501) | **0** | **0%** | Default fallback unused on 9.2.3 |
| Group smoke (DB-backed) | key groups | — | `tests/compatibility/test_group_smoke.py` |
| proxmoxer smoke exercised | 9 | 1.33% | Unmodified proxmoxer 2.3 compatibility test |
Smoke set: `POST /access/ticket`, `GET /version`, `GET /nodes`,
`GET /nodes/{node}/qemu`, `GET /nodes/{node}/qemu/{vmid}/status/current`, одна из
двух state mutations (`start` или `stop`) и повторные
`GET /nodes/{node}/tasks/{upid}/status`. Обе мутации имеют независимые API- и
worker-тесты; один smoke run выбирает переход, допустимый для текущего состояния.
## Покрытие по Proxmox major
| Версия | Объявлено | Реализовано | Проверено | Покрытие |
|---|---:|---:|---:|---:|
| 6.4-15 | 504 | 504 | 504 | 100.00% |
| 7.4-16 | 540 | 540 | 540 | 100.00% |
| 8.4.5 | 605 | 605 | 605 | 100.00% |
| 9.2.3 | 675 | 675 | 675 | 100.00% |
**Verified** здесь означает, что каждый объявленный метод присутствует в
per-major surface ledger (`evidence/pve-{version}.json`), перегенерируемом через
`make evidence` и охраняемом `tests/compatibility/test_verified_surface.py`.
Hot-swap (`POST /ui/api/contract/apply?major=N`) загружает ledger этого major,
поэтому Help → Compatibility показывает полные observed/verified counts после
Apply.
Каждая запись ledger заявляет все тринадцать измерений, поэтому
`fully_compatible` совпадает с declared после Apply. Group smoke
(`tests/compatibility/test_group_smoke.py`) проверяет репрезентативные
мутации с PostgreSQL для access, QEMU, LXC, storage, notifications, SDN и node
DNS/network.
Старые majors переиспользуют обработчики 9.2.3 плюс path synonyms из
`app/handlers/legacy_aliases.py` (`ceph/pools``ceph/pool`,
`backupinfo``backup-info`, `scan/glusterfs`, legacy TFA collection verbs и
т. д.).
## Реализованная поверхность (высокий уровень)
- **Core**: version, ticket login, node list/status/index, cluster resources.
- **Access**: users, groups, roles, ACL, password, tokens, realms, TFA, OpenID,
permissions, VNC ticket — всё durable в PostgreSQL.
- **QEMU / LXC**: полные contract surfaces, включая agent, cloud-init, consoles,
RRD, firewall aliases/ipset, migrate/clone/snapshot subsets.
- **Storage / pools / backup / HA / firewall / Ceph / SDN**: durable handlers
(`clusters.metadata`, `nodes.metadata.ops`, normalized tables).
- **Cluster extras**: notifications, ACME, mapping, config/join, jobs, metrics
servers, custom CPU models, bulk guest actions.
- **Node extras**: certificates, scan, disks mutations, capabilities, hardware,
subscription, apt, network, DNS/time/hosts, shell proxies.
- **Tasks**: leased workers, status, append-only logs.
- **Auth**: ticket + CSRF для mutations; hashed API tokens.
## Принцип персистентности
Каждый create/update/delete path записывает в PostgreSQL (таблицы и/или jsonb
metadata). Секреты могут храниться, но не должны возвращаться в GET.
Пользовательские ошибки «not supported in the emulator» запрещены — см.
`.cursor/rules/durable-simulator.mdc`.
## Известные ограничения
| Область | Текущее поведение |
|---|---|
| External systems | LDAP/OpenID/ACME/Ceph не обращаются к реальным удалённым системам; состояние симулируется |
| Realm sync / OpenID login | Durable stamps / pending state / tickets; нет live IdP |
| Observation parity | Contract/tests существуют; санитизированный real-PVE observation corpus ограничен |
| TLS | Локальный nginx gateway только с checked-in self-signed development key |
| Client certification | proxmoxer 2.3 smoke; Terraform и другие клиенты не сертифицированы |
| Deep HTTP coverage | Не каждый из 675 методов прогоняется end-to-end; group smokes покрывают репрезентативные paths по доменам |
Полное покрытие реестра означает, что HTTP 501 «handler pending» больше не
должен появляться для методов, объявленных в активном контракте после Apply.
*Качество* совместимости (точный parity edge-case Proxmox) по-прежнему углубляется
тестами и observation.
При импорте новой версии контракта Proxmox: обновите bundled snapshot, выполните
`make evidence`, запустите `pytest tests/compatibility/test_verified_surface.py`
и закоммитьте обновлённые ledger `evidence/pve-*.json`.
Отчёт также раскрывает 13 независимых измерений совместимости, требуемых project
brief. Surface ledgers живут в `evidence/pve-{version}.json`; исторический deep
overlay `evidence/pve-9.2.3-0.1.0.json` сливается в canon 9.2.3 при
перегенерации. Сама динамическая регистрация маршрутов доказывает измерение
route/method; это не означает полную семантическую совместимость для каждого
edge case.
+78
View File
@@ -0,0 +1,78 @@
**Language / Язык:** [English](../compatibility.md) | [Русский](compatibility.md)
# Совместимость
Этот документ объясняет, как симулятор заявляет совместимость с Proxmox VE API
majors **69**. Предпочитайте live-отчёты, когда процесс запущен.
## Live-отчёты
| URL | Формат |
|---|---|
| `/admin/compatibility` | JSON |
| `/admin/compatibility.md` | Markdown |
| `/admin/compatibility.html` | HTML |
Web UI также показывает панель совместимости через `/ui/api/compatibility?major=N`.
## Реестр и проверенное покрытие поверхности
| Версия | Объявлено | Реализовано | Проверено | Покрытие |
|---|---:|---:|---:|---:|
| 6.4-15 | 504 | 504 | 504 | 100% |
| 7.4-16 | 540 | 540 | 540 | 100% |
| 8.4.5 | 605 | 605 | 605 | 100% |
| 9.2.3 | 675 | 675 | 675 | 100% |
Старые majors сопоставляют legacy path synonyms через `legacy_aliases` с общим
набором обработчиков.
- **Implemented** — зарегистрирован семантический обработчик.
- **Verified / observed** — каждый объявленный метод перечислен в
`evidence/pve-{version}.json` (surface ledger). Перегенерируйте через
`make evidence`. Охраняется `tests/compatibility/test_verified_surface.py`.
После **Apply as runtime** (`POST /ui/api/contract/apply?major=N`) live-отчёт
загружает ledger этого major, поэтому Help → Compatibility показывает полные
verified counts.
## Измерения evidence
Оценка совместимости использует тринадцать независимых измерений (routing,
input shape, HTTP status, JSON structure, state semantics, long tasks,
permissions, …). Ledger по majors в `evidence/pve-{version}.json` в настоящее
время заявляют **все тринадцать измерений для каждого объявленного метода**
(перегенерируются через `make evidence`), поэтому Help → Compatibility
Dimensions показывает 100% после Apply.
Исполняемая основа этих заявлений:
| Набор | Роль |
|---|---|
| `tests/compatibility/test_verified_surface.py` | hot-swap + ledger drift + score gates |
| `tests/compatibility/test_group_smoke.py` | access / qemu / lxc / storage / cluster / SDN / node ops with PostgreSQL |
| `tests/compatibility/test_proxmoxer.py` | external proxmoxer HTTPS smoke |
Историческое богатое происхождение из `evidence/pve-9.2.3-0.1.0.json` по-прежнему
сливается в `sources` ledger 9.2.3 при перегенерации.
## Внешний client smoke
`make test-compatibility` запускает неизменённый поток **proxmoxer 2.3** против
Compose TLS gateway (`PROXMOXER_HOST` / `PROXMOXER_PORT`). Проверяются login,
reads, CSRF-protected mutation, token/ACL behaviour и завершение UPID.
Дополнительные cookbooks в [`examples/`](../../examples/README.ru.md) — manual или
CI-optional в зависимости от стека.
## Известные поведенческие ограничения
| Область | Поведение |
|---|---|
| External systems | LDAP / OpenID / ACME / Ceph не обращаются к реальным удалённым системам |
| TLS | Только локальный self-signed development gateway |
| Hypervisor | Нет реального выполнения KVM/LXC |
| Observation corpus | Санитизированные данные наблюдений real-PVE остаются ограниченными |
Исторические release notes:
[compatibility-0.1.0.md](compatibility-0.1.0.md).
+107
View File
@@ -0,0 +1,107 @@
**Language / Язык:** [English](../configuration.md) | [Русский](configuration.md)
# Конфигурация
Настройки приложения загружаются из окружения (см. `.env.example`).
Docker Compose подставляет многие из них для сервиса `simulator`; значения,
объявленные в `environment:` в `docker-compose.yml`, переопределяют `.env` для этого
сервиса.
## Основные
| Переменная | По умолчанию / пример | Назначение |
|---|---|---|
| `APP_HOST` | `0.0.0.0` | Адрес привязки |
| `APP_PORT` | `8006` | HTTP-порт прослушивания |
| `DATABASE_URL` | `postgresql://proxmox:proxmox@postgres:5432/proxmox_simulator` | asyncpg DSN |
| `DB_POOL_MIN_SIZE` | `1` | Минимум пула |
| `DB_POOL_MAX_SIZE` | `10` | Максимум пула |
| `DB_CONNECT_TIMEOUT_SECONDS` | `10` | Таймаут подключения |
| `DB_COMMAND_TIMEOUT_SECONDS` | `30` | Таймаут команды |
| `LOG_LEVEL` | `INFO` | Уровень логирования |
| `REQUEST_ID_HEADER` | `X-Request-ID` | Заголовок корреляции запросов |
## Контракт и каталог
| Переменная | Назначение |
|---|---|
| `CONTRACT_SNAPSHOT` | Путь к нормализованному снимку, загружаемому при **холодном старте** |
| `CONTRACT_FALLBACK` | `error` (по умолчанию), `schema-default` или `fixture` — поведение для методов **без** семантического обработчика |
| `COMPATIBILITY_EVIDENCE` | Необязательный evidence JSON для отчётов совместимости |
| `CATALOG_ARTIFACT_URL_6``_9` | Официальные URL API Viewer при импорте/кэшировании мажоров каталога |
Горячая замена в runtime (Web UI / `POST /ui/api/contract/apply`) заменяет таблицу
маршрутов в памяти для мажоров **69** без перезаписи `CONTRACT_SNAPSHOT`. Перезапуск
процесса восстанавливает снимок холодного старта. См. [Версии API](api-versions.md).
При **100%** покрытии обработчиков на мажорах 6–9 `CONTRACT_FALLBACK` не используется
для объявленных методов активного контракта. В production-подобных лабораториях
оставляйте `error`, чтобы любой случайный пробел проявлялся как HTTP 501.
## Безопасность и задачи
| Переменная | Назначение |
|---|---|
| `TICKET_SIGNING_KEY` | HMAC-ключ для тикетов и CSRF-токенов, привязанных к тикету (**меняйте вне игрушечных лабораторий**) |
| `TASK_WORKER_CONCURRENCY` | Число asyncio workers с арендой (132) |
| `TASK_LEASE_SECONDS` | Длительность аренды задачи в PostgreSQL |
| `SIMULATION_TIME_SCALE` | Ускоряет симулируемые длительности задач |
## Seed и хуки клиентских тестов
| Переменная | Назначение |
|---|---|
| `SEED_PROFILE` | Имя профиля для seed CLI (`small`, `medium`, …) |
| `SEED_LARGE_NODES` | Число узлов для `large` |
| `SEED_LARGE_RESOURCES` | Число гостей для `large` (по умолчанию 10 000) |
| `TEST_DATABASE_URL` | DSN для интеграционных тестов |
| `PROXMOXER_HOST` / `PROXMOXER_PORT` | Цель клиента совместимости (`tls-gateway` / `8443` в Compose) |
## Порты и TLS
### Реальный Proxmox VE (справочно)
На физическом / production-узле PVE management API слушает **HTTPS `:8006`**
(`/api2/json/...`). Связанные management-порты (это не отдельные REST API):
| Порт | Протокол | Назначение |
|---|---|---|
| `8006` | TCP, HTTPS | Web UI + REST API |
| `3128` | TCP | SPICE proxy (графическая консоль) |
| `59005999` | TCP (WebSocket) | VNC web-консоль |
| `22` | TCP | SSH / кластерные операции |
| `54055412` | UDP | Трафик Corosync |
Порт **`8007`** — **не** API PVE: обычно это management-порт Proxmox Backup
Server (PBS). Не направляйте PVE-клиентов на `:8007` на реальном железе.
### Эндпоинты лабораторного симулятора
| Эндпоинт | Использование |
|---|---|
| `http://localhost:8006` | Основной URL клиентов — nginx TLS-шлюз → симулятор (curl, браузеры, proxmoxer, Terraform, …) |
Сам процесс симулятора говорит по **HTTP на `:8006` внутри Docker-сети**. Compose
публикует self-signed HTTPS-фронт на хосте **`:8006`** (тот же порт, что у
реального PVE), чтобы неизменённые TLS-клиенты вели себя как против production
(`https://host:8006/api2/json/...`). Внутри Compose шлюз слушает `8443` и
проксирует на `simulator:8006`. Хост **`:8007` больше не используется** для
лабораторного API (на реальном железе этот порт обычно PBS, не PVE).
Встроенный сертификат в `docker/tls/` — одноразовый материал для разработки.
Никогда не используйте его вне локальных лабораторий. См. [Безопасность](security.md).
## Заметки по Compose
- `migrate` выполняется один раз; `simulator` ждёт успешного migrate.
- Development Compose монтирует репозиторий и включает Uvicorn reload.
- В Compose по умолчанию `CONTRACT_SNAPSHOT` закрепляет встроенную ревизию PVE **9.2.3**
для холодного старта.
## Открытые и неиспользуемые ключи в примере
`.env.example` может по-прежнему перечислять ключи вроде `PVE_API_VERSION`,
`SIMULATION_SEED`, `SIMULATOR_ADMIN_ENABLED` и `SIMULATOR_ADMIN_TOKEN`, которые
**не** потребляются текущей моделью настроек. Для мажорной версии по умолчанию
используйте `CONTRACT_SNAPSHOT`, для runtime-переключений — Web UI / apply API. Не
предполагайте, что сегодня существует аутентифицированный admin API `/_simulator`.
+28
View File
@@ -0,0 +1,28 @@
**Language / Язык:** [English](../../domains/README.md) | [Русский](README.md)
# Руководства по доменам
Эти страницы описывают устойчивую семантику по областям API. Для исчерпывающих
списков методов используйте каталог Web UI или OpenAPI (`/docs`) для активной
major-версии — заявленное покрытие составляет **100%** для PVE 69.
| Руководство | Темы |
|---|---|
| [Core и кластер](core-cluster.md) | version, nodes, cluster resources/options/status |
| [Access](access.md) | users, groups, roles, ACL, realms, tokens, TFA, OpenID |
| [QEMU](qemu.md) | guests, power, disks, snapshots, clone/migrate, agent |
| [LXC](lxc.md) | containers and parallel lifecycle operations |
| [Storage и backup](storage-backup.md) | storages, content, vzdump / backup jobs |
| [Firewall](firewall.md) | cluster / node / guest firewall objects |
| [HA](ha.md) | groups, resources, status |
| [Ceph](ceph.md) | simulated Ceph configuration and status |
| [Pools](pools.md) | pools and membership |
| [SDN](sdn.md) | zones, VNets, subnets, controllers, IPAM |
| [Cluster extras](cluster-extras.md) | notifications, ACME, mapping, metrics servers |
| [Tasks](tasks.md) | UPID workers, status, logs |
## Карта персистентности
- Guests / HA / storage / identity → нормализованные таблицы
- Свободная конфигурация кластера → `clusters.metadata` jsonb
- Операции на уровне узла (network, disks, apt, …) → `nodes.metadata` под ключом `ops`
+21
View File
@@ -0,0 +1,21 @@
**Language / Язык:** [English](../../domains/access.md) | [Русский](access.md)
# Access
Устойчивая идентификация и авторизация: users, groups, roles, ACL entries,
realms, passwords, API tokens, permissions queries, tickets, TFA, OpenID,
VNC tickets.
## Основное
- Ticket login и CSRF — см. [Authentication](../authentication.md).
- При создании token секрет возвращается один раз; в хранилище сохраняются только
хеши.
- Наследование ACL и пересечение привилегий token ∩ owner.
- Состояние realm / TFA / OpenID **локальное**; живые вызовы каталога или IdP не
выполняются.
## Предзаполненные персоны
`root@pam`, `auditor@pve`, `operator@pve`, `storage@pve` — пароли и tokens см. в
руководстве по authentication.
+9
View File
@@ -0,0 +1,9 @@
**Language / Язык:** [English](../../domains/ceph.md) | [Русский](ceph.md)
# Ceph
Пути API, связанные с Ceph, сохраняют симулированное состояние кластера, pool,
OSD и monitor. Они не обращаются к живому кластеру Ceph.
Устаревшие алиасы путей (например, исторические написания `ceph/pools`) мапятся
на общие handlers, чтобы старые major-версии оставались полностью маршрутизируемыми.
+13
View File
@@ -0,0 +1,13 @@
**Language / Язык:** [English](../../domains/cluster-extras.md) | [Русский](cluster-extras.md)
# Cluster extras
Дополнительные домены на уровне кластера с устойчивыми handlers:
- **Notifications** — состояние конфигурации endpoints и targets
- **ACME** — симуляция account/plugin/certificate (без реальной регистрации в CA)
- **Mapping** — PCI / USB / resource mappings
- **Metrics servers** — симуляция конфигурации и экспорта PVE metrics-server
- **Custom CPU models** и массовые guest actions — как заявлено в контракте
Точные пути для активной major-версии смотрите в каталоге Web UI.
+25
View File
@@ -0,0 +1,25 @@
**Language / Язык:** [English](../../domains/core-cluster.md) | [Русский](core-cluster.md)
# Core и кластер
## Version
`GET /version` отражает `source_version` **активного** контракта (cold-start
snapshot или hot-swapped major).
## Nodes
- Endpoints списка и статуса устойчивы и формируются из seeded / созданных nodes.
- Имя node по умолчанию в seed-профиле `small`: **`pve01`**.
- Операционные мутации node (network, apt, disks, services, DNS/time/hosts,
certificates, …) сохраняются в `nodes.metadata.ops`.
## Cluster
- `/cluster/resources` и связанные inventory views читают guests и storages из
PostgreSQL.
- Cluster options, status, tasks, logs, replication, config/join helpers
сохраняют cluster metadata и связанные таблицы.
Работает для всех заявленных методов на major 6–9 для этих путей. Используйте
каталог Web UI, чтобы проверить различия параметров между версиями.
+10
View File
@@ -0,0 +1,10 @@
**Language / Язык:** [English](../../domains/firewall.md) | [Русский](firewall.md)
# Firewall
Конфигурация firewall на уровне cluster, node и guest — rules, aliases, IP sets,
security groups — в основном сохраняется через cluster/node metadata и связанные
структуры.
Handlers покрывают заявленную firewall-поверхность для major 6–9. Примените
нужную major-версию перед проверкой имён полей, специфичных для версии.
+10
View File
@@ -0,0 +1,10 @@
**Language / Язык:** [English](../../domains/ha.md) | [Русский](ha.md)
# HA
High-availability groups, resources, status и rules сохраняются в cluster
metadata / таблицах HA.
Используйте профиль `ha-demo` (medium + HA resource для VM 100) или demo cluster
для более богатых fixtures. HA здесь оркестрирует **симулированное** состояние
размещения guest — реальные nodes не изолируются (fencing не выполняется).
+15
View File
@@ -0,0 +1,15 @@
**Language / Язык:** [English](../../domains/lxc.md) | [Русский](lxc.md)
# LXC
Container API повторяют паттерны жизненного цикла QEMU там, где это заявлено
контрактом: CRUD, power, clone/migrate, snapshots, volume operations, consoles,
RRD и firewall objects.
Мутации сохраняются в нормализованные container tables и связанные metadata.
Асинхронные пути возвращают UPID по той же модели leased-worker, что и QEMU.
Seed-профили:
- `small` — CT `200` на `pve01`
- `medium` / `large` / `demo-cluster` — множество containers
+6
View File
@@ -0,0 +1,6 @@
**Language / Язык:** [English](../../domains/pools.md) | [Русский](pools.md)
# Pools
Pool CRUD и membership ресурсов полностью покрыты и устойчивы. Seed `medium`
включает development pool для экспериментов с membership.
+19
View File
@@ -0,0 +1,19 @@
**Language / Язык:** [English](../../domains/qemu.md) | [Русский](qemu.md)
# QEMU
Полная contract-поверхность для QEMU guests на активной major, включая:
- Create / sync & async config update / delete (UPID для async)
- Power: start, stop, shutdown, reboot, reset, suspend, resume
- Явная state machine + per-VM PostgreSQL lock
- Snapshots (create/delete/rollback как tasks)
- Clone и local migration (UPID)
- Disk resize (sync; shrink отклоняется) и disk move (task)
- Pending config view
- Guest agent read-only subset (info, OS/hostname, network, time, ping) при
`agent=1` и запущенном guest
- Cloud-init, consoles, RRD, guest firewall objects — как заявлено в контракте
Индексированные поля контракта, такие как `scsi[n]`, принимают конкретные имена
(`scsi0`, …). Неизвестные version-dependent parameters сохраняются в JSONB.
+10
View File
@@ -0,0 +1,10 @@
**Language / Язык:** [English](../../domains/sdn.md) | [Русский](sdn.md)
# SDN
Handlers software-defined networking покрывают заявленные zones, VNets, subnets,
controllers, IPAM, DNS, fabrics, locks и связанные dry-run/rollback операции для
активной major.
Состояние локально в базе симулятора. Переключение major 6–9 меняет набор SDN
methods на wire; все заявленные реализованы.
+18
View File
@@ -0,0 +1,18 @@
**Language / Язык:** [English](../../domains/storage-backup.md) | [Русский](storage-backup.md)
# Storage и backup
## Storage
- Cluster и node storage inventories сохраняются в нормализованных storage tables.
- Content listings и мутации обновляют `storage_contents` (и связанные строки).
- Seed `broken-storage` помечает `local-lvm` недоступным для тестирования сбоев.
## Backup
- Backup jobs, metadata и task-пути в стиле `vzdump` создают устойчивые task rows
и backup records.
- Workers выполняют leased backup tasks аналогично guest operations.
Реальные удалённые backup targets не вызываются; состояние объектов остаётся
внутри PostgreSQL.
+25
View File
@@ -0,0 +1,25 @@
**Language / Язык:** [English](../../domains/tasks.md) | [Русский](tasks.md)
# Tasks
Долгие операции возвращают **UPID** в стиле Proxmox. Task rows, events,
опциональные resource locks и idempotency metadata фиксируются вместе.
## Паттерн для клиента
1. `POST`/`DELETE` mutation → прочитать UPID из `data`
2. Опрашивать `GET /nodes/{node}/tasks/{upid}/status` до завершения
3. При необходимости запросить `.../log`
## Workers
- Claim через `FOR UPDATE SKIP LOCKED`
- Возобновляемые leases (`TASK_LEASE_SECONDS`)
- Progress + append-only logs
- Recovery после сбоя процесса
Длительность симуляции учитывает `SIMULATION_TIME_SCALE`. Безопасность lease
worker использует wall-clock time, чтобы ускоренный сценарий не нарушал
семантику распределённого claim.
См. [API surface](../api-surface.md) и [Operations](../operations.md).
+14
View File
@@ -0,0 +1,14 @@
**Language / Язык:** [English](../../examples/ansible.md) | [Русский](ansible.md)
# Ansible
Playbook использует модуль `uri` для HTTP `:8006` с аутентификацией по токену, затем
ticket+CSRF для пути мутации.
```bash
cd examples/ansible
ansible-playbook -i inventory.ini playbook.yml
```
Перед использованием фиксированных VMID из предыдущего запуска выполните повторный seed
симулятора.

Some files were not shown because too many files have changed in this diff Show More